This article will show you how to get a notification when an Athena table is updated. For example, when someone writes a file to the S3 location that is used as the source of the Athena table.
To get the notification, I will implement an AWS Lambda function that gets triggered by a CloudWatch event. Here is the Serverless configuration that triggers the function:
events:
- cloudwatchEvent:
event:
source:
- "aws.glue"
detail-type:
- "Glue Data Catalog Database State Change"
detail:
typeOfChange:
- "updateTable"
In the Python function, I have to extract the event details to get the database and the table name:
def main(event, context):
database_name = event['requestParameters']['databaseName']
table_name = event['requestParameters']['tableInput']['name']
number_of_records = event['requestParameters']['tableInput']['parameters']['recordCount']
update_time = event['eventTime']
# here you can store the last update time in DynamoDB, send a notification to a Slack channel, or do whatever you want
Want to build AI systems that actually work?
Download my expert-crafted GenAI Transformation Guide for Data Teams and discover how to properly measure AI performance, set up guardrails, and continuously improve your AI solutions like the pros.