Making OFFSET LIMIT queries in AWS Athena

There is no OFFSET support in AWS Athena, but we can use a workaround to get the same behavior.

Let’s assume that I want to execute this query:

SELECT * FROM some_table
ORDER BY column_A
OFFSET 20 LIMIT 10 -- this doesn't work

to get the desired outcome, I need three things:

  • a window function that assigns a number to every row
  • ordering method that sorts the table, so I get a deterministic outcome
  • filtering function that selects only the rows I want
    SELECT row_number() over(ORDER BY column_A) AS rn, * FROM some_table)
WHERE rn BETWEEN 20 AND 30; -- 30 = 20 (offset) + 10 (limit)
Older post

How to get an alert if an AWS lambda does not get invoked during the last 24 hours

How to get a notification when AWS Lambda stops begin used

Newer post

What is s3:TestEvent, and why does it break my event processing?

S3 sends s3:TestEvent to SQS after setting up the bucket notifications