When we want to get an XCom variable in the on_failure_callback, we will face a nasty bug. Using the ti key to retrieve a value from the context gives us a None value.

# This does not work
context.get("ti").xcom_pull(key="test")

It happens because in the on_failure_callback the task instance is passed to the function using the task_instance key. Therefore to get a value from XCom, we must execute this code:

task: TaskInstance = context.get('task_instance')
task.xcom_pull(key="test")

Alternatively, we can import XCom and access it directly:

from airflow.models import XCom
XCom.get_one(execution_date = context.get('execution_date'), key='test')
Stop AI Hallucinations Before They Cost You.
Join engineering leaders getting weekly tactics to prevent failure in customer-facing AI systems. Straight from real production deployments.
Stop AI Hallucinations Before They Cost You.
Join engineering leaders getting weekly tactics to prevent failure in customer-facing AI systems. Straight from real production deployments.
Older post

Add the row insertion time to a MySQL table

Automatically add the insertion and update time in MySQL

Newer post

How to get a notification when a new file is uploaded to an S3 bucket

Get a Slack notification when a file is uploaded to an S3 bucket

Engineering leaders: Is your AI failing in production? Take the 10-minute assessment
>
×
Stop AI Hallucinations Before They Cost You.
Join engineering leaders getting weekly tactics to prevent failure in customer-facing AI systems. Straight from real production deployments.