Occasionally, when we are making a custom Airflow operator, we may end up in a situation when Airflow complains about not existing jinja template. It may happen when we have an Airflow operator, which calls another operator inside its execute function when both operators have a common template field.
Table of Contents
In this situation, Airflow may raise an error saying that: jinja2.exception.TemplateNotFound: content_of_the_field. It is strange because if we have not used a jinja template inside the field value, we may expect it to pass the string without using it as a template. Unfortunately, that does not happen.
Dealing with the issue is quite simple. The wrapping Airflow operator should include the field’s content as a jinja constant and pass it into the inner operator. For example, when our code looks like this:
class SomeOperator(BaseOperator):
template_fields = ('the_field')
def __init__(self, field, *args, **kwargs):
super().__init__(*args, **kwargs)
self.field = field
self.extra_args = args
self.extra_kwargs = kwargs
def execute(self, context):
self.another_operator = SomeOtherOperator(field=self.field, *self.extra_args, **self.extra_kwargs)
self.another_operator.execute(context)
Get Weekly AI Implementation Insights
Join engineering leaders who receive my analysis of common AI production failures and how to prevent them. No fluff, just actionable techniques.
We can fix the jinja template issue like this:
class SomeOperator(BaseOperator):
template_fields = ('the_field')
def __init__(self, field, *args, **kwargs):
super().__init__(*args, **kwargs)
self.field = "{{ '" + field + "' }}"
self.extra_args = args
self.extra_kwargs = kwargs
... rest of the code