How can I run a DAG in another python version?

Keywords: Apache Airflow - Microsoft Azure - How to - Other

Description:
Currently using the Multi-tier Apache Airflow image and it’s running python 3.8.

I need to run a DAG using python 3.6 (doesn’t work on python 3.8). What are the steps to install python 3.6 and run this DAG (and this DAG only) on python 3.6?

Hi @ssanchez,

We do not have any guide to install a different Python version in a running Bitnami Airflow Multi-tier solution. Please note that we build the components from scratch and install them inside the /opt/bitnami folder. You can try to use apt to install a different version of Python system-wide. However, the Bitnami solution will continue using the installed one inside /opt/bitnami. If that DAG runs as a separate process, you will probably be able to configure it to use the “new” Python binary instead of the one inside the Bitnami’s installation directory.

We suggest you ask in the Apache Airflow forums to know if there is any other alternative to configure what Python binary to use to run your DAG.

Thanks

I can install a new python binary now but Airflow cannot “see” the new environment I created.

I can create for example a virtual environment in /opt/bitnami/airflow/venv36

but Airflow won’t see it. even a simple BashOperator running ls /opt/bitnami/airflow/venv36 will fail.

How can I make this directory “visible” to airflow?

Hi @ssanchez,

That’s something related to the application so we suggest you ask the Airflow’s support team about it.

As I mentioned previously, we do not have a guide to configure the application the way you want. You should ask the Airflow’s team to learn if that’s possible. Apart from that, we suggest you check the startup scripts for Airflow you can find inside the /opt/bitnami/scripts/airflow folder.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.