https://lakefs.io/ logo
Title
v

Vaibhav Kumar

02/19/2023, 3:24 PM
Hi Team, I am working on an issue. In the github Provider are we not creating Airflow for testing the action ? In the below code where will this DAG run? I can’t see airflow env creation in the above provider.yaml?
- name: Run lakeFS DAG
run: |
cd astro
astro dev run connections add conn_lakefs --conn-type=HTTP --conn-host=<http://172.17.0.1:8000> --conn-extra='{"access_key_id":"${{ env.KEY }}","secret_access_key":"${{ env.SECRET }}"}'
astro dev run dags unpause lakeFS_workflow
astro dev run dags trigger lakeFS_workflow
sleep 30
a

Ariel Shaqed (Scolnicov)

02/19/2023, 3:29 PM
Hi @Vaibhav Kumar! I think that this line starts an Airflow instance. Does that help? If you post a PR, I will read it and run tests on it!
:dancing_lakefs: 1
v

Vaibhav Kumar

02/19/2023, 3:34 PM
ok , so astro comes up with default airflow?
a

Ariel Shaqed (Scolnicov)

02/19/2023, 3:50 PM
Yeah, AFAIU it's some weird development wrapper that Astronomer provide. It brings up Airflow in some bunch of containers.
I think this essentially implements these instructions from the Astronomer website.
v

Vaibhav Kumar

02/20/2023, 12:56 PM
@Ariel Shaqed (Scolnicov) I have raised the below PR https://github.com/treeverse/airflow-provider-lakeFS/pull/48 Please review it
🙏 1
:jumping-lakefs: 1
@Ariel Shaqed (Scolnicov) I have made all the changes requested by you. Kindly review it.