https://lakefs.io/ logo
Title
v

Vaibhav Kumar

03/16/2023, 5:07 PM
Hi Team Do we have any plans to include anything data quality in LakeFS?
b

Barak Amar

03/16/2023, 5:13 PM
Support long-running quality checks
:jiggling-lakefs: 1
i

Iddo Avneri

03/16/2023, 5:15 PM
@Vaibhav Kumar - what did you have in mind? Can you share a bit more details? The reason I’m asking is because today we have examples of : 1. Schema validation with pre-merge hooks 2. Apache airflow integrations to run DAGs the check quality of ingested data before promoting to production. Maybe one of these can be helpful
v

Vaibhav Kumar

03/16/2023, 5:37 PM
Something like this https://griffin.apache.org/ or data contracts? @Iddo Avneri