Hi Team
Do we have any plans to include anything d...
# devv
Vaibhav Kumar
03/16/2023, 5:07 PMHi Team
Do we have any plans to include anything data quality in LakeFS?
b
Barak Amar
03/16/2023, 5:13 PM
Barak Amar
03/16/2023, 5:14 PM
Support long-running quality checks
jiggling lakefs 1
i
Iddo Avneri
03/16/2023, 5:15 PM@Vaibhav Kumar - what did you have in mind?
Can you share a bit more details?
The reason I’m asking is because today we have examples of :
1. Schema validation with pre-merge hooks
2. Apache airflow integrations to run DAGs the check quality of ingested data before promoting to production.
Maybe one of these can be helpful
v
Vaibhav Kumar
03/16/2023, 5:37 PMSomething like this https://griffin.apache.org/ or data contracts? @Iddo Avneri
4 Views