Channels
announcements
blockers_for_windward
career-opportunities
celebrations
cuddle-corner
data-discussion
data-events
dev
events
general
help
iceberg-integration
lakefs-for-beginners
lakefs-hubspot-cloud-registration-email-automation
lakefs-releases
lakefs-suggestions
lakefs-twitter
linen-dev
memes-and-banter
say-hello
stackoverflow
test
Title
v
Vaibhav Kumar
03/16/2023, 5:07 PMHi Team
Do we have any plans to include anything data quality in LakeFS?
b
Barak Amar
03/16/2023, 5:13 PMSupport long-running quality checks
:jiggling-lakefs: 1
i
Iddo Avneri
03/16/2023, 5:15 PM@Vaibhav Kumar - what did you have in mind?
Can you share a bit more details?
The reason I’m asking is because today we have examples of :
1. Schema validation with pre-merge hooks
2. Apache airflow integrations to run DAGs the check quality of ingested data before promoting to production.
Maybe one of these can be helpful
v
Vaibhav Kumar
03/16/2023, 5:37 PMSomething like this https://griffin.apache.org/ or data contracts? @Iddo Avneri