Channels
announcements
blockers_for_windward
career-opportunities
celebrations
cuddle-corner
data-discussion
data-events
dev
events
general
help
iceberg-integration
lakefs-for-beginners
lakefs-hubspot-cloud-registration-email-automation
lakefs-releases
lakefs-suggestions
lakefs-twitter
linen-dev
memes-and-banter
say-hello
stackoverflow
test
Title
e
Edmondo Porcu
05/01/2023, 7:44 PMHow does LakeFS work with data catalogs? Are there reference material about it?
i
Iddo Avneri
05/01/2023, 8:17 PMHi @Edmondo Porcu, long time! Which one did you have in mind?
I think you might have been looking into Unity in the past?
e
Edmondo Porcu
05/01/2023, 8:19 PMYes, we are about to replace some core pipelines with newer, well written ones. It was more urgent than working on the underlying infrastructure to succeed with data at scale, but we are now starting to have some time to address the problem. We are considering Amundsen, DataHub, Atlas, Open Metadata, and we are still in the process of looking how we could leverage LakeFS to ensure versioning of parts of the data lake…
i
Iddo Avneri
05/01/2023, 8:22 PMI’m not 100% sure on which ones are the easiest to integrate with. Let me circle this with a few folks and get back to you.
s
Shirshanka Das
05/03/2023, 3:45 AMCurious about this as well! DataHub has good integration with Iceberg, Delta etc, but we haven't yet gotten a contribution around LakeFS. We like versioned things 🙂 so definitely happy to help in making this integration happen. @Edmondo Porcu: what sort of discovery experience did you have in mind?
:jumping-lakefs: 2