Title
#general
r

Ranjeetha Raja

10/18/2022, 3:19 PM
Hello LakeFS community, is there a plan to enable support for open table formats like Iceberg, Delta Lake etc? If yes, what is the tentative ETA?
Ariel Shaqed (Scolnicov)

Ariel Shaqed (Scolnicov)

10/18/2022, 3:37 PM
Hi @Ranjeetha Raja, Welcome to the lake! Both of these are on our roadmap, and Iceberg is "high priority". You can get some support for Delta even just by using the s3 gateway - obviously we need to improve that too.
r

Ranjeetha Raja

10/18/2022, 3:49 PM
thank you Ariel for the prompt response. Any ETA for the Iceberg support?
Ariel Shaqed (Scolnicov)

Ariel Shaqed (Scolnicov)

10/18/2022, 4:00 PM
I'm really sorry, I have no current eta on this. And I was sure I'd linked https://docs.lakefs.io/understand/roadmap.html#table-format-support to our roadmap... Could you please add a note on the issue, ideally giving sizes and/or what would be minimally useful? It can help us boost priorities
einat.orr

einat.orr

10/18/2022, 4:05 PM
H @Ranjeetha Raja, let me add some deta on @Ariel Shaqed (Scolnicov) initial answer. Delta lake is supported, with the exception of "merge" in case both source and destination branches have changed. Hudi and Iceberg support is expected by the end of December. We will share the GitHub issues with you once those are all available.
r

Ranjeetha Raja

10/18/2022, 4:37 PM
Thank you Ariel and Einat!
Ariel Shaqed (Scolnicov)

Ariel Shaqed (Scolnicov)

10/18/2022, 4:39 PM
Sure! And adding that note on the issue can always really help us move things 😃
5:10 PM