Hello LakeFS community, is there a plan to enable support for open table formats like Iceberg, Delta Lake etc? If yes, what is the tentative ETA?

Hi @Ranjeetha Raja, Welcome to the lake! Both of these are on our roadmap, and Iceberg is "high priority". You can get some support for Delta even just by using the s3 gateway - obviously we need to improve that too.

thank you Ariel for the prompt response. Any ETA for the Iceberg support?

I'm really sorry, I have no current eta on this. And I was sure I'd linked https://docs.lakefs.io/understand/roadmap.html#table-format-support to our roadmap... Could you please add a note on the issue, ideally giving sizes and/or what would be minimally useful? It can help us boost priorities

H @Ranjeetha Raja, let me add some deta on @Ariel Shaqed (Scolnicov) initial answer. Delta lake is supported, with the exception of "merge" in case both source and destination branches have changed. Hudi and Iceberg support is expected by the end of December. We will share the GitHub issues with you once those are all available.

Thank you Ariel and Einat!

Sure! And adding that note on the issue can always really help us move things :-)

@Ranjeetha Raja This repo includes “Delta Lake Demo” notebook: https://github.com/treeverse/lakeFS-samples/tree/main/03-apache-spark-python-demo https://github.com/treeverse/lakeFS-samples/blob/main/03-apache-spark-python-demo/Delta%20Lake%20Demo.ipynb