So this question isn't necessarily specific to lakeFS, but hoping to get some thoughts. My company is in early stages of an effort to improve data management. At what point does it make sense or become necessary to adopt an open table format in addition to lakeFS? I'm hesitant to try to introduce too many new things at once, but its not clear to me if later adoption would be more painful.

I'd have seen them as satisfying two different requirements

Do you need data version control? -> lakeFS Do you need support for tabular data, transactions, changing schemas, etc etc? -> open table format

Maybe this can help @Christopher Burke - https://lakefs.io/blog/open-table-formats-vs-data-version-control-systems/?utm_ca[…]edium=social&utm_content=blog_otf-vs-data-version-control_230427

BTW - I’m curious if it does 🙂 - because this question is why I wrote it 🙂 So feedback welcomed.

Thanks - definitely helps to clarify the benefits of each. I think I'm stuck on weighing the benefits of adopting something like lakeFS + some open table format vs lakeFS and just parquet files. I'm thinking walk before you run, so to speak, but not sure if adding open table format at a later time would become a pain point. Just found your Best Practices to Easily Adopt lakeFS blog, though, so I'll see where that gets me. 🙂

Thanks. That one mostly explains the underlying structure of the files on the object store once you implement lakeFS (Very bad title - my fault) 🙂