https://lakefs.io/ logo
Title
j

Juarez Rudsatz

05/06/2023, 8:40 PM
Hi everyone, Last week I attended some webinars where some Google specialists heavily promoted BiqQuery over traditional Data Lake solutions. So, now I have some questions in my mind: • Can I use lakeFS with BigQuery in GCP? ◦ There are plans to support BigQuery storage within LakeFS? • How does BQ branching fare against the time travel feature in BQ (and Delta Lake) overall? • Is BQ superior when compared to Parquet, ORC, Delta Lake, Iceberg, or other data lake solutions? ◦ What are the catches? ◦ Please share your opinion. Thanks,
o

Oz Katz

05/09/2023, 11:22 AM
Hey @Juarez Rudsatz! Welcome 🙂 You can run lakeFS on top of GCP, with GCS as the object store (see the deployment guide).
Can I use lakeFS with BigQuery in GCP?
Currently, BigQuery does not support external connections other than AWS S3 and Azure blob store, so at the moment there's no BigQuery support, unfortunately.
How does BQ branching fare against the time travel feature in BQ (and Delta Lake) overall?
I can't really say. I assume a BigQuery solutions architect or account manager could probably help with that.
Is BQ superior when compared to Parquet, ORC, Delta Lake, Iceberg, or other data lake solutions?
Again, probably more of a BigQuery question that is orthogonal to lakeFS.. I will say that "superior" is a subjective term, so I would say it really depends on your use case and constraints 🙂
Hope this helps 🙂
👍🏻 1