03/14/2023, 8:49 PM
Again, have anyone being able to implement dbt with lakeFS locally. Can you point me to a resource I am going through the docs, but I am yet to understand a thing. Or is it a feature that comes with lakeFS cloud I think these are the two I need clarification on for now Cheers

Guy Hardonag

03/14/2023, 9:21 PM
Hi @Aviator, the DBT feature is part of the open source, running a local environment with DBT can be complicated ( regardless to lakeFS ) to run everything locally you will also need to run trino or spark locally The documentation can be a bit confusing, in https://docs.lakefs.io/integrations/dbt.html it describes how to manage branches with DBT and lakeFS assuming everything is already running. the lakeFS repository contains our “Everything Bagel” which is a docker-compose that runs many tools on your docker machine (including lakeFS, Spark, Trino, and DBT). it could be a good resource to understand the configurations you need Adding some resources about the everything bagel: https://lakefs.io/blog/the-docker-everything-bagel-spin-up-a-local-data-stack/ https://lakefs.io/blog/the-everything-bagel-ii-versioned-data-lake-tables-with-lakefs-and-trino/
❤️ 1

Robin Moffatt

03/14/2023, 10:02 PM
@Aviator the links that Guy provides are good ones. We should update the docs though to make all this easier! Are there any particular bits that you're struggling with that we can make sure to cover?


03/15/2023, 6:25 AM
Thank you @Robin Moffatt Thats all for now