I first started lakefs by running "docker-compose up", then do some initial setup, etc, then run "docker-compose down" to stop lakefs, then run "docker-compose up" to start lakefs again, but all previous setup are lost, I have to re-setup again. Does anyone know how I can solve this?

Hi @donald, the docker compose was to have a quick start to experiment with lakefs - it is working on local storage and postgres inside a container. Keeping the storage on a dedicated database and cloud based storage is one way to do it. But for local experimentation you can also use volumes and mount the database and lakefs's files to your host. Is this what you are looking for?

BTW, have you had a look at https://demo.lakefs.io? Gives you a managed environment for a week. You can't do everything on it, but you can do a lot. And setting it up takes about zero time 🙂 (As you might be able to tell, we're really proud of it!)

Previously I installed one Postgres locally and then run lakefs binary directly by configuring lakefs to my local postgres database. We are satisfied with lakefs . Now we are going to setup one pilot project with 10G image data. So I am trying to figure out what is the best practice to setup the infrastructure for this. That is why I am trying docker-compose by following the guide. Cloud is not option as we can not store the data in the cloud. Do you know any document how to setup lakefs for enterprise level?

by the way, when started the lakefs again after stop the lakefs by running "docker-compose down", I first got an warning message, "postgres_1 | WARNING: no logs are available with the 'none' log driver", then got an error "failed to connect to `host=postgres user=lakefs database=postgres`: server error (FATAL: the database system is starting up (SQLSTATE 57P03))" and said "lakefs_lakefs_1 exited with code 2"

Did you consider using MinIO as a storage mechanism?

Running postgres with volume mount or setup dedicated to your host will work. I will post an example here soon.

about the error you posted. if it is reproduced, a list of containers and volumes using docker will help me identify the reason.

from the log messages it looks like postgres failed to start and lakefs exit after waiting for postgres aborted.

the first two services will setup the environment. you should comment them out or remove them and manually configure if you going to docker-compose up -d and down again

@Barak Amar Many thanks for your example.