Hi, I'm using lakeFS on my local machine. I uploaded some data to my repo and now I would like to remove it. To upload data I used:

aws --endpoint-url=<http://localhost:8000> --profile local s3 cp D:\lakefs_upload <s3://example-repo/branch_full_data/> --recursive

Then I to remove it:

aws --endpoint-url=<http://localhost:8000> --profile local s3 rm <s3://example-repo/branch_full_data/> --recursive

I can see that the data is not on my repo repo and that the branch is now empty however, the data seems to be stored somewhere else as the storage space is not being recovered on my PC. Is the data stored somewhere else when it is uploaded? Thanks

Hi @Marija Vella 🙂 When you delete the objects from lakeFS the data is not deleted from the storage namespace. In order to hard delete your objects you should run garbage collection. However, this is not compatible with local storage.

Hi Lynn, thanks for your reply. Are there any other solutions for local storage?

Unfortunately, we don't have another solution for local storage.

Also please note that we do not support

local

block storage for any production use; see this section on the lakeFS architecture page for details.. It is intended as an easy on-ramp for checking out lakeFS, but because it relies on less performant and more expensive storage it will never be as easy, cheap, or fast to run as a lakeFS installation backed by an object store. Currently we spend considerably less time testing and developing lakeFS to work on local storage than we do for object storage such as S3, Azure or GCS.

@Marija Vella lakeFS supports min.io that you can deploy locally. https://docs.lakefs.io/integrations/minio.html lakeFS MinIO This section explains how to use MinIO as the underlying storage for lakeFS.