Title
#lakefs-for-beginners
m

Marija Vella

11/18/2022, 10:22 AM
Hi, I'm using lakeFS on my local machine. I uploaded some data to my repo and now I would like to remove it. To upload data I used:
aws --endpoint-url=<http://localhost:8000> --profile local s3 cp D:\lakefs_upload <s3://example-repo/branch_full_data/> --recursive
Then I to remove it:
aws --endpoint-url=<http://localhost:8000> --profile local s3 rm <s3://example-repo/branch_full_data/> --recursive
I can see that the data is not on my repo repo and that the branch is now empty however, the data seems to be stored somewhere else as the storage space is not being recovered on my PC. Is the data stored somewhere else when it is uploaded? Thanks
Lynn Rozen

Lynn Rozen

11/18/2022, 10:43 AM
Hi @Marija Vella 🙂 When you delete the objects from lakeFS the data is not deleted from the storage namespace. In order to hard delete your objects you should run garbage collection. However, this is not compatible with local storage.
m

Marija Vella

11/18/2022, 10:45 AM
Hi Lynn, thanks for your reply. Are there any other solutions for local storage?
Lynn Rozen

Lynn Rozen

11/18/2022, 10:52 AM
Unfortunately, we don't have another solution for local storage.
Ariel Shaqed (Scolnicov)

Ariel Shaqed (Scolnicov)

11/20/2022, 8:09 AM
Also please note that we do not support
local
block storage for any production use; see this section on the lakeFS architecture page for details.. It is intended as an easy on-ramp for checking out lakeFS, but because it relies on less performant and more expensive storage it will never be as easy, cheap, or fast to run as a lakeFS installation backed by an object store. Currently we spend considerably less time testing and developing lakeFS to work on local storage than we do for object storage such as S3, Azure or GCS.
einat.orr

einat.orr

11/20/2022, 12:28 PM
@Marija Vella lakeFS supports min.io that you can deploy locally. https://docs.lakefs.io/integrations/minio.html lakeFS MinIO This section explains how to use MinIO as the underlying storage for lakeFS.