Hey everyone, Question I have regading the retention of objects , Im not sure I understood well the documentation. If I configure on main branch retention for x days . I will delete only objects that i deleted from the current branch after x days or delete their old commits. I want commits older then x days to be deleted it is something possible to archive ?

Commit holds a snapshot of pointers to the data. As such, it’s a metadata entity. The retention available today in lakeFS will not delete the Commit entity. It will delete the data itself based on the retention rules. Commits will always be available. When trying to reach a deleted entry in a commit, the server will return

409 Gone

response.

Want to make sure I understood correctly: so if for example retention is 5 days And i have commit from 6 day before now I will see it but Won't be able to access the data and nothing will be stored in the s3 for it. Without me activly deleting the object from the main branch

Right - if the retention rules marked that object to be deleted (e.g. it’s not at the HEAD of the branch) (Correcting my previous statement) Retention rules apply to deleted objects (not at the HEAD of the branch). So if an object was not deleted, lakeFS retention rules will not apply to it.

I try to add retention with rest API with user that have only read and right access and get premmsion error. There is a policy I can assign to user/group with premmisions to edit retention of repo so I won't need to use in my code the main admins credentials ?

See retention in https://docs.lakefs.io/reference/authorization.html

Oh so I just need to write my own poliy and assign to writer cool I will test it

Regrading just to deleted objects - So currently there is no way to keep versions of object just for last x days to be memory and cost efficient ?

I'm afraid so, you'll need to be explicit and delete the objects. Alternatively if they only exists in a single branch, you can delete the entire branch.

In my opinion feature equivalent to s3 bucket lifecycle of delete nocurrent versions can be very valuable in lakefs

I think you can still define it on s3 itself for any prefix that lakeFS doesn't use to manage the metadata. Let me consult the team and get back to you?

If it can solved with lifecycle rule it might be good for us if we be able to make different amount of days for different repositories . Thanks 🙂