https://lakefs.io/ logo
#help
Title
y

Yaphet Kebede

09/06/2023, 5:09 PM
Hi All, just a quick question on a race condition that i have noticed in commits. Say i have two processes, on uploads a bunch of files that need to go into a single commit, while this process is still going, process b start uploading a set of its own files. Now these are all staged in uncommited changes. And Process A is now able to commit extra files that have been uploaded by Process B, is this something that can be avoided?
i

Iddo Avneri

09/06/2023, 5:13 PM
Could you have the different process upload to different ingest branches and commit and merge them together once done?
y

Yaphet Kebede

09/06/2023, 5:20 PM
ah ok that make sense, so really uploads (uncommited changes) are not client/session based, they are identified based on the branch they target?
i

Iddo Avneri

09/06/2023, 5:20 PM
Generally speaking, it is good practice to work on a single branch and then merge changed atomically once done.
👍 2
gratitude thank you 1
👍🏼 1
You can look into the data lineage example in our samples and see a very simple example of this.
s

Sivan Bercovici

09/19/2023, 1:17 PM
.
2 Views