Hi @Barak Amar and VE-team, Really excited to see lakeFS on KV is now a proposal! Did you incorporate my "lock freedom" suggestions on the PR into the proposal? These are intended to ensure that at all times at least one commit makes progress. These will be really important in order to support streaming workflows. For instance, for almost any use of Kafka with lakeFS, we will have periodic commits which may contain many objects. As soon as there is a hiccup we will have concurrent commits -- all of these after the first large commit will also be large! Without lock freedom, concurrent commits get in one another's way, and progress might not occur. As more commits pile up, the probability of any one of them succeeding drops rapidly, and the system will have unbounded delays and might effectively livelock.

The proposal still open - just merged. It is not included as I see it. How the lock-free mechanism identify a new commit vs retry of a failed commit? the bad of pending comments will have to be hashed based to identify that the specific request is already submitted? Or it is there just to 'wait' and not try/and fail?

As discussed F2F, I shall send it as a PR. I think I like managing proposals as PRs to an "unaccepted" directory!! 🙂 !!