while writing tests, I noticed that I have to insert some pauses otherwise lakeFS won't work as expected For example, if I don't wait here, it will say the repo id is not unique, or if I don't wait here it will commit nothing because it won't see yet any staging files to commit... Is this normal? Would the jvm client be different (is the python wrapper asynchronous)?

Hi @Giuseppe Barbieri, can you please provide the code in a different way? we don’t have access to https://codebase.helmholtz.cloud/ @Ariel Shaqed (Scolnicov) will probably be available to look into it.

uh sorry, I fixed the links

Cool. I hope to have something useful to say by tomorrow.

to give more background, I switched now to

lakectl

, which looks much better in this regard however, a couple of times I also got the same issue..

Anyway those links don't work for me without logging in, sorry. Could you try to debug them in some incognito mode on your browser, do you think?

sorry, forgot to save

Gotcha, thanks! • We do in fact cache repo details: this is the single most accessed key, and if we did not it would be hammered. I know that you are not looking for this answer. At least we're in good company... S3 behaves the same way.

Yeah ok, but why then with

lakectl

the problem is much more rare? I don't get it.. is it really all on the REST delay?

If you have multiple lakeFS instances and the repo is not heavily loaded on each, then you might be seeing different HTTP connection re-use patterns. If you really wanted to go down this rabbit-hole then you could start with tcpdump. However we will probably not be able to assist you much on this question.

If you really wanted to go down this rabbit-hole then you could start with tcpdump.

Uhm, would you mind elaborate a little more?

Hi Giuseppe, We're unfortunately flooded with work right now. Everything we've seen so far is intended behaviour. I'm afraid I will not have time to dig into this. I believe that if we do then we may learn a great deal about your particular setup, but not advance lakeFS. So I'm sorry, but I will not be going into this right now.

I understand, thanks for the transparency

Luckily it is open-source so you don't need me 🙂 • This cache is in ref/manager.go. On the data/metadata path we can return stale cache data for repositories and auth data (auth.cache.go). • We also cache protection rules and actions. • Other things should be read-after-write consistent and never stale.