• Yoni Augarten

    Yoni Augarten

    1 year ago
    Using volumes for committed cache: will it work if multiple pods shared the same directory for caching?
    Yoni Augarten
    Itai Admi
    +1
    3 replies
    Copy to Clipboard
  • Ariel Shaqed (Scolnicov)

    Ariel Shaqed (Scolnicov)

    1 year ago
    I want to try to speed up bulk iterator operations by prefetching. The easy way to express a "value that will show up" is with a future. Futures have a pretty simple implementation in terms of channels, see e.g. ChanOnlyOne which effectively inlines an implementation of futures into a sync.Map. So...1. use an existing futures library? [go-promise](https://github.com/fanliao/go-promise) is sort-of promising and makes the most sense, but introduces needless promises along the way. 2. Write my own? I could, how familiar are futures (not promises, but probably familiar to anyone who's used those)? 3. Inline my own implementation? Least scary, but hardest to grasp because it melds multiple concepts into the same objects. Right now I'm going with 3 and mixing channels into it, but will happily hear options.
    Ariel Shaqed (Scolnicov)
    Oz Katz
    2 replies
    Copy to Clipboard
  • Ariel Shaqed (Scolnicov)

    Ariel Shaqed (Scolnicov)

    1 year ago
    TIL while working on https://github.com/treeverse/lakeFS/issues/1319 that gomock has trouble testing goroutines. Specifically https://github.com/golang/mock/issues/346 is open (tl;dr: mock calls
    t.Fatal
    on an unexpected failure because it has nothing better to do, and
    t.Fatal
    is broken and doesn't work in goroutines. This is a good argument for not using gomock at all in new code -- it is extremely dependent on the implementation. For now I may rewrite batch_test.go to use fakes. Any better ideas?
    Ariel Shaqed (Scolnicov)
    Barak Amar
    5 replies
    Copy to Clipboard
  • Ariel Shaqed (Scolnicov)

    Ariel Shaqed (Scolnicov)

    1 year ago
    Question... Planning a Spark DataSource (v2...) to read Rocks metadata. It will run on a JVM. Is there a particular community preference for language (Java or Scala) in terms of readability? In terms of hackability? In any other terms?
    Ariel Shaqed (Scolnicov)
    Oz Katz
    9 replies
    Copy to Clipboard
  • Ranganath s

    Ranganath s

    1 year ago
    hi team - can some one help me how can i build lakefs from source ? or point to some doc for the same. thank you..
    Ranganath s
    Oz Katz
    6 replies
    Copy to Clipboard
  • Shamika Kumar

    Shamika Kumar

    1 year ago
    hi team, I noticed what looks like a bug when I try to filter repositories
    Shamika Kumar
    Barak Amar
    +1
    3 replies
    Copy to Clipboard
  • Shamika Kumar

    Shamika Kumar

    1 year ago
    Also, something else I noticed, the option to delete import-from-inventory branch should be removed from the UI
    Shamika Kumar
    Oz Katz
    4 replies
    Copy to Clipboard
  • Oz Katz

    Oz Katz

    1 year ago
    question (@Barak Amar @Itai Admi @Ariel Shaqed (Scolnicov) maybe you’ll know): why are we using advisory locks to manage write concurrency control to the branch? from the looks of it, we get the same guarantees by doing SELECT FOR UPDATE to get an exclusive lock for committing, merging, deleting branches, with SELECT FOR SHARE when reading the staging token for writing entries to it. It’s simpler since it removes the branch locker and extra conn pool (and avoids the double round trip/double tx overhead). what am I missing?
    Oz Katz
    Ariel Shaqed (Scolnicov)
    +1
    7 replies
    Copy to Clipboard
  • Ariel Shaqed (Scolnicov)

    Ariel Shaqed (Scolnicov)

    1 year ago
    [Start of a separate thread...] I've been thinking about the branch locker because we have issues with it. I now think the branch locker requirement is difficult to implement because it is hard to understand. I would prefer to replace it with an "everything waits" requirement, and leave it to clients to time out. This is easy to explain to users (and to computers). If the user cares about concurrency control, then it is reasonable to expect uploads to block while waiting for commits. If they do not (a "continual uploads and commits" scenario), then imposing a 0 timeout will be worse than allowoing a configurable timeout. Note that in the upcoming thick client scenario, the upload itself will anyway need to happen only once -- but that doesn't affect the "always block" scenario.
    Ariel Shaqed (Scolnicov)
    1 replies
    Copy to Clipboard
  • s

    Souvik

    1 year ago
    <!here> I'm beginner to LakeFS and have done initial hosting on top AWS EKS cluster. Now I have a question related to implementation. Can we do some sort of user activity monitoring (user details, sessions etc)when lakefs is running onto an EKS cluster? If so how can we achieve that?
    s
    Oz Katz
    5 replies
    Copy to Clipboard