• Edmondo Porcu

    Edmondo Porcu

    6 months ago
    Hi. Are there priorities for the good-first-issue issues on Github?
    Edmondo Porcu
    1 replies
    Copy to Clipboard
  • Paul Singman

    Paul Singman

    6 months ago
    There is a community member in the awesome ML.Ops slack asking for more details on the dataset we used to test lakefs with
    lakectl abuse
    command documented here. Can someone help me with answers to his questions (see screenshot)?
    Paul Singman
    Oz Katz
    2 replies
    Copy to Clipboard
  • Oz Katz

    Oz Katz

    6 months ago
    Hey @Yusuf K! as @Itai Admi mentioned, we’re indeed talking to the Snowflake team about this use case (External table support for lakeFS). We'd love to work with you to make sure we're building something that meets your needs!
    Oz Katz
    1 replies
    Copy to Clipboard
  • Edmondo Porcu

    Edmondo Porcu

    6 months ago
    Does anyone have a view on this? https://github.com/treeverse/lakeFS/issues/2364 I am happy to help, but the current error handling style is not very clear to me
    Edmondo Porcu
    1 replies
    Copy to Clipboard
  • Ariel Shaqed (Scolnicov)

    Ariel Shaqed (Scolnicov)

    6 months ago
    Hi @Barak Amar and VE-team, Really excited to see lakeFS on KV is now a proposal! Did you incorporate my "lock freedom" suggestions on the PR into the proposal? These are intended to ensure that at all times at least one commit makes progress. These will be really important in order to support streaming workflows. For instance, for almost any use of Kafka with lakeFS, we will have periodic commits which may contain many objects. As soon as there is a hiccup we will have concurrent commits -- all of these after the first large commit will also be large! Without lock freedom, concurrent commits get in one another's way, and progress might not occur. As more commits pile up, the probability of any one of them succeeding drops rapidly, and the system will have unbounded delays and might effectively livelock.
    Ariel Shaqed (Scolnicov)
    Barak Amar
    2 replies
    Copy to Clipboard
  • Ariel Shaqed (Scolnicov)

    Ariel Shaqed (Scolnicov)

    6 months ago
    Are you writing your own Graveler code, or interested in doing so? To be very clear: If you are a lakeFS user, or if you program inside lakeFS, then you do not care if we do the following. It is only important if you have a copy of any
    .proto
    file from github.com/treeverse/lakeFS somewhere in your project. Graveler serializes its actual commit and range values using protocol buffers. One constantly annoying factor with protocol buffers is that there is no way to publish them. Instead, most projects just copy them over from their source project. A project called Buf is trying to create a protocol buffer repository (think "NPM for protobufs"); it is now in beta. Would you be interested in consuming protocol buffers from there? Would you trust this project? Thanks!
    Ariel Shaqed (Scolnicov)
    Itai Admi
    2 replies
    Copy to Clipboard
  • Itai Admi

    Itai Admi

    6 months ago
    Any idea why many checks are skipped for forked PRs? For example this PR. Is it the secrets issue? I thought that’s resolved..
    Itai Admi
    1 replies
    Copy to Clipboard
  • Ariel Shaqed (Scolnicov)

    Ariel Shaqed (Scolnicov)

    6 months ago
    Ariel Shaqed (Scolnicov)
    Itai Admi
    2 replies
    Copy to Clipboard
  • Ariel Shaqed (Scolnicov)

    Ariel Shaqed (Scolnicov)

    6 months ago
    Hi lakeFS developers! This PR changes the name of the "test spark metadata client" action. So right now: • Any PRs on the lakeFS repo that branched out prior to 10bced0f on lakeFS can be green and merged without administrative force; • Any PRs on the lakeFS repo that branched out at or after 10bced0f4 will not be green and need administrative force to merge to master. In 3 hours' time I plan to change the name of the required test to the new names. So starting at around 14:00 UTC today (16:00 IST) these states will be reversed. After this time, your old PRs will not be green until you rebase them onto a more recent of lakeFS (or merge, I guess, for people who don't rebase their development branches...). I will be happy to help if you run into any difficulties!
    Ariel Shaqed (Scolnicov)
    1 replies
    Copy to Clipboard
  • Edmondo Porcu

    Edmondo Porcu

    6 months ago
    Is there already a setup to execute tests against a mock infrastructure within the CI/CD pipeline?
    Edmondo Porcu
    Ariel Shaqed (Scolnicov)
    +1
    10 replies
    Copy to Clipboard