• Yoni Augarten

    Yoni Augarten

    1 week ago
    Hey all, The docs mention that in order to upgrade from a version lower than 0.80.0, one must first upgrade to 0.80.0, and only then to a later version. What does this mean for Kubernetes users? Will a Helm upgrade command work if I'm now using a version before 0.80.0?
    Yoni Augarten
    Itai Admi
    +1
    6 replies
    Copy to Clipboard
  • Ariel Shaqed (Scolnicov)

    Ariel Shaqed (Scolnicov)

    1 week ago
    The Airflow test action is failing to set up its lakeFS container. Did something change, maybe due to kv?
    Ariel Shaqed (Scolnicov)
    1 replies
    Copy to Clipboard
  • e

    Elad Lachmi

    1 week ago
    Hi everyone, I'd like to start working on the conversion of the lakeFS webui to TypeScript I believe that as the code base grows, TS's value will hugely outweigh it's drawbacks IMHO it will also make contributing to the lakeFS UI much simpler I'll commit to migrating all of our existing code base, so we don't need to start with a lax
    tsconfig
    , which will require additional refactoring in the future Any feedback is welcome! Thoughts?
    e
    Jonathan Rosenberg
    3 replies
    Copy to Clipboard
  • e

    Elad Lachmi

    6 days ago
    Hi again, I'm working on a new File Viewer for the lakeFS webui This is the issue (related PR mentioned), in case anyone would like to chime in on the issue and/or PRhttps://github.com/treeverse/lakeFS/issues/4225
  • Ariel Shaqed (Scolnicov)

    Ariel Shaqed (Scolnicov)

    6 days ago
    To recap: I'm seeing reasonable speeds for writing 4000 partitions from Spark, but as soon as I switch SaveMode over to overwrite things are slow again. Can we count DeleteObjects operations and their paths, to see how overwrite deletes the old data?
  • Barak Amar

    Barak Amar

    6 days ago
    Merged by mistake a failed integration test into master branch - working on it.
    Barak Amar
    1 replies
    Copy to Clipboard
  • Ariel Shaqed (Scolnicov)

    Ariel Shaqed (Scolnicov)

    6 days ago
    @Barak Amar let's continue our lakeFSFS recursive deletes discussion here rather than on DMs.
    Ariel Shaqed (Scolnicov)
    Barak Amar
    14 replies
    Copy to Clipboard
  • Barak Amar

    Barak Amar

    5 days ago
    Hi, planning to release lakeFS - let me know if there is something to wait for or any issues you see.
  • Ariel Shaqed (Scolnicov)

    Ariel Shaqed (Scolnicov)

    5 days ago
    Starting to release lakeFSFS -- let me know if I should wait.
  • Ariel Shaqed (Scolnicov)

    Ariel Shaqed (Scolnicov)

    5 days ago
    Hi devs, Something I've been thinking about. We see Spark running a very large number of "statObject" calls and "listObject" - and I would like to speed those up. (This is not an immediate suggestion, or even a suggestion, I'm just trying to wrap my head around things and understand the feasible solution space!) Now almost all of these calls seem very racy: delete a file, see if its "directory" is empty; create a file, see if we want to give it a directory marker - stuff like that. The point about these calls is that the calling code does not expect a consistent answer! Supposewe could identify on the Spark side these calls (because some calls really need consistency - it's just that others don't...). Then we could ask lakeFS for an inconsistent ("eventually" consistent) answer! Would it be possible? How much faster would it be, say on DynamoDB and on PostgreSQL?
    Ariel Shaqed (Scolnicov)
    Itai Admi
    +1
    4 replies
    Copy to Clipboard