• Shamika Kumar

    Shamika Kumar

    1 year ago
    hey team, could someone help me take a look at -> https://github.com/treeverse/lakeFS/pull/1410 for this issue -> https://github.com/treeverse/lakeFS/issues/1372
    Shamika Kumar
    1 replies
    Copy to Clipboard
  • Ariel Shaqed (Scolnicov)

    Ariel Shaqed (Scolnicov)

    1 year ago
    Interested in hearing opinions about https://github.com/treeverse/lakeFS/issues/1481 -- how to deliver protobuf changes to the Spark client and other customer code. So far I have 3 possibilities:1. Do nothing, just copy proto files until we actually get tooling to do it. Might mean the Spark client never supports any new features (because we forget to copy them over) until a customer runs into a missing field and complains. 2. Move to a third git repo and use git submodules. Known to work, never seen it happily used. Probably lowest friction for Treeverse developers. 3. Build proto packages inside the lakefs repo (or a new proto-only repo) for each language, publish to many different package repositories (Maven Central, Go via GitHub, probably PyPI and npmjs at some stage). Adds maximal friction to core development. Happy to read your comments on the issue or here. Thanks!
    Ariel Shaqed (Scolnicov)
    Yoni Augarten
    +1
    3 replies
    Copy to Clipboard
  • Ariel Shaqed (Scolnicov)

    Ariel Shaqed (Scolnicov)

    1 year ago
    Sample cross-Spark-version module (library?) build.sbt: https://github.com/simplexspatial/osm4scala/blob/master/build.sbt. Going with something like that, unless we can find something simpler.
    Ariel Shaqed (Scolnicov)
    1 replies
    Copy to Clipboard
  • Ariel Shaqed (Scolnicov)

    Ariel Shaqed (Scolnicov)

    1 year ago
    singleflight and cancellation: the net package just cancels cancellation:
    // We don't want a cancellation of ctx to affect the
    	// lookupGroup operation. Otherwise if our context gets
    	// canceled it might cause an error to be returned to a lookup
    	// using a completely different context. However we need to preserve
    	// only the values in context. See Issue 28600.
    	lookupGroupCtx, lookupGroupCancel := context.WithCancel(withUnexpiredValuesPreserved(ctx))
    Ariel Shaqed (Scolnicov)
    1 replies
    Copy to Clipboard
  • Guy Hardonag

    Guy Hardonag

    1 year ago
    Not important. Would you like me to change it?
    Guy Hardonag
    1 replies
    Copy to Clipboard
  • Oz Katz

    Oz Katz

    1 year ago
    Fixing this issue is a breaking change: https://github.com/treeverse/lakeFS/issues/1730 - I suggest doing it anyway, and sooner rather than later. If anyone feels otherwise, would be happy to discuss.
    Oz Katz
    Ariel Shaqed (Scolnicov)
    3 replies
    Copy to Clipboard
  • Ariel Shaqed (Scolnicov)

    Ariel Shaqed (Scolnicov)

    1 year ago
    In validating config, I've discovered I don't understand metastore config. This is a
    lakectl
    command. It uses configuration functions in
    config.go
    , which works because
    viper
    has a default global configuration object which is different in
    lakectl
    and
    lakefs
    . But at the very least it creates a dependency between
    pkg/config
    and
    cmd/lakectl
    . Does anyone know why we do this? More importantly, is it OK for me to split off lakectl into another package?
    Ariel Shaqed (Scolnicov)
    2 replies
    Copy to Clipboard
  • Oz Katz

    Oz Katz

    1 year ago
    We’ve started working on our Airflow integration (🤩) working closely with the team at Astronomer (https://www.astronomer.io/). If anyone has any specific requirements or feedback we’d be happy to collaborate here: https://github.com/treeverse/lakeFS/issues/1771
    Oz Katz
    1 replies
    Copy to Clipboard
  • sumesh kanayi

    sumesh kanayi

    1 year ago
    Hi i am new to lakefs and i can write little bit of go ,can we extend lakefs to support other s3 storage options as well ,example: https://github.com/joyent/manta . Can some one please help me here with any pointers/documentations so that i can try implementing the same
    sumesh kanayi
    Itai Admi
    +2
    30 replies
    Copy to Clipboard
  • Shamika Kumar

    Shamika Kumar

    1 year ago
    hi team, I noticed what looks like a bug.. I get a 404 not found error in the console whenever I select a commit from the branch selector dropdown
    Shamika Kumar
    Tal Sofer
    +1
    7 replies
    Copy to Clipboard