• n

    Niro

    3 weeks ago
  • Or Tzabary

    Or Tzabary

    3 weeks ago
    hi all, I started working on the failure in loading a user with % in his name issue. first, I thought that we're missing encoding somewhere, but going deeper into the issue, it seems that it's something broken with react's history library and how it works with percent signs in the URL. Encoding this using
    encodeURIComponent
    or
    encodeURI
    doesn't help as % is used as part of the encoded value. Another approach is to encode using base64 which is URL-safe. I believe the right approach would be to disallow some characters for user/group/policy names as we use them in the URL and to avoid other potential bugs. Comparing to AWS IAM's restrictions, they allow only specific characters to be used in the (alphanumeric and '+=,.@-_' characters.) I think that restricting to these characters is a the right approach to take. this means it'll be a breaking change in case someone used % in previous versions, but to be honest, the UX was already broken with entries with % in its name. Thoughts? @Elad Lachmi thanks for the help
    Or Tzabary
    e
    +1
    26 replies
    Copy to Clipboard
  • Ariel Shaqed (Scolnicov)

    Ariel Shaqed (Scolnicov)

    2 weeks ago
    I'm reading up on the Spark DataSource V2 API. Does anyone have good reading material? I found this series of blogs, but would be happier with some more official documentation. THANKS!
  • n

    Niro

    2 weeks ago
    Hi All, I'd like to understand the usage of lakeFS StageObject API (PUT/repositories/{repository}/branches/{branch}/objects). Specifically if there are any usecases which provide a physical address which is inside the repository storage namespace. We are in the works of adding a garbage collection process for uncommitted data and considering allowing this API only on physical addresses which are outside the repository namespace
    n
    Oz Katz
    +2
    7 replies
    Copy to Clipboard
  • Ariel Shaqed (Scolnicov)

    Ariel Shaqed (Scolnicov)

    2 weeks ago
    TIL a good way to compute a nonserializable value once in an object that needs to be Serializable to use in Spark:
    @transient lazy val notSerializable = computeFromSerializables(param1, param2)
    🤷🏼
  • Ariel Shaqed (Scolnicov)

    Ariel Shaqed (Scolnicov)

    2 weeks ago
    I want to commit some ranges SSTable to the lakeFS repo for a benchmark. Should be around 1 MiB to 20 MiB in size. Does anyone have one that I could use? (A range SSTable contains object metadata (including pathnames), so you should feel comfortable revealing such information in the file. It does NOT contain object data (the actual contents of the objects)!)
  • Amit Kesarwani

    Amit Kesarwani

    2 weeks ago
    Can delete_object Python API be enhanced to recursively delete all objects under the specified path?
    lakectl fs rm
    command already has this functionality but Python API doesn’t.
    Amit Kesarwani
    Guy Hardonag
    4 replies
    Copy to Clipboard
  • Itai Admi

    Itai Admi

    1 week ago
    We have a lot of goodies waiting to be released in v0.81.0. I want to start a release soon, should I wait for anything? Some of the features/bugs:
    - Fix: Pyramid delete before open (#4062)
    - Lookup commit when possible while commit prefix search (#4154)
    - support aws named profile for dynamodb (#4163)
    - s3 block adapter: enable skip verify certificate of storage endpoint (#4117)
    - DB Cleanup (#4160)
    - Add pgxpool metrics to kv/postgres implementation (#4137)
    - Feature/badger as kv (#4166)
    Itai Admi
    Barak Amar
    3 replies
    Copy to Clipboard
  • Ariel Shaqed (Scolnicov)

    Ariel Shaqed (Scolnicov)

    1 week ago
    Of interest when writing a Spark OutputCommitter: write some performance info into an additional report file.