Thread
#dev
    n

    Niro

    2 weeks ago
    Hi All, I'd like to understand the usage of lakeFS StageObject API (PUT/repositories/{repository}/branches/{branch}/objects). Specifically if there are any usecases which provide a physical address which is inside the repository storage namespace. We are in the works of adding a garbage collection process for uncommitted data and considering allowing this API only on physical addresses which are outside the repository namespace
    Oz Katz

    Oz Katz

    2 weeks ago
    Perhaps the lakeFS Hadoop Filesystem is using that API?
    Yoni Augarten

    Yoni Augarten

    2 weeks ago
    @Niro the lakeFS HadoopFS uses linkPhysicalAddress - which AFAIK is almost the same and is going to be unified with Stageobject
    n

    Niro

    2 weeks ago
    The intention is to allow LinkPhysicalAddress only on previously issued addresses via GetPhysicalAddress. I'm wondering if there's another different usecase for StageObject
    Yoni Augarten

    Yoni Augarten

    2 weeks ago
    I see
    Interesting
    Ariel Shaqed (Scolnicov)

    Ariel Shaqed (Scolnicov)

    2 weeks ago
    AFAIK ingest uses StageObject outside of a received address. This is a feature that has seen real use, e.g. we saw someone ingesting data into the playground!
    (I know because they opened a bug that involved granting the playground some fairly subtle S3 permissions)