Hey team, is it possible to copy an object pointer from one lakefs instance and copy to another instance? In other words, is it possible to mirror pointers in multiple lakefs instances?

Hi @Matthieu Oliveira No, this isn’t currently supported. However, we do have an export to S3 (a manual operation, but it should be easy to schedule). We also have a “continuous import” feature on the roadmap for the near future, which will periodically import changes from an S3 bucket. You could theoretically combine those two if you’re looking for an active-passive/read-replica type of configuration If you’re looking for two-way synchronization, that’s a more complex operation (conflicts and such). That’s not currently supported and isn’t on the near future roadmap AFAIK.

Awesome, I highly appreciate this

Sure, no problem

Hi Matthieu, You might also do this yourself using the lakeFS API. Say you have lakeFS A and lakeFS B, and repositories on each one. (They will necessarily use different storage namespaces for their repositories!) • Create an object on lakeFS A and commit it. If you do this with an uncommitted object, it will be very easy for uncommitted garbage collection to remove it. So PLEASE only do this for committed objects! • Use statObject on lakeFS A to determine the physical address. • Use stageObject on lakeFS B to to stage the object into a repository. Obviously it can be in any repo, branch and path on lakeFS B. • (Use commit on lakeFS B to commit all changes, of course). Now the object is still owned by lakeFS A. For instance, if A decided to garbage-collect it then B will obviously lose it and return "410 Gone" if you attempt to access it. Bbut B will never garbage-collect the object. Given that this can be messy, please consult with a more complete use-case before you decide to do this!