https://lakefs.io/ logo
#help
Title
h

HT

09/06/2023, 1:31 AM
Hi team ! How to list all branch that can reach a given commit ? Is there a way to do that from the python sdk ?
i

Iddo Avneri

09/06/2023, 1:43 AM
Do you mean all branches that were merged into a commit? Can you explain the use case?
h

HT

09/06/2023, 1:47 AM
in our case, when we download from lakefs, we use our helper that then write a small file that saying "this folder data come from this repo, this path and this commit" Then people modify/play with the local folder ... Then want to push back to lakefs. Ideally, from the commit hash, I can list all branch that can reach that commit. If there is only one: then I can just push and merge against that branch. If there are multiple branch taht can reach that commit: then I will ask the user to specifiy which branch do he want to push&merge against
i

Iddo Avneri

09/06/2023, 1:49 AM
I see. I will let someone chime in about the code. But - did you look at local lakectl?
h

HT

09/06/2023, 1:50 AM
you mean the
local
command from
lakectl
? I did discover it recently and yes that look similar to my use case. But not sure if that have a python sdk ? and how to use it. I will dig a little further
i

Iddo Avneri

09/06/2023, 1:51 AM
See if this is helpful.
h

HT

09/06/2023, 1:54 AM
Hum ... this is new ... did not come across this yet ! When was that page create/updated ?
lakeFS 0.106.1 : ok, so really recently
sounds like I reinvent some wheels here šŸ˜›
Hopefully this will make it easier in the future.
h

HT

09/06/2023, 1:57 AM
is there python sdk behind it ?
we need to exlcude and check some file before one can push ....
i

Iddo Avneri

09/06/2023, 1:58 AM
Not sure
a

Ariel Shaqed (Scolnicov)

09/06/2023, 4:19 AM
Hi @HT, I'm not aware of any way to do this that does not traverse essentially all commits from all branch heads. This information is not readily available to lakeFS internals, either. git branch has a --contains flag. I think their approach would be too slow on a kv like ours, though. Can you open an issue please? Ideally you'd detail nonfunctional requirements there: ā€¢ How often do you need to perform this task? ā€¢ Does the response need to be consistent? That is, if I commit to a branch or create a branch and then immediately search for that commit or for another commit contained in that branch, is it ok to miss it? ā€¢ What is the expected response time? ā€¢ Is it interactive? Obviously you might not have answers to all of these, I made this laundry list to get information about the ones you might know
h

HT

09/06/2023, 4:40 AM
What I am trying to do is actually very likely been done by
lakectl local
Is there a python sdk for that ?
y

Yoni Augarten

09/06/2023, 7:29 AM
Hey @HT, lakectl local is a command line tool that uses your local installation of Git. As such, its functionality cannot be achieved by an API on the lakeFS server.
h

HT

09/06/2023, 7:41 AM
No python sdk ?
Is there doc about how it work behind the scene?
y

Yoni Augarten

09/06/2023, 7:42 AM
Yeah sorry I didn't really answer your question there: out Python SDK only implements our core API functionality, so presently no Python SDK for that. Let me check about that doc.
h

HT

09/06/2023, 7:43 AM
And I guess lakectl is written in go and not python ...
āœ… 1
I was in my way to implement what local is doing in python ... Just don't want to reinvent the wheel...
y

Yoni Augarten

09/06/2023, 7:48 AM
Unfortunately the only doc we have is the blog post above. Once you've gone through it I'm here to answer any other questions you may have.
h

HT

09/06/2023, 8:00 AM
I will probably implement the same thing as lakectl local in python as I don't know if you guys intend to implement it. But I do want my implementation to be compatible with the cli
y

Yoni Augarten

09/06/2023, 8:01 AM
I think maybe as a first step try out
lakectl local
and see that it fits your use case, WDYT?
šŸ‘ 1
h

HT

09/06/2023, 8:02 AM
So are you creating a hidden file to track where the data come from?
How do you track the origin of the data from a given folder?