Hi team, please is there is a way to read data from LakeFS using lakefs_client just the same way it can be used to upload file to LakeFS storage. I am trying to run an ML pipeline on kubeflow

Hey Jude, welcome to lakeFS! By

lakefs_client

do you mean the python generated client of lakeFS? If so, just like uploading a file, there’s a way to download it. If you’ll kindly share more details about your flow and use-case, I’ll be more than happy to help you find the matching command.

Exactly that, how I can download a file using python generated client from lakefs

client.get_object(repository, ref, path)

should get you what you need 🙂

ref

is either the

branch

or the

commit

.

Thank you!

NP

@Itai Admi when I ran the command I got this error

AttributeError: 'LakeFSClient' object has no attribute 'get_object'

I think the exact command is

client.objects.get_object

Okay let me try that now. One more thing, lets say you already have a spark process running on kubeflow I believe there is a way that can also be configured to read and write data to lakefs right?

it works!

Yap, there are several ways to read/write data from lakeFS using Sparks, these docs sums it up pretty nicely.