Hello All, I was wondering if there is a golang c...
# help
u
Hello All, I was wondering if there is a golang client for LakeFS .. Here is a little bit more information regarding our use case: • We are trying to use LakeFS as a foundation for our MLOps thinking for our platform (Versioned data sets will solve a lot of problems for us in the near future). • We would also like to support Multi-Cloud capability (as our customers can have the flexibility of the cloud provider) . We have our data pipelines written in GO ... • One of the workflows in our data pipelines is to ingest data from pulsar and generate them as parquet files into a particular repository in Lakefs. I see there is a python client which supports object uploads as a first class API... Is there a client SDK for go ? I would like to upload the parquet files via the LakeFS api as opposed to importing from S3 as that might be cleaner approach for multi-cloud support ... Could you please let me know if HTTP API is the only pathway for golang clients or you would recommend any other approach for our use case?
u
Hi @Ananth Gundabattula and welcome! :) As for the moment there is no golang client for lakeFS. I believe that our API is your best solution right now, but let me check if we there is another one on our roadmap. I'll get back to you ASAP.
u
Thanks @Lynn Rozen
u
Hi @Ananth Gundabattula, As Lynn said, we don't "publish" the generated Go client code anywhere in the sense of checking it in. However we do of course generate and use it in the code. This line https://github.com/treeverse/lakeFS/blob/d83d9b6b3a665311e355a55974e5775dd6865d78/pkg/api/serve.go#L3 generates it; you could do the same. The cli is a big user of the client code; look under https://github.com/treeverse/lakeFS/tree/a2103c366a075fc4497695fc9e073b10adf82dfc/cmd/lakectl/cmd for many examples of how we use it. I am not sure that the user effort for using a published generated code in Go is much different from the effort to generate and then use it. If you disagree, please let's have the discussion on an issue!
u
Thanks @Ariel Shaqed (Scolnicov) . The above links gives me a pathway. I will try the above and get back to you if I hit any issues.