Q: How do you guys map a particular git commit to commits on datasources in lakeFS? Would be nice to have clear references in the respective git repo.

Hey Leonard! Linking between git commits and lakeFS commits is possible using the commit metadata (See our Commit object in the API Reference). The

metadata

field is where the committer can store any custom user metadata, like the matching git commit hash. Do you have in mind any specific use-case for linking the git repo with a lakeFS repo?

Ok thanks. We will have MLOps CI and versioning of orchestrated experiments and ML pipelines. the versioning is very much mapped to the git commit hash and branching. We would need to know that for a particular commit of code to git repo. what are the specific related versions of the datasets that has been used.

The trivial solution would be to add some sort of a

lakefs.commit

file in your git repo, or structure the git commit message in a way that the lakeFS commit can be extracted from it. I opened an issue to explore a way to automate this process for a better user experience.

yeah, we where thinking of doing something like this exactly.