Hello! I was wondering if it's possible to setup a hook that will execute a Rust binary when data is modified. The binary will operate on the data e.g. calculating MD5 of the data

Hey @Jay, when it comes to triggering of actions, lakeFS currently supports the following events. Triggering an action for every object change is quite costly, so I’d like to learn more about your use-case to see how it fits to lakeFS actions. For the execution itself, running a standalone binary isn’t supported. You can either configure a webhook and run your workload on a designated server, or write your logic as a Lua script to be executed in lakeFS itself. I hope this makes sense..

The first use case is that we'd like to compute a digital signature on the state of the new data -- like every time a new file is uploaded. It sounds like the best way to do that is with lua, using the

crypto/hmac/sign_sha256(message, key)

function. Is there an example of what the argument to the sign_sha256 call should be to refer to the files in the repository or, better, to refer to a hash that captures the state of the repository after a new version is committed?

Here is an example: https://github.com/treeverse/lakeFS-samples/blob/main/00_notebooks/hooks-schema-and-pii-validation.ipynb and 2 Lua scripts: https://github.com/treeverse/lakeFS-samples/tree/main/00_notebooks/hooks

@Jay out of curiosity, is there a special meaning to calculating the md5 sum of the entire data? Are you doing it for a specific prefix on the branch? Would relying on the commit hash not be sufficient?