hi, im trying to get talend to work with lakefs for scheduling my spark jobs , is there any way that i can integrate this
i
Idan Novogroder
07/19/2022, 7:22 AM
Hi Rahul,
By scheduling, do you mean triggering as a result of an action in lakeFS (commit, merger etc...)?
If your answer is yes, I guess hooks will be the right answer for you.
You can find more information in this section about hooks: https://docs.lakefs.io/setup/hooks.html
You can find information about lakeFS integration with Spark in here: https://docs.lakefs.io/integrations/spark.html