What is the recommended way to handle multiple data sources that update at different frequencies? Say we have datasets D1, D2, D3 ... each that update at different times (say monthly, weekly, yearly etc.). Each dataset may contain multiple CSV files.
Should each of them have their own branch or should they be just folders within a single branch or some other better way to handle them?
Ultimately would like to have the
main
branch always point to the latest version of all these datasets. What is the best way to organize this with LakeFS?