Heya Axolotls :levitating-lakefs:
I am looking for a way to improve lakeFS documentation. One of the criteria is a reference tool.
For example, when updating a specific API, it will alert us on a tutorial that uses this API that requires us to update it as well.
Are you familiar with such a tool? any suggestions?
Also, which documentation needs improvement asap? 💡
03/08/2022, 2:04 PM
Not super familiar with a reference tool like you are describing, but if I could suggest creating terraform scripts for the various cloud platforms, that would be incredibly useful. I know for myself, I spent quite a bit of time creating one for AWS.
03/08/2022, 6:29 PM
Hey! Thanxalotl:levitating-lakefs:! What an interesting idea. I'm always cautious about these things, especially from a security perspective, because each architecture is different and has different requirements. I'd love to hear how you went about it!
03/10/2022, 6:20 PM
I think we need more "recipe" style guides with code examples on how to go about the main use cases. A few examples:
• Using Spark + DBT to run data quality checks in isolation
• Schema validation using Webhooks/Airflow hooks
• using tagging with Jupyter notebooks for reproducible experiments
I think code examples that are easy to copy/paste and modify are much much easier than a from-scratch reference, and atm we mostly have the later
03/20/2022, 7:19 AM
I like these step-by-step guides by Upsolver, It’s a nice reference for a friendly recipes