• c

    Comte Frédéric

    1 month ago
    Does some lakefs user use hive-metastore too ? Do you create table which refer to a repository and a branch ?
    c
    Iddo Avneri
    6 replies
    Copy to Clipboard
  • Yoni Augarten

    Yoni Augarten

    1 month ago
    Hey @Comte Frédéric! Please see my answer in this thread. As I mentioned there, we are actively looking to improve the lakeFS+metastore experience. So I'm happy to discuss the use case with you and see how lakeFS can improve.
    Yoni Augarten
    c
    16 replies
    Copy to Clipboard
  • c

    Comte Frédéric

    1 month ago
    I mean having an endpoint to get temporary access to lakeFS with oidc ?
    c
    Adi Polak
    +1
    10 replies
    Copy to Clipboard
  • f

    Farman Pirzada

    1 month ago
    I have a question about LakeFS in general. If one were to utilize this tool for machine learning, how would it differentiate from what MLFlow offers? I came across this great article: https://pub.towardsai.net/data-versioning-for-efficient-workflows-with-mlflow-and-lakefs-892df1f8e7d8 and wanted to learn more. I realize data versioning and model lifecycles are different, but MLFlow looks like it can be used as a suite of tools too.
  • einat.orr

    einat.orr

    1 month ago
    My 2 cents are that MLflow is a logger, with a model registery and other functionality based on it's logging capabilities. lakeFS works on a lower level so to speak, as it actually versions the data in the data lake, ensuring versions are kept even when file/table names stay the same. The protection lakeFS versioning provide is preliminary to the vale of MLflow, that can naturally come on top.
  • einat.orr

    einat.orr

    1 month ago
    In addition, lakeFS allows the git abstractions such as branching, committing and merging, that help day to day lifecycle management of the data.
  • einat.orr

    einat.orr

    1 month ago
    By letting MLflow log lakeFS paths you get a very robust environment.
  • einat.orr

    einat.orr

    1 month ago
    I hope this helps.
  • f

    Farman Pirzada

    1 month ago
    thank you so much for this!