Thread
#data-architecture-discussion
    Yoni Augarten

    Yoni Augarten

    1 month ago
    Hey @Comte Frédéric! Please see my answer in this thread. As I mentioned there, we are actively looking to improve the lakeFS+metastore experience. So I'm happy to discuss the use case with you and see how lakeFS can improve.
    c

    Comte Frédéric

    1 month ago
    Thanks, I don't know if lakeFS can improve. I understand the choice to only link hive-metastore only with the client side of lakefs (lakectl) but It's quite frustrating to force our user to manage that manually
    Yoni Augarten

    Yoni Augarten

    1 month ago
    I agree, we have some thoughts about automating this process. Namely, have these commands run automatically when creating a branch and when merging back. Do you think this can bring you value?
    c

    Comte Frédéric

    1 month ago
    do you mean having this feature running server side?
    Yoni Augarten

    Yoni Augarten

    1 month ago
    Yes, and automated
    c

    Comte Frédéric

    1 month ago
    I think this feature can be interesting
    I am not yet a lakefs user , i am just an administrator of a dataplatform and I wonder how I can offer some service to my users based on lakefs
    my dataplatform is based on kubernetes and s3
    Yoni Augarten

    Yoni Augarten

    1 month ago
    @Comte Frédéric, sounds interesting. lakeFS regularly publishes a Helm chart that can easily be installed on your Kubernetes cluster. lakeFS supports S3 as its underlying storage - this is the storage used by the majority of our users. Moreover, lakeFS exposes an S3-compatible API allowing you to access your data using your existing tools. I'm happy to hear more about your use case, so feel free to add more details 🙂.
    In fact I let my users deploy their own lakefs thanks to your charts
    ( a subchart of mine )
    This chart is automacally deployed with S3 keys from the users thanks to my platform ( opensource soft called onyxia , the one we develop )
    But lakeFS is just a choice for our users and not a platform choice. Onyxia do not use lakeFS to do some intelligent thinks for the users
    (I am not really clear)
    https://datalab.sspcloud.fr if you want to see it
    For example you can see that on my s3 browser , that lakefs is working