• d

    donald

    5 months ago
    I was running lakefs binary directly by configured lakefs to my local postgres database. Now I am trying to run lakefs docker by using docker-compose. First I run "docker-compose up", then I went to web UI to do some initial setup, created a new repository and uploaded some files. Then I run "docker-compose down" to stop lakefs. Then I run "docker-compose up" again, the lakefs started, but I have to do initial setup again and cannot see my previous setup and repository. Does anyone know how lakef keeps the previous setup/repository after stop and restart lakefs docker?
    d
    Oz Katz
    3 replies
    Copy to Clipboard
  • Bjorn Olsen

    Bjorn Olsen

    5 months ago
    Hey all. I've just finished a draft write-up on my time playing with LakeFS. Would appreciate a review and some feedback, before I publish 🙂 Especially where I've marked "TODO review" No rush.https://medium.com/p/1237f73d4a1c
    Bjorn Olsen
    einat.orr
    +1
    5 replies
    Copy to Clipboard
  • Edmondo Porcu

    Edmondo Porcu

    5 months ago
    Has anyone tried lakefs with Delta Live Tables on Databricks?
    Edmondo Porcu
    Paul Singman
    3 replies
    Copy to Clipboard
  • Evandre Da Silva

    Evandre Da Silva

    5 months ago
    Hi everyone! I'm starting with LakeFS and I have a quetion: What is the best way to encrypt all objects on lakeFS using linux + docker?
    Evandre Da Silva
    Oz Katz
    2 replies
    Copy to Clipboard
  • t

    Temilola Onaneye

    5 months ago
    Hi everyone, is there anyone using LakeFS with AWS and AZURE. Please I have some questions1. would you recommend LakeFS on either of the cloud provider over the other in terms of performance 2. Is there any noticeable difference in performance between Scala and Pyspark when using Lakefs
    t
    Ariel Shaqed (Scolnicov)
    3 replies
    Copy to Clipboard
  • Xubo Fei

    Xubo Fei

    5 months ago
    does lakeFS support okta?
    Xubo Fei
    1 replies
    Copy to Clipboard
  • Pooja Gattamaneni

    Pooja Gattamaneni

    4 months ago
    Hi @einat.orr, I have seen a thread where you were planning to eliminate RDS altogether. So I want to LakeFS to version the datafiles we will be uploading to S3 buckets. Is there a way we can d it without using RDS. Any suggestions would be much appreciated.
    Pooja Gattamaneni
    einat.orr
    +1
    3 replies
    Copy to Clipboard
  • 陳志傑

    陳志傑

    4 months ago
    Hello, As a beginner I install lakeFS on the local desktop. I have some naive questions uploading the data, I succeed by using aws cli, but failed with other methods ( lakectl, rclone). Bear with me to begin with a naive problem. For example, when I want to create a branch by using lakectl, the following command does not work: lakectl branch create lakefs://mls-identification/ingest/ -s lakefs://mls-identification/main/ The error message "Invalid branch: not a valid branch uri" appeared. However, when I remove the final '/' , i.e. lakectl branch create lakefs://mls-identification/ingest -s lakefs://mls-identification/main , it works well. Another issue is that, when I tried to use labelctl ingest to copy the data, I cannot make the right command. For example, I tried the following: lakectl ingest --from /home/ccchen/Documents/dr_hsieh/midline_final_label/final_label --to lakefs://mls-identification/main It showed the error message "Invalid 'to': not a valid path uri". On the other hand, when I slightly modified the address lakectl ingest --from /home/ccchen/Documents/dr_hsieh/midline_final_label/final_label --to lakefs://mls-identification/main/ This time the error message changed to "error creating object-store walker: no storage adapter found: for scheme:". I also tried to use rclone as follows: sudo rclone sync --config /home/ccchen/.config/rclone/rclone.conf /home/ccchen/Documents/dr_hsieh/midline_final_label/final_label lakefs://mls-identification/main There's no error message, but there's no uncommitted changes. Could anyone give me some suggestions? Thanks a lot!
    陳志傑
    Itai Admi
    3 replies
    Copy to Clipboard
  • Oz Katz

    Oz Katz

    3 months ago
    Hi <!everyone> 👋 We’ve been busy 🐝 at Treeverse working on launching our managed solution. Now we’re happy to share it with you before the rest of the world learns about it: Please welcome lakeFS Cloud! :lakefs: ☁️ lakeFS Cloud provides all the benefits of the Git-like version control for data lakes you all know and ❤️ in a fully-managed service. 🎉 Releasing lakeFS Cloud is a significant milestone 🚀 for Treeverse. While we are offering a Treeverse-managed lakeFS system, we are equally focused on the future of lakeFS as an open-source project. To learn more about it, read the lakeFS commitment to open-source. The core system, clients, and products used in the integrations will remain open-source and we will continue to develop them and add new features. On lakeFS Cloud, we offer best-in-class experience in operating, managing lakeFS, and providing easy “snap-in” integrations. Your participation in our journey, defining the data lifecycle management paradigm, and contributing to our success are greatly appreciated. I want to personally thank you all for taking an active part in our journey 🙏 . The collective knowledge of this community is just incredible 💫 🙌 I'm looking forward to continue collaborating with you all! As always, if you have any questions, please feel free to reach out to me 🙂
    Oz Katz
    Ankit Srinivas
    2 replies
    Copy to Clipboard
  • Ariel Shaqed (Scolnicov)

    Ariel Shaqed (Scolnicov)

    3 months ago
    Found on HN this comparison of Iceberg, Hudi, Delta: https://news.ycombinator.com/item?id=31723103. Best read with the comments, this space might be unpleasantly crowded soon. @Lynn Rozen @Yoni Augarten @Tal Sofer @Jonathan Rosenberg this lake is a large ecosystem 😃 !
    Ariel Shaqed (Scolnicov)
    Jonathan Rosenberg
    2 replies
    Copy to Clipboard