Hi there :wave: I'm wondering if lakeFS would supp...
# help
l
Hi there 👋 I'm wondering if lakeFS would support sth like fuse-zip, or google mount-zip ? I want to copy some files inside a large .ZIP file stored by LakeFS into my local machines. I'd be such a burden to download that chunky 10GB file for just some 1MB documents.
o
Hey @Lâm Nguyễn (Nicholas)! no fuse/mount support for zip files, however, there's a simpler way! Take a look at cloudzip - it works with lakeFS and allows downloading specific files from a zip archive without having to download the whole thing
Fun fact, I wrote it in order to support this exact use case a very short while ago 🙂 If you have some time, I'd love to hear your feedback/thoughts, as I find this use case fascinating 😊
jumping lakefs 3
l
that’s a really helpful quick response! thanks! I’m gonna look into it!
h
I am trying to use cloudzip with our self-hosted lakefs server in Azure @Oz Katz how to do you tell it which server to use ? the endpoint server ?
o
it will attempt to read the standard lakectl config from environment variables (
LAKECTL_ENDPOINT_URL
,
LAKECTL_ACCESS_KEY_ID
and
LAKECTL_SECRET_ACCESS_KEY
). If not set, it will attempt to read these values from a
~/.lakectl.yaml
config file
👍 1
h
I don;t use lakectl so I don;t have that set 😛
Thanks
o
ah got it. which clients do you typically use?
h
s3fs
python sdk
and because lakectl is go, I cannot "integrate" it nicely ....
and I use rclone when I need to do file transfer
o
thanks!