https://lakefs.io/ logo
Title
c

Chandra Akkinepalli

03/22/2023, 7:54 PM
I am trying to deploy lakefs on kubernetes(openshift) using the quickstart documentation. The docs talk about postgres db, is it expected to have a pre-built postgres/dynamodb or will it be built by helm if not configured? ( so far have not been successful deploying)
g

Guy Hardonag

03/22/2023, 8:08 PM
Hi @Chandra Akkinepalli, Yes it’s expected that you have a prebuilt postgres/dynamoDB The lakeFS chart will not build it for you.
If it still doesn’t work, please share the values you provided
c

Chandra Akkinepalli

03/22/2023, 8:11 PM
Thanks @Guy Hardonag, i played with an earlier version of lakefs which had helm chart that installed postgres + lakefs pods , wanted to make sure thats not the case anymore.
g

Guy Hardonag

03/22/2023, 8:17 PM
Thats not the case anymore,But if not configured otherwise lakeFS should run with a local database implementation. It isn’t suggested for production use-case
y

Yoni Augarten

03/22/2023, 9:37 PM
Just to emphasize, the helm chart will deploy postgres for you if you don't override any helm values, but as Guy said this will save data locally and hence is not suitable for production. You can tweak it to use volumes to persist your data.
Sorry, I was looking at an outdated chart. Guy is right: without overriding any values, lakeFS runs in "local" mode, where metadata is saved locally in the lakeFS pod. Not suitable for production, but great for playing around. If it doesn't work, please let us know.
c

Chandra Akkinepalli

03/23/2023, 1:34 AM
unfortunately my Openshift environment doesnn't allow for adding any ephemeral data, its failing to create postgres db at /tmp . I have created a seperated postgres instance and gave access to lakefs, but the quickstart seems to be trying to create catalog metadata locally, i haven't found configuration that i can set to either direct it to a PVC or writable path.
something like this
time="2023-03-22T20:57:10Z" level=fatal msg="failed to create catalog" func=cmd/lakefs/cmd.glob..func8 file="cmd/run.go:163" error="create tiered FS for committed metaranges: creating base dir: /lakefs/data/cache/meta-range - mkdir /lakefs: permission denied"
i

Iddo Avneri

03/23/2023, 1:42 AM
Hi Chandra, as we continue looking into it - would it be useful to run the “local” mode as Yoni suggested? Want to try to “unblock” you.
c

Chandra Akkinepalli

03/23/2023, 1:52 AM
sure. I have already spun up lakefs using podman locally
i

Iddo Avneri

03/23/2023, 1:55 AM
That’s good. Thanks for the update.
c

Chandra Akkinepalli

03/24/2023, 7:05 PM
Any thoughts on the above error, is there a config i am missing ?
i

Iddo Avneri

03/24/2023, 7:10 PM
Hi Chendra, not yet. We should have something start of next week.
👍 1
y

Yoni Augarten

03/28/2023, 12:34 PM
Hey @Chandra Akkinepalli, sorry for the delay in response. The relevant configuration is:
committed.local_cache.dir
You can set it to a writable path.
c

Chandra Akkinepalli

03/28/2023, 3:54 PM
Thank You @Yoni Augarten. Yes, i was able to do that get it working over the weekend. Appreciate you getting back to me on this.
👌🏻 1
Do you know if intra application communications can be SSL encrypted? I have seen a post sometime back that it was not possible and the SSL connection has to be terminated at the Load balancer?
y

Yoni Augarten

03/28/2023, 3:57 PM
@Chandra Akkinepalli hi again! Glad you got it working. What do you mean by intra-application communication? What are the components you are referring to?
Do you mean between the load balancer and lakeFS?
c

Chandra Akkinepalli

03/28/2023, 3:58 PM
yes.
y

Yoni Augarten

03/28/2023, 4:02 PM
The lakeFS code does not perform SSL termination. To my knowledge, most software today leaves this task to external component, like a reverse proxy or a load balancer. However, since I'm not a networking expert, could you please explain the motivation for this? I will consult with the team.
c

Chandra Akkinepalli

03/28/2023, 5:09 PM
What i am tryin got understand is , if there at point of data transfer from/to underlying Object store is the data unencrypted?
y

Yoni Augarten

03/28/2023, 5:21 PM
The communication with the object store is done using the standard clients (for example AWS Golang SDK for S3) - hence the communication is encrypted. The only thing not encrypted is the communication between your load balancer and lakeFS.