I am trying to deploy lakefs on kubernetes(openshift) using the quickstart documentation. The docs talk about postgres db, is it expected to have a pre-built postgres/dynamodb or will it be built by helm if not configured? ( so far have not been successful deploying)

Hi @Chandra Akkinepalli, Yes it’s expected that you have a prebuilt postgres/dynamoDB The lakeFS chart will not build it for you.

Adding a link to the README https://github.com/treeverse/charts/tree/master/charts/lakefs

If it still doesn’t work, please share the values you provided

Thanks @Guy Hardonag, i played with an earlier version of lakefs which had helm chart that installed postgres + lakefs pods , wanted to make sure thats not the case anymore.

Thats not the case anymore,But if not configured otherwise lakeFS should run with a local database implementation. It isn’t suggested for production use-case

Sorry, I was looking at an outdated chart. Guy is right: without overriding any values, lakeFS runs in "local" mode, where metadata is saved locally in the lakeFS pod. Not suitable for production, but great for playing around. If it doesn't work, please let us know.

unfortunately my Openshift environment doesnn't allow for adding any ephemeral data, its failing to create postgres db at /tmp . I have created a seperated postgres instance and gave access to lakefs, but the quickstart seems to be trying to create catalog metadata locally, i haven't found configuration that i can set to either direct it to a PVC or writable path.

something like this

time="2023-03-22T20:57:10Z" level=fatal msg="failed to create catalog" func=cmd/lakefs/cmd.glob..func8 file="cmd/run.go:163" error="create tiered FS for committed metaranges: creating base dir: /lakefs/data/cache/meta-range - mkdir /lakefs: permission denied"

Hi Chandra, as we continue looking into it - would it be useful to run the “local” mode as Yoni suggested? Want to try to “unblock” you.

sure. I have already spun up lakefs using podman locally

That’s good. Thanks for the update.

Any thoughts on the above error, is there a config i am missing ?

Hi Chendra, not yet. We should have something start of next week.

Hey @Chandra Akkinepalli, sorry for the delay in response. The relevant configuration is:

committed.local_cache.dir

You can set it to a writable path.

Thank You @Yoni Augarten. Yes, i was able to do that get it working over the weekend. Appreciate you getting back to me on this.

Do you know if intra application communications can be SSL encrypted? I have seen a post sometime back that it was not possible and the SSL connection has to be terminated at the Load balancer?

@Chandra Akkinepalli hi again! Glad you got it working. What do you mean by intra-application communication? What are the components you are referring to?

Do you mean between the load balancer and lakeFS?

yes.

The lakeFS code does not perform SSL termination. To my knowledge, most software today leaves this task to external component, like a reverse proxy or a load balancer. However, since I'm not a networking expert, could you please explain the motivation for this? I will consult with the team.

What i am tryin got understand is , if there at point of data transfer from/to underlying Object store is the data unencrypted?

The communication with the object store is done using the standard clients (for example AWS Golang SDK for S3) - hence the communication is encrypted. The only thing not encrypted is the communication between your load balancer and lakeFS.