Hi team I tried to deploy lakefs using helm onto k8s cluster

Question

Hi team I tried to deploy lakefs using helm onto k8s cluster But the pod failed on live probe and read probe with error ` dial tcp 172 19 8000 connect connection refused ` The last messages I found in the pod are ```time= 2023 10 18T06 26 08Z level=info msg= lakeFS run func=cmd lakefs cmd glob func8 file= cmd run go 91 version=0 113 0 time= 2023 10 18T06 26 08Z level=info msg= initialized Auth service func=pkg auth NewAuthService file= build pkg auth service go 187 service=auth service time= 2023 10 18T06 26 08Z level=info msg= initialize blockstore adapter func=pkg block factory BuildBlockAdapter file= build pkg block factory build go 32 type=local time= 2023 10 18T06 26 08Z level=info msg= initialized blockstore adapter func=pkg block factory buildLocalAdapter file= build pkg block factory build go 79 path= home lakefs lakefs data block type=local``` I don t find any message like ` Initial setup completed successfully ` and ` starting HTTP server ` in the log Here is my helm cmd `helm install example lakefs amd64 lakefs lakefs version 0 13 3 f lakefs config helm amd64 local yaml`and the config file ```image repository <http docker io treeverse lakefs|docker io treeverse lakefs> pullPolicy IfNotPresent Keys used for existingSecret secrets authEncryptSecretKey 123 lakefsConfig | database type local blockstore type local ``` Any wrong setting Thanks

gang ye · Answer

Or is there any debug mode we can enable to see where it gets stuck which causes the HTTP server not running as expected

Jonathan Rosenberg · Answer

Hi < gang ye> Would you mind trying to run it without the `lakefsConfig` and `image` sections These are the default values anyway

gang ye · Answer

Hi < Jonathan Rosenberg> Ideally the image config should not impact Our k8s cluster can only connect with internal docker lib So we would need to set it

Jonathan Rosenberg · Answer

Hi again I used your configurations without the quotes that surround local and it worked Can you make sure that the lakeFS pod is actually running

gang ye · Answer

I tried still same No log message saying `starting HTTP server`after ```time= 2023 10 18T19 50 47Z level=info msg= initialized Auth service func=pkg auth NewAuthService file= build pkg auth service go 187 service=auth service time= 2023 10 18T19 50 47Z level=info msg= initialize blockstore adapter func=pkg block factory BuildBlockAdapter file= build pkg block factory build go 32 type=local time= 2023 10 18T19 50 47Z level=info msg= initialized blockstore adapter func=pkg block factory buildLocalAdapter file= build pkg block factory build go 79 path= home lakefs lakefs data block type=local``` and the read probe and write probe refuse connection

gang ye · Answer

I enabled trace log here is the log ```time= 2023 10 18T20 09 03Z level=info msg= lakeFS run func=cmd lakefs cmd glob func8 file= cmd run go 91 version=0 113 0 time= 2023 10 18T20 09 03Z level=info msg= initialized Auth service func=pkg auth NewAuthService file= build pkg auth service go 187 service=auth service time= 2023 10 18T20 09 03Z level=debug msg= failed to collect account metadata func=pkg stats NewMetadata file= build pkg stats metadata go 34 error= not found time= 2023 10 18T20 09 03Z level=info msg= initialize blockstore adapter func=pkg block factory BuildBlockAdapter file= build pkg block factory build go 32 type=local time= 2023 10 18T20 09 03Z level=info msg= initialized blockstore adapter func=pkg block factory buildLocalAdapter file= build pkg block factory build go 79 path= home lakefs lakefs data block type=local time= 2023 10 18T20 09 03Z level=trace msg= DEBUG POST func= pkg logging logrusEntryWrapper Tracef file= build pkg logging logger go 264 service=stats collector time= 2023 10 18T20 09 33Z level=trace msg= ERR POST request failed Post dial tcp 54 175 118 214 443 i o timeout func= pkg logging logrusEntryWrapper Tracef file= build pkg logging logger go 264 service=stats collector time= 2023 10 18T20 09 33Z level=trace msg= DEBUG POST retrying in 1s 2 left func= pkg logging logrusEntryWrapper Tracef file= build pkg logging logger go 264 service=stats collector```

gang ye · Answer

after I disabled stats then it works I guess it s related to our internal K8s cluster cannot connect with lakefs stats

gang ye · Answer

Hi < Jonathan Rosenberg> One last question after running ```kubectl port forward $POD NAME 8000 8000 namespace default``` how to access the local port to create repo Is this the right link

Amit Kesarwani · Answer

Try setup or

Iddo Avneri · Answer

< gang ye> Is your environment now working correctly

gang ye · Answer

it s working as expected Thanks