Hi everyone I'm new to lakefs, and I have some problem running python-client quickstart. I deployed lakefs on kubernetes with minio as object store and postgres as metadata store. When I try to create a repo using python:

repo = models.RepositoryCreation(name='example-repo', storage_namespace='<s3://my-bucket/example-repo>', default_branch='main')

The request fails with error message:

HTTP response body: {"message":"failed to create repository: failed to access storage"}

And the log of lakefs pod shows:

time="2023-01-13T01:56:54Z" level=error msg="failed to get S3 object bucket my-bucket key example-repo/dummy" func="pkg/logging.(*logrusEntryWrapper).Errorf" file="build/pkg/logging/logger.go:258" error="RequestError: send request failed\ncaused by: Get \"<http://my-bucket.minio-service.kubeflow.svc.cluster.local:9000/example-repo/dummy>\": dial tcp: lookup my-bucket.minio-service.kubeflow.svc.cluster.local: no such host" host="localhost:8000" method=POST operation=GetObject operation_id=CreateRepository path=/api/v1/repositories request_id=38ff0f56-99a6-4f4a-b6bd-bba8bf7e7c87 service_name=rest_api user=admin
time="2023-01-13T01:56:54Z" level=warning msg="Could not access storage namespace" func="pkg/api.(*Controller).CreateRepository" file="build/pkg/api/controller.go:1393" error="RequestError: send request failed\ncaused by: Get \"<http://my-bucket.minio-service.kubeflow.svc.cluster.local:9000/example-repo/dummy>\": dial tcp: lookup my-bucket.minio-service.kubeflow.svc.cluster.local: no such host" reason=unknown service=api_gateway storage_namespace="<s3://my-bucket/example-repo>"

It seems that 'my-bucket' was parsed as a prefix of my minio endpoint. Here's my lakefs deploy configuration:

blockstore:
  type: s3
  s3:
    endpoint: <http://minio-service.kubeflow.svc.cluster.local:9000>
    credentials:
      access_key_id: access_key_id
      secret_access_key: secret_access_key
    discover_bucket_region: false

Hi @edwardlol Zhou, Sorry to hear you've been having difficulties. It sounds like right now your lakeFS installation will not work, regardless of whether you use Python or access it directly from the web UI or lakectl. Let me try to help!

Yes I also tried to create repo on web UI and the same error occurs

Now the S3 protocol has two ways to address a bucket: host style and path style. It looks like host style, the default, is failing on your setup - it requires a tricky DNS configuration. Rather than fix that, I suggest you try path-style. The manual for integrating with MinIO has the additional line

force_path_style: true

in blockstore.s3. Could you try to add that to your configuration, and then restart lakeFS?

Ooh, just figured out that the on-prem storage integration guide has this in the Kubernetes tab, too!

Yes, set both

force_path_style

to true and

discover_bucket_region

to false and now I can run the quickstart. Thanks!!

Yay! Welcome to the lake... sunglasses lakefs