https://lakefs.io/ logo
#help
Title
# help
r

Raphaël

08/10/2023, 2:43 PM
Hi all, I tried to deploy locally lakefs with compose, the code is below :
Copy code
version: '3'
services:
  lakefs:
    image: "treeverse/lakefs:${VERSION:-latest}"
    ports:
      - "8000:8000"
    environment:
      - LAKEFS_AUTH_ENCRYPT_SECRET_KEY=${LAKEFS_AUTH_ENCRYPT_SECRET_KEY:-some random secret string}
      - LAKEFS_DATABASE_TYPE=${LAKEFS_DATABASE_TYPE:-postgres}
      - LAKEFS_DATABASE_POSTGRES_CONNECTION_STRING=${LAKEFS_DATABASE_POSTGRES_CONNECTION_STRING:-<postgres://myuser:mypass@postgres/mydb?sslmode=disable>}
      - LAKEFS_BLOCKSTORE_TYPE=${LAKEFS_BLOCKSTORE_TYPE:-s3}
      - LAKEFS_BLOCKSTORE_S3_DISCOVER_BUCKET_REGION=${LAKEFS_BLOCKSTORE_S3_DISCOVER_BUCKET_REGION:-false}
      - LAKEFS_BLOCKSTORE_S3_ENDPOINT=${LAKEFS_BLOCKSTORE_S3_ENDPOINT:-<http://minio-api.dmx.geodatacenter.eu>}
      - LAKEFS_BLOCKSTORE_S3_FORCE_PATH_STYLE=${LAKEFS_BLOCKSTORE_S3_FORCE_PATH_STYLE:-true}
      - LAKEFS_BLOCKSTORE_S3_CREDENTIALS_ACCESS_KEY_ID=${MINIO_ROOT_USER:-dev}
      - LAKEFS_BLOCKSTORE_S3_CREDENTIALS_SECRET_ACCESS_KEY=${MINIO_ROOT_PASSWORD:-secure123}
      - LAKEFS_LOGGING_LEVEL=${LAKEFS_LOGGING_LEVEL:-WARNING}
      - LAKEFS_COMMITTED_LOCAL_CACHE_DIR=${LAKEFS_COMMITTED_LOCAL_CACHE_DIR:-/home/lakefs/.local_tier}
    entrypoint: ["/app/lakefs", "run"]
I used a minio which is located on a kube. I don't have problem to access it using s3cmd or tools like this but here I've the following error about a signature :
Copy code
lakefs_1    | time="2023-08-10T14:16:51Z" level=warning msg="Could not access storage namespace" func="pkg/api.(*Controller).CreateRepository" file="build/pkg/api/controller.go:1567" error="s3 error: <?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<Error><Code>SignatureDoesNotMatch</Code><Message>The request signature we calculated does not match the signature you provided. Check your key and signing method.</Message><Key>dummy</Key><BucketName>lakefs</BucketName><Resource>/lakefs/dummy</Resource><RequestId>177A0B5ADA59970E</RequestId><HostId>dd9025bab4ad464b049177c95eb6ebf374d3b3fd1af9251148b658df7ac2e3e8</HostId></Error>" reason=unknown service=api_gateway storage_namespace="<s3://lakefs>"
I have checked my credentials and there are correct. Anyone have an idea to fix it ? Thanks !
i

Iddo Avneri

08/10/2023, 2:57 PM
Hi and welcome to the lake @Raphaël I’m not exactly answering the qeustion, but perhaps this can be useful. Did you try cloning the sample repository -
Copy code
git clone <https://github.com/treeverse/lakeFS-samples.git>
cd lakeFS-samples
And running lakeFS with MinIO:
Copy code
docker compose --profile local-lakefs up
Not sure where you are in your journey, but this can unstuck you at least potentially?
r

Raphaël

08/10/2023, 3:01 PM
Thanks for answering but I want to use my minio bucket in particular :)
y

Yoni Augarten

08/10/2023, 3:13 PM
Hey @Raphaël, I understand you are using lakeFS on top of MinIO and getting an error message when trying to create a repository. Can you confirm that when trying to create the repository, a request from lakeFS reaches your MinIO installation? You can verify that, for example, by observing the "Trace" tab in the MinIO UI. I just want to make sure that this error is indeed returned by MinIO.
t

Thomas

08/11/2023, 7:51 AM
I don't find the trace tab on the UI, can you give me more details please
y

Yoni Augarten

08/11/2023, 7:55 AM
Hey @Thomas, do you have any way to log requests on the minio side? I'm not a minio expert myself
t

Thomas

08/11/2023, 8:14 AM
Actually I don't have logs about minio but If I tried to communicate with the bucket using boto3 it works so I think my minio installation is ok
y

Yoni Augarten

08/11/2023, 8:15 AM
I want to make sure that lakeFS is configured properly to talk to minio.
t

Thomas

08/11/2023, 9:12 AM
But I don't have minio logs, do you have any other solution to check ?
j

Jonathan Rosenberg

08/11/2023, 9:26 AM
Hi Do you also run Minio using Docker compose? Can you share your Minio configurations as well?
r

Raphaël

08/11/2023, 11:03 AM
Hi, no minio is is on kube so it was deployed using Helm. I send you the deployment file.
j

Jonathan Rosenberg

08/11/2023, 11:04 AM
Thanks I’ll look into it
r

Raphaël

08/11/2023, 11:33 AM
signatureDoesNotMatch
error can be caused by a mismatch of dates ? @Jonathan Rosenberg
j

Jonathan Rosenberg

08/11/2023, 11:34 AM
Possibly It can be caused by a number of factors
r

Raphaël

08/11/2023, 11:52 AM
I've exactly this error (foud on aws doc) but I don't really know how to fix it...
j

Jonathan Rosenberg

08/11/2023, 12:03 PM
These two seem to not match:
Copy code
- LAKEFS_BLOCKSTORE_S3_CREDENTIALS_ACCESS_KEY_ID=${MINIO_ROOT_USER:-dev}
- LAKEFS_BLOCKSTORE_S3_CREDENTIALS_SECRET_ACCESS_KEY=${MINIO_ROOT_PASSWORD:-secure123}
and
Copy code
- name: MINIO_ACCESS_KEY
  value: minio
- name: MINIO_SECRET_KEY
  value: minio123
t

Thomas

08/11/2023, 12:12 PM
The first credentials are those I used to connect on the minio ui. But I tested with the others mentionned in the chart and the error message is exactly the same about the signature.
26 Views