Ronnie Ning
11/23/2022, 6:28 PMlakefs
under the current user. Can we customize where lakefs
can be?Fizza Abid
11/25/2022, 8:12 AMCan only create repository with storage type: local
, how should I give s3 path?Walter Johnson
11/29/2022, 7:44 PMOmkar Patil
11/30/2022, 1:01 PMRonnie Ning
12/01/2022, 9:35 PMAlessandro Mangone
12/05/2022, 12:30 PMSelva
12/06/2022, 5:20 AMAlessandro Mangone
12/06/2022, 10:59 AMFizza Abid
12/06/2022, 12:46 PMgateways:
s3:
domain_name: <http://s3.lakefs.example.com|s3.lakefs.example.com>
Raphaël
12/20/2022, 4:20 PMRaphaël
12/22/2022, 2:17 PMC. bon
12/28/2022, 9:20 PMedwardlol Zhou
01/13/2023, 2:03 AMrepo = models.RepositoryCreation(name='example-repo', storage_namespace='<s3://my-bucket/example-repo>', default_branch='main')
The request fails with error message:
HTTP response body: {"message":"failed to create repository: failed to access storage"}
And the log of lakefs pod shows:
time="2023-01-13T01:56:54Z" level=error msg="failed to get S3 object bucket my-bucket key example-repo/dummy" func="pkg/logging.(*logrusEntryWrapper).Errorf" file="build/pkg/logging/logger.go:258" error="RequestError: send request failed\ncaused by: Get \"<http://my-bucket.minio-service.kubeflow.svc.cluster.local:9000/example-repo/dummy>\": dial tcp: lookup my-bucket.minio-service.kubeflow.svc.cluster.local: no such host" host="localhost:8000" method=POST operation=GetObject operation_id=CreateRepository path=/api/v1/repositories request_id=38ff0f56-99a6-4f4a-b6bd-bba8bf7e7c87 service_name=rest_api user=admin
time="2023-01-13T01:56:54Z" level=warning msg="Could not access storage namespace" func="pkg/api.(*Controller).CreateRepository" file="build/pkg/api/controller.go:1393" error="RequestError: send request failed\ncaused by: Get \"<http://my-bucket.minio-service.kubeflow.svc.cluster.local:9000/example-repo/dummy>\": dial tcp: lookup my-bucket.minio-service.kubeflow.svc.cluster.local: no such host" reason=unknown service=api_gateway storage_namespace="<s3://my-bucket/example-repo>"
It seems that 'my-bucket' was parsed as a prefix of my minio endpoint.
Here's my lakefs deploy configuration:
blockstore:
type: s3
s3:
endpoint: <http://minio-service.kubeflow.svc.cluster.local:9000>
credentials:
access_key_id: access_key_id
secret_access_key: secret_access_key
discover_bucket_region: false
Beegee Alop
01/15/2023, 1:05 PMAapta Bhatt
01/16/2023, 12:02 PMOmkar Patil
01/20/2023, 6:46 AMConor Simmons
01/27/2023, 4:55 PMConor Simmons
01/31/2023, 3:03 PMConor Simmons
01/31/2023, 6:38 PMAnna Schooneveld
02/17/2023, 5:34 PMAnna Schooneveld
02/17/2023, 5:35 PMPaul
02/27/2023, 2:18 PMRobert Angeli
02/28/2023, 4:32 PMRobert Angeli
02/28/2023, 4:32 PMRobin Moffatt
03/01/2023, 9:59 AMgit add
, is that right? Once I've written files to a branch, I either commit them all, or none? e.g. here I can't just commit sensitive/flyers.csv
and leave the drone-registrations/*
uncommitted?
diff <lakefs://drones03/main>
Ref: <lakefs://drones03/main>
+ added drone-registrations/Registations-P107-Active-2016.parquet
+ added drone-registrations/Registations-P107-Active-2017.parquet
+ added drone-registrations/Registations-P107-Active-2018.parquet
+ added drone-registrations/Registations-P107-Active-2019.parquet
+ added drone-registrations/Registations-RecFlyer-Active-2020.parquet
+ added drone-registrations/Registations-RecFlyer-Active-2021.parquet
+ added sensitive/flyers.csv
Anna Schooneveld
03/01/2023, 2:56 PMMohammad Eslami
03/05/2023, 5:18 PMBrad
03/10/2023, 3:23 AMAviator
03/14/2023, 8:47 PMAviator
03/14/2023, 8:49 PMAviator
03/14/2023, 8:49 PMGuy Hardonag
03/14/2023, 9:21 PMRobin Moffatt
03/14/2023, 10:02 PMAviator
03/15/2023, 6:25 AM