Hi Team I am trying set up lakefs with Minio following this doc I have created the below YAMl config under my lakefs folder. I am runing it as binary

./lakefs --config /Users/simar/lakeFS/cmd/lakefs/config.yaml run

but getting the below error

simar@192 lakeFS % ./lakefs --config /Users/simar/lakeFS/cmd/lakefs/config.yaml run

INFO[0000]/Users/simar/lakeFS/cmd/lakefs/cmd/root.go:80 <http://github.com/treeverse/lakefs/cmd/lakefs/cmd.initConfig()|github.com/treeverse/lakefs/cmd/lakefs/cmd.initConfig()> Configuration file              fields.file=/Users/simar/lakeFS/cmd/lakefs/config.yaml file=/Users/simar/lakeFS/cmd/lakefs/config.yaml phase=startup

{"error":"InvalidParameterValue: Unsupported action GetCallerIdentity\n\tstatus code: 400, request id: 175F5FE4DE5BD201","file":"pkg/cloud/aws/metadata.go:64","func":"pkg/cloud/aws.(*MetadataProvider).GetMetadata","level":"warning","msg":"Failed to to get AWS account ID for BI","time":"2023-05-15T22:33:42+05:30"}

{"file":"pkg/logging/logger.go:266","func":"pkg/logging.(*logrusEntryWrapper).Fatalf","level":"fatal","msg":"Mismatched adapter detected. lakeFS started with adapter of type 's3', but repository 'test' is of type 'local'","time":"2023-05-15T22:33:42+05:30"}

-- Config.yaml
logging:
 format: json
 level: WARN
 output: "-"

database:
 type: "local"

auth:
 encrypt:
  secret_key: "10a718b3f285d89c36e9864494cdd1507f3bc85b342df24736ea81f9a1134bcc"

blockstore:
 type: s3
 s3:
  force_path_style: true
  endpoint: <http://localhost:9000>
  discover_bucket_region: false
  credentials:
   access_key_id: minioadmin
   secret_access_key: minioadmin

Hi @Vaibhav Kumar Let me take a look

I tried many things like using export variables as well. So I think yes.

It means that you have a repo that is configured in local, yet for Minio you need it to be configured in S3 (Minio under the hood). Is it a problem to remove previously configured repos for you?

No that would not be a problem for me How to clean the old stuff

The simplest way would be to ‘rm -rf ~/lakefs/metadata’, but notice, that will result with a clean lakeFS installation, i.e. you’ll need to go through the setup again.

I pulled code from github and directly running the

lakefs run

and sometime I was using the

docker compose

mentioned in deployment section. I feel it should be ok.

What do you mean by “feeling it should be ok”?

I cleared the path mentioned by you, reran everything again. The error is something different now.

make sure minio is running and that the

access_key_id

and

secret_access_key

are correct

minio is also running on localhost:9000. PFB screenshot. I am using the same config.yaml as mentioned above. Can you please check if that is correct.

Is lakeFS running on its own container?

nope, I ran it in binary mode using lakefs run as mentioned in the thread. Pasting logs for your reference ,

simar@192 lakeFS % ./lakefs --config /Users/simar/lakeFS/cmd/lakefs/config.yaml run
INFO[0000]/Users/simar/lakeFS/cmd/lakefs/cmd/root.go:80 <http://github.com/treeverse/lakefs/cmd/lakefs/cmd.initConfig()|github.com/treeverse/lakefs/cmd/lakefs/cmd.initConfig()> Configuration file              fields.file=/Users/simar/lakeFS/cmd/lakefs/config.yaml file=/Users/simar/lakeFS/cmd/lakefs/config.yaml phase=startup
{"error":"InvalidParameterValue: Unsupported action GetCallerIdentity\n\tstatus code: 400, request id: 175FB0D31AF28E9F","file":"pkg/cloud/aws/metadata.go:58","func":"pkg/cloud/aws.(*MetadataProvider).GetMetadata.func1","level":"warning","msg":"Tried to to get AWS account ID for BI","time":"2023-05-16T23:16:46+05:30"}
{"error":"InvalidParameterValue: Unsupported action GetCallerIdentity\n\tstatus code: 400, request id: 175FB0D32E9E17CA","file":"pkg/cloud/aws/metadata.go:58","func":"pkg/cloud/aws.(*MetadataProvider).GetMetadata.func1","level":"warning","msg":"Tried to to get AWS account ID for BI","time":"2023-05-16T23:16:46+05:30"}
{"error":"InvalidParameterValue: Unsupported action GetCallerIdentity\n\tstatus code: 400, request id: 175FB0D33E6C11F9","file":"pkg/cloud/aws/metadata.go:58","func":"pkg/cloud/aws.(*MetadataProvider).GetMetadata.func1","level":"warning","msg":"Tried to to get AWS account ID for BI","time":"2023-05-16T23:16:46+05:30"}
{"error":"InvalidParameterValue: Unsupported action GetCallerIdentity\n\tstatus code: 400, request id: 175FB0D34C75FA56","file":"pkg/cloud/aws/metadata.go:58","func":"pkg/cloud/aws.(*MetadataProvider).GetMetadata.func1","level":"warning","msg":"Tried to to get AWS account ID for BI","time":"2023-05-16T23:16:47+05:30"}
{"error":"InvalidParameterValue: Unsupported action GetCallerIdentity\n\tstatus code: 400, request id: 175FB0D35E28C309","file":"pkg/cloud/aws/metadata.go:58","func":"pkg/cloud/aws.(*MetadataProvider).GetMetadata.func1","level":"warning","msg":"Tried to to get AWS account ID for BI","time":"2023-05-16T23:16:47+05:30"}
{"error":"InvalidParameterValue: Unsupported action GetCallerIdentity\n\tstatus code: 400, request id: 175FB0D3660017D3","file":"pkg/cloud/aws/metadata.go:58","func":"pkg/cloud/aws.(*MetadataProvider).GetMetadata.func1","level":"warning","msg":"Tried to to get AWS account ID for BI","time":"2023-05-16T23:16:47+05:30"}
{"error":"InvalidParameterValue: Unsupported action GetCallerIdentity\n\tstatus code: 400, request id: 175FB0D372D6228D","file":"pkg/cloud/aws/metadata.go:58","func":"pkg/cloud/aws.(*MetadataProvider).GetMetadata.func1","level":"warning","msg":"Tried to to get AWS account ID for BI","time":"2023-05-16T23:16:47+05:30"}
{"error":"InvalidParameterValue: Unsupported action GetCallerIdentity\n\tstatus code: 400, request id: 175FB0D37B89ECAE","file":"pkg/cloud/aws/metadata.go:58","func":"pkg/cloud/aws.(*MetadataProvider).GetMetadata.func1","level":"warning","msg":"Tried to to get AWS account ID for BI","time":"2023-05-16T23:16:48+05:30"}
{"error":"InvalidParameterValue: Unsupported action GetCallerIdentity\n\tstatus code: 400, request id: 175FB0D38586DB0F","file":"pkg/cloud/aws/metadata.go:58","func":"pkg/cloud/aws.(*MetadataProvider).GetMetadata.func1","level":"warning","msg":"Tried to to get AWS account ID for BI","time":"2023-05-16T23:16:48+05:30"}
{"error":"InvalidParameterValue: Unsupported action GetCallerIdentity\n\tstatus code: 400, request id: 175FB0D394A54FD0","file":"pkg/cloud/aws/metadata.go:58","func":"pkg/cloud/aws.(*MetadataProvider).GetMetadata.func1","level":"warning","msg":"Tried to to get AWS account ID for BI","time":"2023-05-16T23:16:48+05:30"}
{"error":"InvalidParameterValue: Unsupported action GetCallerIdentity\n\tstatus code: 400, request id: 175FB0D3A0359F7E","file":"pkg/cloud/aws/metadata.go:58","func":"pkg/cloud/aws.(*MetadataProvider).GetMetadata.func1","level":"warning","msg":"Tried to to get AWS account ID for BI","time":"2023-05-16T23:16:48+05:30"}
{"error":"InvalidParameterValue: Unsupported action GetCallerIdentity\n\tstatus code: 400, request id: 175FB0D3A86268E9","file":"pkg/cloud/aws/metadata.go:58","func":"pkg/cloud/aws.(*MetadataProvider).GetMetadata.func1","level":"warning","msg":"Tried to to get AWS account ID for BI","time":"2023-05-16T23:16:48+05:30"}
{"error":"InvalidParameterValue: Unsupported action GetCallerIdentity\n\tstatus code: 400, request id: 175FB0D3B49E2602","file":"pkg/cloud/aws/metadata.go:58","func":"pkg/cloud/aws.(*MetadataProvider).GetMetadata.func1","level":"warning","msg":"Tried to to get AWS account ID for BI","time":"2023-05-16T23:16:48+05:30"}
{"error":"InvalidParameterValue: Unsupported action GetCallerIdentity\n\tstatus code: 400, request id: 175FB0D3BE323081","file":"pkg/cloud/aws/metadata.go:58","func":"pkg/cloud/aws.(*MetadataProvider).GetMetadata.func1","level":"warning","msg":"Tried to to get AWS account ID for BI","time":"2023-05-16T23:16:49+05:30"}
{"error":"InvalidParameterValue: Unsupported action GetCallerIdentity\n\tstatus code: 400, request id: 175FB0D3CB4F0429","file":"pkg/cloud/aws/metadata.go:58","func":"pkg/cloud/aws.(*MetadataProvider).GetMetadata.func1","level":"warning","msg":"Tried to to get AWS account ID for BI","time":"2023-05-16T23:16:49+05:30"}
{"error":"InvalidParameterValue: Unsupported action GetCallerIdentity\n\tstatus code: 400, request id: 175FB0D3CB4F0429","file":"pkg/cloud/aws/metadata.go:64","func":"pkg/cloud/aws.(*MetadataProvider).GetMetadata","level":"warning","msg":"Failed to to get AWS account ID for BI","time":"2023-05-16T23:16:49+05:30"}
lakeFS dev-d2524dbb.with.local.changes - Up and running (^C to shutdown)...


   ██╗   █████╗ ██╗ ██╗███████╗███████╗███████╗
   ██║   ██╔══██╗██║ ██╔╝██╔════╝██╔════╝██╔════╝
   ██║   ███████║█████╔╝ █████╗ █████╗ ███████╗
   ██║   ██╔══██║██╔═██╗ ██╔══╝ ██╔══╝ ╚════██║
   ███████╗██║ ██║██║ ██╗███████╗██║   ███████║
   ╚══════╝╚═╝ ╚═╝╚═╝ ╚═╝╚══════╝╚═╝   ╚══════╝

│
│ If you're running lakeFS locally for the first time,
│   complete the setup process at <http://127.0.0.1:8000/setup>
│

│
│ For more information on how to use lakeFS,
│   check out the docs at <https://docs.lakefs.io/quickstart/repository>
│

│
│ For support or any other question,              >(.＿.)<
│   join our Slack channel <https://docs.lakefs.io/slack>     ( )_
│

Version dev-d2524dbb.with.local.changes

{"check_url":"<https://audit.lakefs.io/audit>","error":"audit check request failed: 424 Failed Dependency (Status code: 424)","file":"pkg/version/audit.go:111","func":"pkg/version.(*AuditChecker).CheckAndLog","level":"error","msg":"Audit check failed","time":"2023-05-16T23:16:52+05:30","version":"dev-d2524dbb.with.local.changes"}
{"error":"s3 error: \u003c?xml version=\"1.0\" encoding=\"UTF-8\"?\u003e\n\u003cError\u003e\u003cCode\u003eNoSuchBucket\u003c/Code\u003e\u003cMessage\u003eThe specified bucket does not exist\u003c/Message\u003e\u003cKey\u003edummy\u003c/Key\u003e\u003cBucketName\u003easd\u003c/BucketName\u003e\u003cResource\u003e/asd/dummy\u003c/Resource\u003e\u003cRequestId\u003e175FB0D8DBF6611F\u003c/RequestId\u003e\u003cHostId\u003ee3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855\u003c/HostId\u003e\u003c/Error\u003e","file":"pkg/block/s3/adapter.go:275","func":"pkg/block/s3.(*Adapter).streamToS3","host":"localhost:8000","level":"error","method":"POST","msg":"bad S3 PutObject response","operation":"PutObject","operation_id":"CreateRepository","path":"/api/v1/repositories","request_id":"76528b9b-fe52-49ad-94f7-829cc94b6bd9","service_name":"rest_api","status_code":404,"time":"2023-05-16T23:17:11+05:30","url":"<http://localhost:9000/asd/dummy>","user":"admin"}
{"error":"s3 error: \u003c?xml version=\"1.0\" encoding=\"UTF-8\"?\u003e\n\u003cError\u003e\u003cCode\u003eNoSuchBucket\u003c/Code\u003e\u003cMessage\u003eThe specified bucket does not exist\u003c/Message\u003e\u003cKey\u003edummy\u003c/Key\u003e\u003cBucketName\u003easd\u003c/BucketName\u003e\u003cResource\u003e/asd/dummy\u003c/Resource\u003e\u003cRequestId\u003e175FB0D8DBF6611F\u003c/RequestId\u003e\u003cHostId\u003ee3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855\u003c/HostId\u003e\u003c/Error\u003e","file":"pkg/api/controller.go:1409","func":"pkg/api.(*Controller).CreateRepository","level":"warning","msg":"Could not access storage namespace","reason":"unknown","service":"api_gateway","storage_namespace":"<s3://asd/>","time":"2023-05-16T23:17:11+05:30"}

Then it will not run on the same localhost. One is containerized and the other isn’t. I can see that you run

docker-compose up

. Take a look at the everything bagel docker-compose file. You can copy the first three containers and run it (with you keys and secrets).

yes , bagel was working fine for me earlier as well. I have added few log/print statements in lakefs code to test an issue. the problem with using bagel is I am not able to see those logs any where in docker logs as well when I am creating a hadoop client from my local .I am working on issue 2801.

ok try entering localhost:9001 in your browser. Do you get to the Minio dashboard?

yes. The above link is working. I can see the dashboard

I can see that:

The specified bucket does not exist

. Did you create it?

no I didn't create it

then please do

Tried

did you try running lakeFS with S3 before?

but I cleared the meta already

I understand, this is just a clarifying question. Do you have any previous experience with AWS S3?

yes, I do have

cool. So before you run lakeFS you need to have a bucket that you’ll use to create a repo in lakeFS. In your case, you need to create a bucket called

asd

in Minio before you try to create a repository that is mapped to it.

Ok so before creating a repo in lakefs we should have one bucket with the same name in minio? Any reason why we choose

asd

as a name?

Ok so before creating a repo in lakefs we should have one bucket with the same name in minio?

yes

Any reason why we choose

asd

as a name?

Just because you used it. That can be any name

And but going back to the old issue , by using bagel I don't see the log.info somewhere in logs I added under this file which I am accessing from my client from local.

This is a client. It is used as a Spark package and isn’t related to the server- bagel, binary or cloud. You should compile the client using

mvn package

and run it with Spark

Sure will post it there. thanks @Jonathan Rosenberg

Thank you @Vaibhav Kumar.

@Iddo Avneri @Jonathan Rosenberg I have followed the above link and moved forward in the problem. I have posted my findings there. Kindly take a look.

Thank you @Vaibhav Kumar. Will do