Hi, I'm trying to connect to AWS and create repo in S3 bucket but when I run this:

docker run --pull always -p 8000:8000 -e LAKEFS_BLOCKSTORE_TYPE='s3' -e AWS_ACCESS_KEY_ID='<My Access Key ID>' -e AWS_SECRET_ACCESS_KEY='<My Secret Access Key>' treeverse/lakefs run --local-settings

I get this:

Using local-settings parameters configuration. This is suitable only for testing! It is NOT SUPPORTED for production.

time="2023-11-08T20:59:25Z" level=info msg="lakeFS run" func=cmd/lakefs/cmd.glob..func8 file="cmd/run.go:91" version=1.1.0

time="2023-11-08T20:59:25Z" level=info msg="initialized Auth service" func=pkg/auth.NewAuthService file="build/pkg/auth/service.go:188" service=auth_service

time="2023-11-08T20:59:25Z" level=warning msg="Tried to to get AWS account ID for BI" func="pkg/cloud/aws.(*MetadataProvider).GetMetadata.func1" file="build/pkg/cloud/aws/metadata.go:81" error="operation error STS: GetCallerIdentity, https response error StatusCode: 403, RequestID: bcfe8cca-10c1-4bee-b9c8-db359a3bf938, api error InvalidClientTokenId: The security token included in the request is invalid."

I'm 100% sure that

AWS_ACCESS_KEY_ID

and

AWS_SECRET_ACCESS_KEY

I'm sending as parameters to docker are correct but I'm not sure how to include security token.

Hey @Alex Treyvus, the metadata-provider error is a red herring. Is your lakeFS installation otherwise work? Are you able to create repo, upload files, etc?

Hi @Itai Admi, I can't create repo on AWS S3 bucket in our restricted environment but it works perfectly fine and I can create repo in AWS S3 bucket in my personal AWS account. So, this is definitely security problem. I was just trying to understand why when I run docker lakefs against restricted AWS I get

InvalidClientTokenId: The security token included in the request is invalid.

I start docker like this:

docker run --pull always -p 8000:8000 -e LAKEFS_BLOCKSTORE_TYPE='s3' -e AWS_ACCESS_KEY_ID='<My AWS ACCESS KEY ID>' -e AWS_SECRET_ACCESS_KEY='My AWS SECRET ACCESS KEY' treeverse/lakefs run --local-settings

I don't know, for example, what is '--local-settings' or are there any parameters I need to change or modify in this command to connect to AWS S3 bucket in restricted environment?

Hey @Alex Treyvus - what do you mean by restricted environment? If you use the same AWS creds from the same env lakeFS is running, can you read/write to your bucket? In your command,

--local-settings

means that you’ll be running with a local filesystem database. If you wouldn’t have pass

LAKEFS_BLOCKSTORE_TYPE='s3'

it would be using the local FS as a blockstore too, but that’s not the case and S3 is being used.