https://lakefs.io/ logo
#help
Title
# help
c

Cristian Caloian

09/12/2023, 9:30 AM
Hi, I am trying to access the underlying s3 storage of a repository using
aws --profile lakefs s3 ls s3://<bucket>/<repo> --recursive
. I have configured the
lakefs
profile with credentials from LakeFS, and the corresponding endpoint url. I also have
AuthFullAccess
,
FSFullAccess
and
RepoManagementFullAccess
permissions. However, I still get the following error
An error occurred (AccessDenied) when calling the ListObjectsV2 operation: Forbidden
Do you have an idea what I am missing?
i

Idan Novogroder

09/12/2023, 10:07 AM
Hi @Cristian Caloian, Two questions- 1. What do you mean when you say I have configured the
lakefs
profile with credentials from LakeFS
? do you mean you use access key and secret from lakeFS or do you mean you use the same AWS credentials your lakeFS instance use? 2. About your command-
aws --profile lakefs s3 ls s3://<bucket>/<repo> --recursive
why do you use s3://<bucket>/<repo>? you should use the storage namespace you configured for your repo, e.g.-
s3://<bucket>/<prefix>
h

HT

09/12/2023, 10:11 AM
isn't it supposed to be `s3://<repo>/<branch>/<somePath>`` ?
Oh nevermind, by
underlying s3 storage
you mean the back end S3 bucket that LakeFS is using ... in which case you need the credential for that S3 bucket (nothing to do with Lakefs credential)
i

Idan Novogroder

09/12/2023, 11:22 AM
If you're asking if you should use AWS credentials in your
~/.aws/credentials
file, the answer is yes. Please let me know if it worked for you 🙏🏽
c

Cristian Caloian

09/12/2023, 12:00 PM
I guess my first question is whether, in the context of migrating a repository from one LakeFS installation to a completely different LakeFS installation, I need to copy the underlying S3 storage. If the answer is yes then I think now I understand that it needs the AWS credentials.
I think I figured out… from the linked notebook in the guide, I think it is the underlying storage namespace
Copy code
source_storage_namespace = '<s3://mySourceBucket>'
target_storage_namespace = '<s3://myTargetBucket>'

aws_region = '<AWS region name>' # e.g. us-east-1
aws_access_key_id = 'aaaaaaaaaaaaa'
aws_secret_access_key = 'bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb'
Thanks again for making me realize the distinction between the two 🙏
mario luigi dance 1
3 Views