Bertrand Gallice
08/28/2023, 9:10 AM$ lakectl import \
--from <s3://jaffle_shop_repo/raw_payments.csv> \
--to <lakefs://jaffle-shop-repo/main/raw_payments.csv>
Import failed: import error: error on ingest: NoSuchKey:
status code: 404, request id: tx00000000000000a72c046-0064ec5ed8-b6658b2f-fra1b, host id:
the Lakefs repo uses <s3://jaffle_shop_repo>
as the storage namespace, and is located at the root of the S3 bucket.
It doesn’t seem like a credentials problem, since the lakectl fs upload
command works to add data in the repo, and write this data in the S3 storage repo’s folder.
I suspect that it’s just a uri path problem, tried several variations but none seem to work
Any idea on what could be wrong?Niro
08/28/2023, 9:22 AMBertrand Gallice
08/28/2023, 9:24 AMNiro
08/28/2023, 9:35 AMBertrand Gallice
08/28/2023, 9:42 AM<s3://to_import/>
and <s3://jaffle_shop_repo/to_import>
, got the same error when running those 2 commands :
lakectl import \
--from <s3://to_import> \
--to <lakefs://jaffle-shop-repo/main/to_import>
lakectl import \
--from <s3://jaffle_shop_repo/to_import> \
--to <lakefs://jaffle-shop-repo/main/to_import>
Niro
08/28/2023, 9:44 AMBertrand Gallice
08/28/2023, 9:51 AMNiro
08/28/2023, 9:54 AMBertrand Gallice
08/28/2023, 9:56 AMlakectl fs
, only the import doesn’t workNiro
08/28/2023, 9:56 AMlakectl fs ls
on your repo to list the files and tell me if it works?Bertrand Gallice
08/28/2023, 11:22 AMlakectl fs ls <lakefs://jaffle-shop-repo/main/>
returns 1 object :
object 2023-08-28 13:20:50 +0200 CEST 1.3 kB raw_customers.csv
A file I uploaded using lakectl fs upload
Niro
08/28/2023, 11:36 AMBertrand Gallice
08/28/2023, 11:47 AMNiro
08/29/2023, 1:31 PMBertrand Gallice
08/29/2023, 6:06 PM<https://BUCKET-NAME.fra1.digitaloceanspaces.com>
, and when using <https://fra1.digitaloceanspaces.com>
as an endpoint instead,
Removed the bucket name in the url I was using for LAKEFS_BLOCKSTORE_S3_ENDPOINT, and now the lakectl import command works normally.
Thanks for you help!Niro
08/29/2023, 6:16 PM