I’m setting up an on-premise lakefs instance and so far everything works, except importing files from S3, which returns this error :

$ lakectl import \                                                                     
  --from <s3://jaffle_shop_repo/raw_payments.csv> \
  --to <lakefs://jaffle-shop-repo/main/raw_payments.csv> 
Import failed: import error: error on ingest: NoSuchKey: 
        status code: 404, request id: tx00000000000000a72c046-0064ec5ed8-b6658b2f-fra1b, host id:

the Lakefs repo uses

<s3://jaffle_shop_repo>

as the storage namespace, and is located at the root of the S3 bucket. It doesn’t seem like a credentials problem, since the

lakectl fs upload

command works to add data in the repo, and write this data in the S3 storage repo’s folder. I suspect that it’s just a uri path problem, tried several variations but none seem to work Any idea on what could be wrong?

Hi @Bertrand Gallice, I'll try to help you with this issue, but with your permission let's take it step by step. Which version of lakeFS are you using?

Hi @Niro, I’m using lakectl and lakeFS versions 0.107.1

Thanks, Currently the lakeCTL client supports importing prefixes only and not specific objects. Can you try providing a prefix and update if you're experiencing the same issue?

With the 3 csv files moved in 2 locations,

<s3://to_import/>

and

<s3://jaffle_shop_repo/to_import>

, got the same error when running those 2 commands :

lakectl import \
  --from <s3://to_import> \ 
  --to <lakefs://jaffle-shop-repo/main/to_import>

lakectl import \
  --from <s3://jaffle_shop_repo/to_import> \ 
  --to <lakefs://jaffle-shop-repo/main/to_import>

@Bertrand Gallice thanks, I just noticed you are trying to import data from a path in your repository to another path in the repo? If that's the case, it can really help me pinpoint the issue if you could try to import the data from a different bucket / path which is not part of the lakeFS namespace

I think that refers to the first command, where the folder structure in my S3 bucket looks like that : • root ◦ to_import ▪︎ file1.csv ▪︎ file2.csv ◦ jaffle_shop_repo ▪︎ dummy or do you mean something else ?

Yes, I see - thanks, let me try to reproduce it on my end and get back to you

Thanks ! If that helps, I tried deploying on 2 clouds , DigitalOcean and Scaleway, and the same problem happened in both. I could set up the database up to the GUI and interacting with the repo with

lakectl fs

, only the import doesn’t work

That's valuable information - thank you. I'll get back to you in the next hour

lakectl fs ls <lakefs://jaffle-shop-repo/main/>

returns 1 object :

object          2023-08-28 13:20:50 +0200 CEST    1.3 kB          raw_customers.csv

A file I uploaded using

lakectl fs upload

Thanks, I think we have an issue when constructing the adapter for digital ocean when importing as it requires different semantics than S3. It's not something I have a WA for, we will need to fix this. In the meantime will you be willing to file an issue on this?

alright, thanks for the help!

Hi @Bertrand Gallice I was thinking about your issue and would like to try a possible WA. Are you able to run aws cli and list the objects in your DO bucket successfully from your environment?

Hi @Niro, it works ! The aws s3 ls command would return “\n ‘Bucket’“, when using

<https://BUCKET-NAME.fra1.digitaloceanspaces.com>

, and when using

<https://fra1.digitaloceanspaces.com>

as an endpoint instead, Removed the bucket name in the url I was using for LAKEFS_BLOCKSTORE_S3_ENDPOINT, and now the lakectl import command works normally. Thanks for you help!

Glad to hear it worked out!