Write back from duckDB to lakeFS I get an ambiguous error wh

Question

Write back from duckDB to lakeFS I get an ambiguous error when I try to write to a protected branch I m not familiar enough with the interface between the two tools here Is lakeFS throwing a more verbose exception that duckDB is squashing or is lakeFS just throwing a generic error that could be better Cannot write to a protected branch for example ```D export database <s3 drones03 main drone registrations > format parquet 94 Error Invalid Error IO Error Unexpected response during S3 multipart upload finalization``` It works just fine writing to a branch I m just interested in where to follow up re improving the error

Barak Amar · Answer

S3 had no concept of branch protection so my suggestion is to find the equivalent error you expect from S3 by executing it against S3 We can align the error to match it I assume lakefs current error is generic one for S3 protocol

Ariel Shaqed (Scolnicov) · Answer

Whenever there s any doubt I like to blame S3 clients for deleting all useful information from errors On a serious note I ll take a look on Sunday as to what error message lakefs encodes in these cases However as < Barak Amar> said there aren t that many useful S3 error types in the first place

Ariel Shaqed (Scolnicov) · Answer

Checking the code it seems that writing to a protected branch should give you some forbidden code a 403 Could you check the lakeFS logs and see if we do indeed generate that If we do I ll play my blame the S3 client card sorry

Robin Moffatt · Answer

thanks both This was against lakeFS cloud so I can t see the logs I ll see if I can reproduce it locally

Robin Moffatt · Answer

locally I see this ```time= 2023 03 06T11 22 57Z level=error msg= could not update metadata func= pkg gateway operations PathOperation finishUpload file= build pkg gateway operations operation utils go 51 error= cannot write to protected branch host= 127 0 0 1 8000 matched host=false method=POST operation id=post object path=foo parquet physical address=data gp4jc9og1v5s73bu3j9g cg2sq48g1v5s73bu3jag ref=main repository=drones03 request id=6e0d5032 017a 410d a6e6 fac8ad9bcc66 service name=s3 gateway upload id=79fdb595477b4f86ad73fcebb24b3b13 user=admin```

Ariel Shaqed (Scolnicov) · Answer

Yeah that seems fine I reckon it s some issue in the DuckDB client or some library that it uses and you lose the error message

Ariel Shaqed (Scolnicov) · Answer

<https github com duckdb duckdb blob 1bb227b4107e18ccc02dbfe0ead0c701cbc92b6a extension httpfs s3fs cpp L439 L442|Uggh> nauseated face gt ``` auto open tag pos = result find lt CompleteMultipartUploadResult 0 gt if open tag pos == string npos gt throw IOException Unexpected response during S3 multipart upload finalization gt ```

Ariel Shaqed (Scolnicov) · Answer

The code loses the error message

Robin Moffatt · Answer

logged <https github com duckdb duckdb discussions 6594>