https://lakefs.io/ logo
#help
Title
# help
r

Robin Moffatt

03/06/2023, 10:24 AM
I suspect this is a duckDB issue rather than lakeFS per se, but asking here since y'all so helpful anyway 🙂 Running DuckDB locally to work with data on lakeFS cloud works just great. However, in trying to use it against my local lakeFS install I'm hitting problems. Using
aws s3
I can connect to my lakeFS just fine using the S3 gateway:
Copy code
$ aws s3 --endpoint-url <http://127.0.0.1:8000> ls <s3://drones03/main/>
2023-03-06 09:59:24     119663 Registations-P107-Active-2017.parquet
But if I use DuckDB it errors, I think because it is forcing the connection to HTTPS (which it's not)
Copy code
D SET s3_endpoint='<http://127.0.0.1:8000>';
D select * from read_parquet('<s3://drones03/main/Registations-P107-Active-2017.parquet>');
Error: Invalid Error: IO Error: Connection error for HTTP HEAD to '<https://http>://127.0.0.1:8000/drones03/main/Registations-P107-Active-2017.parquet'

D SET s3_endpoint='127.0.0.1:8000';
D select * from read_parquet('<s3://drones03/main/Registations-P107-Active-2017.parquet>');
Error: Invalid Error: IO Error: SSLConnection error for HTTP HEAD to '<https://127.0.0.1:8000/drones03/main/Registations-P107-Active-2017.parquet>'
Is there a way around this?
92 Views