Hello! I'm configuring Airbyte's S3 destination to...
# help
Hello! I'm configuring Airbyte's S3 destination to use lakeFS as Airbyte sync following these steps. I'm using lakeFS playground and the storage namespace for my repo is
, so what bucket name should I use to configure the Airbyte destination? Using either
could not connect to the bucket
error. What am I missing?
Hi @Vino! I'll try to reproduce your issue, but before that can you please check if treeverse-demo-lakefs-storage-production/user_saved-katydid works?
Sure. I get
Could not connect to the S3 bucket with the provided configuration. Bucket name should not contain '/'
error this time.
Ok, thanks. It was just a hunch - I'll bring up an environment and get back to you in the following hour.
Can you please let me know which Airbyte version you are using?
Sure. While you are reproducing it locally, here are the configs that I used: Airbyte Cloud Env:
Copy code
Destination Type: S3
Destination Name: lakefs-airbyte-repo
S3 Key ID: <lakeFS Playground Access Key ID>
S3 Access Key: <lakeFS Playground Secret Key>
S3 Bucket Name: treeverse-demo-lakefs-storage-production
S3 Bucket Path: main/raw
S3 Bucket Region: us-east-1
Output Format: csv
Normalization: No Flattening
Compression: No compression
Endpoint: <https://saved-katydid.lakefs-demo.io/>
S3 Path Format: <left it empty>
S3 Filename Pattern: <left it empty>
👍🏽 1
@Niro I'm not running on Airbyte OSS. Spun up airbyte cloud free trial instead.
@Vino the failure you've seen is due to code changes in Airbyte. I was able to reproduce it with the latest version and also successfully managed to configure the connector using an older version of Airbyte. I'm continuing the investigation and will provide additional updates soon
jumping lakefs 1
💪🏻 1
💪 1
Ofc. Thank you so much!