import {ListObjectsCommand, S3Client} from "@aws-sdk/client-s3";

import { fromIni } from "@aws-sdk/credential-providers";

const client = new S3Client({

credentials: fromIni({profile: 'local'}),

endpoint: "<http://localhost:8000>",

});

var input = {

Bucket:"call-center3"

};

const command = new ListObjectsCommand(input);

const response = await client.send(command);

console.log(response)

I see several potential issues: 1. Make sure that you pass lakeFS credentials to the SDK, I can’t tell from the example if

local

profile in the file is pointing to lakeFS. 2. Unlike S3, lakeFS has branches. When you try to list objects from a repository you need to pass a prefix as well for the branch (must) and a prefix under the branch (optional).

@Itai Admi My credentials are fine as I am able to use the ListBucketsCommand just fine. Would my prefix be the same thing as my branch name?

Yes, followed by a

/

@Itai Admi I have to apologize, I misstated what I was getting a a response. I am getting a 200 response with no data objects. Just for reference I will post the response from ListBuckets as well.

import {ListObjectsCommand,ListObjectsV2Command,ListBucketsCommand, S3Client} from "@aws-sdk/client-s3";

import { fromIni } from "@aws-sdk/credential-providers";

const client = new S3Client({

credentials: fromIni({profile: 'local'}),

endpoint: "<http://localhost:8000>",

bucketEndpoint: false,

});

var input = {

Bucket:"call-center3",

Prefix:"main/"

};

const command = new ListBucketsCommand({});

const response = await client.send(command);

console.log(response)

Just to make sure I understand - both ListBuckets and ListObjects work as expected?

This code

import {ListObjectsCommand,ListObjectsV2Command,ListBucketsCommand, S3Client} from "@aws-sdk/client-s3";

import { fromIni } from "@aws-sdk/credential-providers";

const client = new S3Client({

credentials: fromIni({profile: 'local'}),

endpoint: "<http://localhost:8000>",

bucketEndpoint: false,

});

var input = {

Bucket:"call-center3",

Prefix:"main/"

};

const command = new ListObjectsV2Command(input);

const response = await client.send(command);

console.log(response)

Produces this response :

{

'$metadata': {

httpStatusCode: 200,

requestId: undefined,

extendedRequestId: undefined,

cfId: undefined,

attempts: 1,

totalRetryDelay: 0

}

}

Silly question: Do you have objects under the

main

branch?

Second set of code is ListObjects

That’s weird. Can you share lakeFS logs? You may be needing to change the level to TRACE for something useful to surface..

Can you direct me to the logs?

How are you running lakeFS?

I have it running in a docker container using the install instructions from the website.

you can get it from the outside too.

docker logs <container_id>

I am not seeing logs that indicate I am making contact either way. How do I turn up the level to TRACE?

in config yaml https://docs.lakefs.io/reference/configuration.html:

logging:
  level: trace

Yes, container needs to restart. But not deleted.

Inside of the docker container I have added a config.yaml in /home/lakefs with the following:

logging:
  format: json
  level: TRACE
  output: "-"

I then restarted my container and added a file through the UI. I don't see an increased level of logging. Do you have any idea what I have done wrong?

That’s unnecessary.. You can add it as env var to the container.

LAKEFS_LOGGING_LEVEL=TRACE

Yeah. I started the container with docker-compose. Do I have to make of a copy of this container and start that one with the log level set or can I restart my container and set the ENV variable at that time? Sorry for making you give me a mini course in docker.

No worries. No need to copy the container, you can use the same one with the env var

I have gotten some log data but it doesn't indicate any errors. I ran the 2 commands simultaneously and here are the log from the 2 request.

time="2022-09-07T16:56:23Z" level=debug msg="performing S3 action" func=pkg/gateway.EnrichWithOperation.func1.1 file="build/pkg/gateway/middleware.go:116" action=list_repos message_type=action
time="2022-09-07T16:56:23Z" level=debug msg="HTTP call ended" func=net/http.HandlerFunc.ServeHTTP file="usr/local/go/src/net/http/server.go:2047" host=localhost log_audit=API method=GET path=/ request_id=62baec00-fc16-444a-9042-1b8a0027e927 sent_bytes=529 service_name=s3_gateway status_code=200 took=25.8603ms user=admin

time="2022-09-07T16:56:23Z" level=debug msg="performing S3 action" func=pkg/gateway.EnrichWithOperation.func1.1 file="build/pkg/gateway/middleware.go:116" action=list_repos message_type=action
time="2022-09-07T16:56:23Z" level=debug msg="HTTP call ended" func=net/http.HandlerFunc.ServeHTTP file="usr/local/go/src/net/http/server.go:2047" host=call-center2.localhost log_audit=API method=GET path="/?prefix=main%2F" request_id=a98659cc-ae3a-44de-a5ef-ce01f248c4f4 sent_bytes=529 service_name=s3_gateway status_code=200 took=2.7925ms user=admin

It seems like both requests are ListBuckets requests (or

list_repos

as lakeFS writes it)

I am using the S3 client from AWS so where do you think the break down is occurring? I am using the ListObjects function but it is running the list_repos action on the LakeFS side.

Can you try using path style access for the S3 client, something like

AmazonS3 s3client = AmazonS3Client.builder()
            .withCredentials((new AWSStaticCredentialsProvider(credentials)))
            .withEndpointConfiguration(new AwsClientBuilder.EndpointConfiguration("host", "region"))
            .withPathStyleAccessEnabled(true)
            .build();

FORCEPATHSTYLE!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! I just had to add that option when building my s3 client.

I'm glad it's working, let us know if there's anything else we can do to help

now the action is list objects