https://lakefs.io/ logo
#help
Title
# help
o

Oliver Haney

02/06/2024, 5:00 PM
Is there a simple way to get an object count for some branch and path in lakefs?
i

Itai Admi

02/06/2024, 5:05 PM
It’s very hack-ish, but would this work?
Copy code
lakectl fs ls lakefs://<repo>/<branch>/<path> --recursive | wc -l
gratitude thank you 1
o

Oliver Haney

02/06/2024, 5:05 PM
ah of course the lakefs cli. I was trying to use the aws cli 🤦 Thank you for the suggestion
i

Itai Admi

02/06/2024, 5:06 PM
NP, I guess you could do the same with aws cli
a

Ariel Shaqed (Scolnicov)

02/06/2024, 5:12 PM
Not sure you will get any better than that -- lakeFS tries to be a lot like S3, and an object store doesn't really do that sort of thing without iterating over everything.
i

Iddo Avneri

02/06/2024, 6:06 PM
On that note. @Oliver Haney - whats the use case for counting number of objects? Maybe we can suggest something else 🙂
jumping lakefs 1
o

Oliver Haney

02/06/2024, 7:03 PM
I have an ingestion job running to build a vector store index. I use branches to segregate data produced from runs of said ingestion job. I just wanted to compare the number of objects produced in each branch in the same amount of time because I made some improvements to the ingestion process.
i

Iddo Avneri

02/06/2024, 7:07 PM
So the files will be completely different files in each branch - correct?
o

Oliver Haney

02/06/2024, 7:08 PM
The second branch I created contains the same objects as the first branch and more.
I just wanted to a dirty comparison without updating the code since the job had already been deployed.
I was successful using the lakefs cli
i

Iddo Avneri

02/06/2024, 7:09 PM
right. Comparing in the UI might work as well. THANK YOU!
2 Views