https://lakefs.io/ logo
#help
Title
# help
r

Ryan Prasad

11/28/2023, 6:01 PM
Hey! Quick question - I was working through the tutorial and merged denmark-lakes back into main. When I query the updated lakes.parquet, I get this. What did I mess up? Querying the lakes.parquet file in the denmark-lakes branch works fine. Also, hopefully you are enjoying reinvent. Another question - would the spatial extension for DuckDB work with lakeFS?
n

Niro

11/28/2023, 6:15 PM
Hi @Ryan Prasad, certainly a lot of exciting things going on in re:invent! Guessing from the error I suspect the parquet written back got corrupted somehow. Can you please detail the step by step reproduction scenario? Also can you share your setup? • Are you using a local installation of lakeFS / quickstart / the lakeFS-Samples repo or any other setup • Underlying storage adapter • Any other additional information that might seem useful
r

Ryan Prasad

11/28/2023, 6:20 PM
lakeFS is running on AWS freebie tier. EC2, RDS, S3. Following tutorial to the T. I just wanted to hear from you before reverting.
n

Niro

11/28/2023, 6:23 PM
Doing a little online search, it seems that this issue usually happens when the parquet metadata is not fully written. I have to say, I don't think I remember this ever happening before. I would like to investigate it further and possibly try to recreate it. Would you mind opening an issue for this? Also can you tell me which lakeFS version is currently deployed on your environment?
r

Ryan Prasad

11/28/2023, 6:27 PM
I'll keep this repo as-is if you like and I can screenshare later when it convenient for you, if you like. I just pulled the latest build on Sunday. 1.2.0
n

Niro

11/28/2023, 6:55 PM
Thanks, I appreciate it. Can you possibly collect the lakeFS logs from that time. I'll open an issue soon and would like to attach the logs to it
r

Ryan Prasad

11/28/2023, 6:57 PM
I already opened an issue on GH. I have not even fired up a terminal today - where do I grab the logs?
n

Niro

11/28/2023, 7:23 PM
By default (unless configured otherwise) the logs are outputted to stdout. So you can try and check if they were captured in
/var/log/messages
r

Ryan Prasad

11/29/2023, 2:19 AM
Here is what I got. I think I was attempting this before 1300 EST before I posted in the chat. I am logging from Slack, but please contact me at rhprasad@outlook.com moving forward.
2 Views