Is create-symlink for Athena a metadata-only operation? It only registers metadata into the Glue catalog?
g
Guy Hardonag
10/18/2021, 1:44 PM
Hey @Sid Senthilnathan,
Thats correct. It currently registers data only into the glue catalog. Do you need to create symlinks for a different purpose?
s
Sid Senthilnathan
10/18/2021, 1:46 PM
No that what we would need. I just wanted to confirm it wasn't copying over data. "Creates partitioned directories with symlink files in the underlying S3 bucket." sounds like it is not just a metadata operation, its actually moving around data in the underlying S3 bucket
Sid Senthilnathan
10/18/2021, 1:46 PM
Which would take a lot of time
g
Guy Hardonag
10/18/2021, 1:56 PM
It doesn’t move data, it creates symlink files.
The symlink file contains a list of object locations.
That way Athena knows which object should’ve been under each path