Ariel Shaqed (Scolnicov)

Ariel Shaqed (Scolnicov)

10/19/2022, 12:54 PM
I'm marking all hadoop-* packages as "provided" in the Spark metadata client in this PR. In some cases it will change the commandline for running GC to include
--packages org.apache.hadoop:hadoop-aws:2.7.7
. But it should make it (much!) easier for the metadata client to co-exist with a wider range of Spark versions. Please make sure to note any objections or questions you might have on the PR!