tl;dr: Where is a good place to upload JARs for us...
# dev
a
tl;dr: Where is a good place to upload JARs for use in testing? They do not belong to our namespace, so I cannot use Maven Central. Longer explanation: • I want to upload hadoop-aws-2.7.4.jar to somewhere so that I can use it to test Spark 3 with ancient Hadoops. ◦ It is not possible to get this version of the JAR from Maven Central because it's been dead for so long. • I can get a copy of this jar: ◦ Unfortunately it lives inside the full Hadoop 2.7.4 download and that's not what I want to test. ◦ So I extracted this jar from that .tar.gz • It's part of org.apache.hadoop, so Maven Central won't allow me to create a profile for that. • I do want to put it somewhere useful... but where?
o
I believe https://github.com/features/packages has a free tier and I think there's maven support. Perhaps worth checking out?
a
Sure. Thing is.. 1. I wouldn't let me register org.apache.hadoop packages. So I'd need to download them... 2. Maven support doesn't include the ability to download. That's Maven Central again. 3. Also, I just want to pull a jar into my Docker container while I'm building it. Right now I don't even have curl on the image I'm working with, so ideally I could use ADD (and get caching too)
Thanks! I ended up just writing a Dockerfile that downloads an entire Hadoop archive each time and extracts what it needs. Caching should help speed it up, plus this will only be used to create an image for testing.
🦜 1