https://lakefs.io/ logo
#dev
Title
# dev
a

Ariel Shaqed (Scolnicov)

01/22/2024, 8:37 AM
This patch release is to get over the fact that 1.9.0 clients are bad. Ideally we would yank that client release from PyPI and do something similar on Maven Central. I would appreciate some links to: ā€¢ How to yank from PyPI ā€¢ How to deprecate or delete on Maven Central (via Sonatype OSS). Thanks if you can provide clues!
n

Nicholas Junge

01/22/2024, 8:40 AM
a

Ariel Shaqed (Scolnicov)

01/22/2024, 8:43 AM
Cool, thanks! Now I just need to figure out how to get keys to our PyPI account -- normally only our CD has them, to prevent humans from touching lakeFS releases šŸ™‚
So... How can I log into our PyPI (prod!) account?
That leaves Java. AFAICT the answer there is "you can't". ;;-(
g

Giuseppe Barbieri

01/23/2024, 10:16 AM
you cant indeed
that's the double edge sword with Maven Central
you may want to consider having your own maven repository.. in this way you'd have full control
a

Ariel Shaqed (Scolnicov)

01/23/2024, 10:45 AM
Yeah, ouch. Probably not important enough to do, especially given that I'd be limited in which repositories I could use. I doubt Sonatype would let me.
g

Giuseppe Barbieri

01/23/2024, 10:46 AM
you could even use a github repository, in your organization, as custom maven repo
šŸ‘€ 1
a

Ariel Shaqed (Scolnicov)

01/26/2024, 6:38 AM
Thanks, I had a look and it's definitely a great way to do it! I still worry about hassle to our users. As one example: I've seen Spark clusters that are allowed to access some websites but not others. Going with the mainstream means not worrying about this. In this event I don't think the advantages are that compelling.
g

Giuseppe Barbieri

01/26/2024, 9:35 AM
for gradle users, it's just a matter of a single line, though
a

Ariel Shaqed (Scolnicov)

01/26/2024, 8:03 PM
Oh sure, for many cases this is a great solution! But Spark and other Hadoop-style clusters are also a use-case, and probably the most common one in the JVM world. In corporate settings not all websites will be accessible from a cluster. I would like to make this decision explicit. The pro to switch to hosting jars on GitHub is to allow us to future. revoke bad versions. The cons are that it introduces a new dependency for our users (a separate hosting service for lakeFS jars), and that it will surprise our users because pulling versions is not common in the Maven world. Our users pay the costs immediately, and we can always inform users they need to upgrade by side channels. If you disagree, let's open an issue and discuss there. Obviously actual numbers of users and configurations for multiple package managers will overrule whatever I have to say.
f

Florentino Sainz

02/05/2024, 12:29 PM
why dont you simply implement a way in CD to mark yanked releases using the "good" credentials? And then implement the appropriate controls in your org (i.e. 2-people approval or whatever) Can be as hacky as adding a file named 1.9.0.yanked on top of the 1.9.0 "branch" which gets auto-built and released but whatever for java no idea, implement a "public" blacklisting API which you query when creating the client and shows hard-to-miss warnings when detected?, you could even implement this in maven central to avoid being network-firewalled but its vhacky. I.e. you can create a new artifaced called "yanked-lakefs" and if your current version is in that repo, just throw a lot of warnings during lakefs initialization
a

Ariel Shaqed (Scolnicov)

02/08/2024, 6:50 AM
Hi, thanks, these are all good ideas! Unfortunately Maven packages in Java have no "installation" phase. Worse, initialization occurs on each Spark worker, not just on the driver -- and network access from there will definitely be frowned upon. Given the low risk combined with the ecosystem expectation of almost never pulling packages, I believe we should do nothing to yank the package from Maven Central.
g

Giuseppe Barbieri

02/14/2024, 10:51 AM
Ariel, to sum up, 1.9.0 on Maven Central should be avoided? If I need to use the API, shall I pull 1.8.0?
a

Ariel Shaqed (Scolnicov)

02/14/2024, 10:52 AM
Please. And I'll push 1.9.1 with this fix. In any case it's not something you'd notice.
šŸ‘ 1
1.11.0 should be out by now. Some mess around Maven Central, it's migrating and I'm not sure which web UI is definitiive.
g

Giuseppe Barbieri

02/14/2024, 1:50 PM
I can confirm I can pull it
jumping lakefs 1