Digging into Julia's package system

Posted Oct 13, 2021 16:47 UTC (Wed) by leephillips (subscriber, #100450)
In reply to: Digging into Julia's package system by willy
Parent article: Digging into Julia's package system

The incident involves the sole maintainer of a package removing it from a repository. In that case Julia will continue to use the version stored on your machine, and when it checks for updates it won’t find anything. Someone in possession of the source, which would be anyone who had `dev`ed it, could recreate the GitHub repository, if the license allowed it. If it were a package with other contributors, there would be other forks of the project on GitHub, and the registry could be changed to point to one of these, I suppose.

Digging into Julia's package system

Posted Oct 13, 2021 17:14 UTC (Wed) by leephillips (subscriber, #100450) [Link] (4 responses)

After reading Nathan Willis’ article about this incident at https://lwn.net/Articles/681410/, I have a few more observations.

Although Julia package development happens almost entirely on GitHub, the process is more decentralized than with npm, as contributors maintain their own forks as part of the GitHub pull request workflow. So one executive of a company deciding to pull down a package would not be so simple.

Every version of every package is identified by a unique UUID within the Manifest and Project files (an implementation detail I did not go into in the article). So switching a dependency on a particular version of a package means changing this identifier in the Manifests of the affected packages. It seems this part of the problem is much more tractable than the situation on npm.

Finally, it’s far less likely that a Julia programmer would create a dependency on a package that does what you can do in one line in Julia. I haven’t come across any public packages that are as trivial as leftpad.

Digging into Julia's package system

Posted Oct 14, 2021 11:14 UTC (Thu) by azumanga (subscriber, #90158) [Link] (3 responses)

To be honest, that sounds as bad as javascript!

Saying every package which has some dependency could "switch to a new version" doesn't feel helpful, you could do that in npm too if you like. I'm not really clear why it would be easier for Julia than it would be for Javascript.

I'm surprised Julia didn't do what Rust did -- there packages in the "package repository" are stored centrally, and unless there is a very serious issue released packages can never be removed. You can disable versions (by 'yanking' them), but users can still get those versions by specifying exact version number.

Digging into Julia's package system

Posted Oct 14, 2021 12:32 UTC (Thu) by Wol (subscriber, #4433) [Link] (1 responses)

> I'm surprised Julia didn't do what Rust did -- there packages in the "package repository" are stored centrally, and unless there is a very serious issue released packages can never be removed.

What do you mean by "centrally". If you mean "on the net somewhere", what happens if that (for various meanings of "that") go down?

Or is that repository mirrored (should you so choose) on your machine, so you can ALWAYS re-install that package if you need? iiuc gentoo downloads everything, and while I've deliberately configured my system to forget it, I think it's easy enough to change that so it keeps it ...

Cheers,
Wol

Digging into Julia's package system

Posted Oct 17, 2021 7:26 UTC (Sun) by roc (subscriber, #30627) [Link]

crates.io packages are stored in S3 and cached locally. S3 isn't really going to go down for technical reasons. Hopefully someone has a copy of the archive in case those S3 resources get deleted.

Digging into Julia's package system

Posted Oct 14, 2021 12:56 UTC (Thu) by leephillips (subscriber, #100450) [Link]

I’m afraid I don’t understand your comment. I didn’t say “switch to a new version” anywhere.