"Critical" projects and volunteer maintainers [LWN.net]

"Critical" projects and volunteer maintainers

Posted Jul 13, 2022 22:38 UTC (Wed) by logang (subscriber, #127618)
Parent article: "Critical" projects and volunteer maintainers

I do agree that more of the problem needs to be pushed on users of libraries. Developers these days add anything at all to their dependency tree without much care about what they are importing. Getting developers to think about what they depend on and what their support needs are is important.

To this end, I think it would be better if publishers of libraries indicated the level of support they will provide in the metadata of their packages: anywhere from "random fixes maybe when I feel like it" to "guaranteed to fix a security issue within 2 days". That would make for a better plan than just forcing developers to become critical maintainers because of the poor choices of other developers.

Rules would be in place such that a package can't have a dependency on any package that has a lower level support guarantee than the one it declares.

Developers adding packages to their project would have some way to declare the support they are expecting. If they're doing some hobby project and don't care, then they can have access to everything. If they are building something real that they may need to support then they are limited to packages that have declared a certain level of support; or they have to contribute to (or adopt) a library and improve it before they can use it. I'd imagine large companies could then have internal policies to prevent their developers from using stuff that nobody has declared support for.

There'd have to be systems in place for maintainers to bow out: dropping the maintenance level for a package would have to create warnings for anything that depends on it which would essentially just be a call for help. Also methods for users to say that a package isn't really maintained to the level it claims.

All this would be voluntary and opt-in but at least there would be some indication of the support intentions of the original publisher.

I haven't read too much about the cargo-vet plan, but seems to me the most important thing is whether there are active maintainers wiling to support their work (which can change at any time); not whether someone has reviewed the code.

to post comments

"Critical" projects and volunteer maintainers

Posted Jul 14, 2022 6:46 UTC (Thu) by NYKevin (subscriber, #129325) [Link] (12 responses)

The problem is, developers are lazy. If pip install doesn't work, they will just copy/paste instead.* And that's even worse, because pip install at least leaves behind a metadata trail, so that when the inevitable CVE comes in, you can find and fix all the projects using a given piece of code. If you copy/paste everything, then there's no metadata, and you have no way of knowing what code has propagated where.

* "License? Eh, it's only a couple of lines,** nobody will ever know."
** Do not do this. This is not how copyright law works.

"Critical" projects and volunteer maintainers

Posted Jul 14, 2022 9:06 UTC (Thu) by Sesse (subscriber, #53779) [Link] (11 responses)

On the flip side: It's really easy to pull in a hundred projects as (indirect) dependencies. It's really hard to cut and paste the same amount of code.

"Critical" projects and volunteer maintainers

Posted Jul 14, 2022 16:39 UTC (Thu) by hkario (subscriber, #94864) [Link] (10 responses)

You are underestimating the capacity of people to "vendor" code in.

"Critical" projects and volunteer maintainers

Posted Jul 15, 2022 15:50 UTC (Fri) by salimma (subscriber, #34460) [Link] (9 responses)

Especially Git-based projects -- it's so easy to just use Git submodules and vendor in code. Though luckily those are also easy to spot when packaging for distributions.

"Critical" projects and volunteer maintainers

Posted Jul 15, 2022 16:34 UTC (Fri) by Sesse (subscriber, #53779) [Link] (8 responses)

Does not compute. It's _really hard_ to vendor in a hundred C++ projects and have them compile and work. (For an extreme outlier, look at Chromium: It has a bit over 300, and a small army of people and tooling to keep that going.) npm create-react-app gives you 2000+ subdependencies without a hitch, for hello world.

"Critical" projects and volunteer maintainers

Posted Jul 15, 2022 16:42 UTC (Fri) by hkario (subscriber, #94864) [Link] (7 responses)

That's the neat thing about updating vendored code: you don't.

"Critical" projects and volunteer maintainers

Posted Jul 15, 2022 17:49 UTC (Fri) by Sesse (subscriber, #53779) [Link] (6 responses)

Note that I said “compile and work”, not “maintain”.

"Critical" projects and volunteer maintainers

Posted Jul 15, 2022 18:05 UTC (Fri) by hkario (subscriber, #94864) [Link] (5 responses)

If you don't change the compiler, then it's unlikely you need to change vendored libraries, even if you modify your project.
And even if you do change the compiler, all that's needed to make the compile again, is make it use older standard explicitly or ignore some warnings, not completely rewrite large pieces of code...

"Critical" projects and volunteer maintainers

Posted Jul 15, 2022 18:13 UTC (Fri) by Sesse (subscriber, #53779) [Link] (1 responses)

I have no idea what this means. What does the compiler have to do with anything?

More to the point, do you have real-world examples of OSS projects vendoring 1000+ C++ libraries?

"Critical" projects and volunteer maintainers

Posted Jul 15, 2022 19:47 UTC (Fri) by Wol (subscriber, #4433) [Link]

What gcc are we on? When did the linux kernel upgrade guidance from - what was it - 4.9?

That experience says compiler upgrades break working code ...

Cheers,
Wol

"Critical" projects and volunteer maintainers

Posted Jul 15, 2022 20:56 UTC (Fri) by bartoc (guest, #124262) [Link] (2 responses)

The problem is that each time a vendored library updates you need to do a bunch of work to ensure your rewritten build system keeps up with any changes in upstream's build system.

You can mitigate this by just like, writing a bash script that downloads and builds all your dependencies and plops them into somewhere where your buildsystem can find them. Then again .... how do you think linux package managers got started :D?

"Critical" projects and volunteer maintainers

Posted Jul 15, 2022 23:37 UTC (Fri) by rgmoore (✭ supporter ✭, #75) [Link] (1 responses)

The problem is that each time a vendored library updates you need to do a bunch of work to ensure your rewritten build system keeps up with any changes in upstream's build system.

This assumes you intend to keep in sync with upstream. More frequently, projects vendor libraries precisely because they want to diverge from upstream, and copy/paste/edit is easier in the short term than trying to maintain their changes as a patch. When upstream changes, they don't adopt those changes because they aren't relevant, and the vendored library gradually diverges from upstream.

"Critical" projects and volunteer maintainers

Posted Jul 20, 2022 9:58 UTC (Wed) by bartoc (guest, #124262) [Link]

Yeah, there are two "modes" of vendoring I think. If you're using git sub-modules you probably do want to keep up with _some_ upstream, even if it's your own fork. If you want to make changes you might use git-subtree instead. Or derive from "upstream" by simply reading and "rewriting" their code. The last case may not even be considered vendoring, you intend to maintain the thing separately and also want to perhaps change behavior.

To be honest I think all these methods can be fine sometimes, esp the submodule thing is basically a poor-man's package manager and if you share a buildsystem with upstream (or use meson's cmake support thing, I guess) it can be fine.

Hell even the gnulib thing of extracting particular (possible differently licensed) components can work, they do keep things up to date ... most of the time. Ofc a lot of gnulib is much less useful than you would like.

I do kinda wish when people vendored stuff they were better about changing the names of functions, sometimes the vendored library doesn't "escape" across ABI boundaries and can be used with the upstream version just fine, if you change all the symbol names (or dynamically link). Actually have any linkers had support for doing this when static linking, like you can specify that undef symbols should be resolved from different archives for different sets of objects?

"Critical" projects and volunteer maintainers

Posted Jul 16, 2022 21:52 UTC (Sat) by marcH (subscriber, #57642) [Link]

> I do agree that more of the problem needs to be pushed on users of libraries. Developers these days add anything at all to their dependency tree without much care about what they are importing. Getting developers to think about what they depend on and what their support needs are is important.

This is already required in some big (hence slow?) companies and there's also progress at the regulatory level, search "Software Bill Of Materials".

Security catastrophes and other scandals move things in the right direction... very slowly.