Debian discusses vendoring—again

Posted Jan 13, 2021 12:12 UTC (Wed) by cesarb (subscriber, #6266)
In reply to: Debian discusses vendoring—again by LtWorf
Parent article: Debian discusses vendoring—again

> The issue is that fedora offers support for 5 months and debian for 5 years,

Fedora offers support for 13 months, not 5 months.

> and if some js library turns out to have a security bug, then the security team is supposed to hunt down possibly hundreds of copies of the same library and fix the issue.

Fedora uses a trick to help with that: as explained at https://fedoraproject.org/wiki/Bundled_Libraries every package which includes a bundled library will have a "Provides: bundled(package) = version", so it's easy to find which packages have a given bundled library, and on which version.

Debian discusses vendoring—again

Posted Jan 13, 2021 18:57 UTC (Wed) by drawks (guest, #144158) [Link] (5 responses)

I recall years ago that there was some suggestion that debian could similarly use the "Built-Using:" metadata tags for indicating that a given package has some vendored code included in it. The "Built-Using:" tag already exists for tagging the handful of statically compiled packages which cannot avoid vendoring (busybox, various installer components, etc). The existing presumption is that the vendored components are available at package build time as satisfiable "Build-Depends:". The difference in the npm situation is that at build time the dependencies which are to be bundled would not be packaged separately, but would come from $ambiguous_source and the build system would have to be trusted to tag in the "Built-Using:" data.

This type of solution is a find technical solution if not 100% philosophically "clean" in the way debian intends. I would posit that such a solution would be fine for third party package providers and it would be a big step towards solving real world problems if Debian would just outline that this is a best practice for third parties and that first party debian packages would still be required to provide individually packaged packaged dependencies at build time. Softening the the limits on the number of versions of a library can be in the debian repositories would further make this a maintainable situation moving forward with a reasonable path for all various upstreams to cut between major versions of their dependencies.

at any rate thats my .02 maybe someone can tell me why I'm completely wrong :)

Debian discusses vendoring—again

Posted Jan 13, 2021 19:35 UTC (Wed) by smcv (subscriber, #53363) [Link] (4 responses)

This isn't what Built-Using is for. A new metadata field could absolutely be used to track what's vendored into each package, but Built-Using is not it.

Built-Using is for the situation when we have some code in one Debian package, and we copy that code into a different Debian package during the second package's build. For instance, if you build busybox-static while you have libc6-dev_2.31-9 installed, version 2.31-9 of libc.a gets statically linked into the busybox-static binary. Now, as long as we have that copy of busybox-static in the archive, we also have to keep a copy of the glibc_2.31-9 source package around, otherwise we'll be missing the source code for some of our binaries (which is a LGPL violation).

Which upstream packages are vendored into the source package for which Debian packages could be tracked in a decentralized way by inventing a new metadata field, vaguely similar to Fedora's use of Provides, but at the moment it's tracked centrally instead: https://salsa.debian.org/security-tracker-team/security-t...

Debian discusses vendoring—again

Posted Jan 13, 2021 20:09 UTC (Wed) by drawks (guest, #144158) [Link] (3 responses)

Your description of how it works doesn't sound too different to vendoring, especially for the go/rust type of situations where you are literally vendoring by way of statically linking. I'd be unopposed ot using a new metadata tag that is more specifically scoped for this case, but my point still stands that this is a usable mechanism in all the ways I have described.

Debian discusses vendoring—again

Posted Jan 14, 2021 17:07 UTC (Thu) by smcv (subscriber, #53363) [Link] (2 responses)

The resulting binary package has similar properties, but the way you get there is different. With a package that uses Built-Using, a no-changes rebuild will make it pick up the current versions of the packages that it was Built-Using (for example its statically linked copy of glibc, or its statically linked copy of a Go or Rust library). With a package that uses vendoring, the vendored versions of its dependencies are part of the package's source code and will not change without an update to the source package.

Debian discusses vendoring—again

Posted Jan 15, 2021 3:37 UTC (Fri) by VolkerWeissmann (guest, #144200) [Link] (1 responses)

If you have
mydependency = "1.2.3"
in your Cargo.toml, then cargo won't update it if you rebuild it.

Debian discusses vendoring—again

Posted Jan 15, 2021 4:36 UTC (Fri) by mathstuf (subscriber, #69389) [Link]

I think you want `"=1.2.3"` there for that. Without any comparison operator, you get `^` or `~` (depending on a zero major version) which means "any compatible version" in practice.