|
|
Subscribe / Log in / New account

Debian discusses vendoring—again

Debian discusses vendoring—again

Posted Jan 13, 2021 7:50 UTC (Wed) by LtWorf (subscriber, #124958)
In reply to: Debian discusses vendoring—again by jafd
Parent article: Debian discusses vendoring—again

The issue is that fedora offers support for 5 months and debian for 5 years, and if some js library turns out to have a security bug, then the security team is supposed to hunt down possibly hundreds of copies of the same library and fix the issue.

Due to how bad js dependencies work, a single project can include multiple copies of the same library. So only within a single project there can be duplicates that need to be hunted down and fixed.

Fedora has none of those issues since it is a testing distribution for whatever new thing red hat wants to experiment this month.


to post comments

Debian discusses vendoring—again

Posted Jan 13, 2021 8:31 UTC (Wed) by epa (subscriber, #39769) [Link] (22 responses)

The security bug in the Javascript library might only be exploitable if some particular function is called in some particular situation. If the library is bundled with an application, but the application doesn't call that function, then the application does not have a security hole in practice and there is no urgent need to patch it.

This suggests that the tracking of security bugs should follow the same granularity as packages. If you package an application from upstream, and it includes its own copy of some library code, it is upstream's job to check for vulnerabilities and publish new releases which fix them. Debian's job is to publish new versions of the library as a standalone package, for the benefit of applications that don't bundle it but do things "the Debian way". This seems like a fairer way to share out the work, so that upstream takes some of the extra costs of bundling, not just taking the benefits and leaving the distribution to do the extra work.

If upstream cannot commit to keep on top of security holes and publish updates for a five year period, or can't commit to providing a stable series rather than "that release is obsolete, upgrade everything", then I guess the application is not suitable for inclusion in a distribution with a 5 year maintenance window. That applies equally well whether or not the application bundles its own libraries.

Debian discusses vendoring—again

Posted Jan 13, 2021 8:53 UTC (Wed) by bangert (subscriber, #28342) [Link] (1 responses)

not being very well versed in the security area, the above seems to me to be wrong.

say you have a function that is vulnerable to a privilege escalation, but the application in question is not calling it, so it is "safe".
now assume, another part of the application has a remote code execution bug - boom, the "safe" privilege escalation is all of a sudden not so safe any more...

apart from that, figuring out if a specific function bundled in a dependency is used in a project is orders of magnitudes harder, than figuring out if the project includes a given dependency (which itself can be difficult enough).

Debian discusses vendoring—again

Posted Jan 13, 2021 10:06 UTC (Wed) by mjg59 (subscriber, #23239) [Link]

In general, everything running inside a single process is in the same privilege domain. Once you've got arbitrary code execution in the app, having more libraries mapped in doesn't matter - they're not going to escalate your privileges.

Debian discusses vendoring—again

Posted Jan 13, 2021 9:05 UTC (Wed) by Lionel_Debroux (subscriber, #30014) [Link] (1 responses)

Well, for multiple reasons, especially cost-related ones - most developers of FLOSS remain unpaid, or at least not compensated to amounts anywhere near the time they spend on it - very few upstreams provide updates to software releases in such a way that these releases would be suitable for 5 years of security maintenance for a distro (i.e. over 5 years of upstream security maintenance, given the freeze period and also the gap between the latest upstream release and the beginning of the freeze period). Also, many upstreams do not maintain a stable release series either.

Debian discusses vendoring—again

Posted Jan 13, 2021 18:27 UTC (Wed) by hkario (subscriber, #94864) [Link]

many upstreams don't maintain a stable API, let alone a stable branch

Debian discusses vendoring—again

Posted Jan 13, 2021 9:52 UTC (Wed) by LtWorf (subscriber, #124958) [Link] (17 responses)

> The security bug in the Javascript library might only be exploitable if some particular function is called in some particular situation.

So instead of patching all the copies of the library you now need to audit each and every function call to the vulnerable library, and then still patch it wherever it is vulnerable.

How is it any less work?

> it is upstream's job to check for vulnerabilities

They use npm, realistically they won't check for vulnerabilities.

But even if they did… would they backport the fixes to whatever version debian is using? If not, then it means they can't be used in a stable distribution, because they'd change everything when fixing the security bug as well.

By the way this behaviour is what pushed distributions to drop mysql and move to mariadb en masse. Because oracle doesn't backport fixes and doesn't say what the issues are. They just say "some security issue is fixed" and so you must upgrade.

> so that upstream takes some of the extra costs of bundling

Won't happen. An incredible amount of those js packages are like 20 lines of code, that depend on other 1-3 similarly small packages.

The authors write them and forget about them in several cases. Also many of them write on windows and have no idea about linux.

> then I guess the application is not suitable for inclusion in a distribution with a 5 year

The issue is that following your criteria, basically no js application is suitable for inclusion.

Debian discusses vendoring—again

Posted Jan 13, 2021 10:08 UTC (Wed) by mjg59 (subscriber, #23239) [Link] (10 responses)

> How is it any less work?

Not having to verify that an update doesn't also break functionality you depend on is less work than having to verify that everything still works fine with a nominally compatible security backport.

Debian discusses vendoring—again

Posted Jan 13, 2021 11:52 UTC (Wed) by LtWorf (subscriber, #124958) [Link] (9 responses)

Backports of security bugs don't change APIs.

Debian discusses vendoring—again

Posted Jan 13, 2021 12:21 UTC (Wed) by mjg59 (subscriber, #23239) [Link] (7 responses)

Ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha ha

(No)

Debian discusses vendoring—again

Posted Jan 13, 2021 14:45 UTC (Wed) by LtWorf (subscriber, #124958) [Link] (6 responses)

Care to point out 1 patch done by a distribution security team that did change API?

Debian discusses vendoring—again

Posted Jan 13, 2021 14:58 UTC (Wed) by pizza (subscriber, #46) [Link] (5 responses)

> Care to point out 1 patch done by a distribution security team that did change API?

Sure. I lost soft scrollback in my linux consoles, because upstream's "fix" for the security hole was to strip out the feature entirely. Maintaining that feature would have required devising an independent fix (which IIRC never happened) or reverying the upstream changes, leaving that hole open.

Debian discusses vendoring—again

Posted Jan 14, 2021 13:11 UTC (Thu) by LtWorf (subscriber, #124958) [Link] (4 responses)

That's not an API.

Debian discusses vendoring—again

Posted Jan 14, 2021 13:56 UTC (Thu) by pizza (subscriber, #46) [Link] (3 responses)

Okay, fine, if you want to split hairs, I'll be more explicit.

The linux kernel API that allowed for software-based console scrollback was completely removed as part of a "stable kernel security update" from my distribution.

Sure, the patch that removed it originated with the upstream kernel, but it was still an API change that the distro pushed out to its users.

Debian discusses vendoring—again

Posted Jan 15, 2021 14:01 UTC (Fri) by jafd (subscriber, #129642) [Link] (2 responses)

Was the scrollback something you could use programmatically via a kernel API? Or maybe it exposed an ioctl(2) API? Or was there another driver that stopped working entirely because it was using some of the functions that had been a part of scrollback and got obliterated?

I'm confident the world could be better without this very bizarre definition of "API" both you and mjg59 are using here.

Debian discusses vendoring—again

Posted Jan 15, 2021 15:20 UTC (Fri) by farnz (subscriber, #17727) [Link]

It was - TIOCLINUX subcode 13 (undocumented, but existed) let you control the scrollback from user space. There are documented ioctls for copy and paste you can use to programmatically extract data from a VC.

Debian discusses vendoring—again

Posted Jan 22, 2021 5:19 UTC (Fri) by HelloWorld (guest, #56129) [Link]

This definition isn't bizarre at all.

Debian discusses vendoring—again

Posted Jan 15, 2021 14:12 UTC (Fri) by jafd (subscriber, #129642) [Link]

Not always, and certainly not in the javascript/NodeJS world.

I won't get into the madness of the neighboring thread (where they are discussing a user-visible change which is not an API change), but here's a very real scenario I'm seeing in a proprietary app that's my job to work on:

1) for $REASONS, we're stuck using a tool of the major version N. It's rather expensive to move to a tool of major version N+1 or N+2.
2) by now, some fourth-order dependencies of the tool had security vulnerabilities discovered, and the only path forward is to bump their major versions. Because JavaScript developers don't believe in backports. A dozen small libraries, but they are all over the place.
3) bumping them will need also bumping the third-order dependencies and so on, which is going to break the tool of version N, no longer maintained.
4) this kind of yak-shaving can take a lot of time, and I'm not counting the regressions that crept in because the migration documentation is far from being complete or accurate. Oh, and new versions of the tool produced buggy output which wasn't our fault. Back to the drawing board we are.

So, basically, until we find time to bring the tool to N+2, we're hosed according to "npm audit". Thankfully the tool is not user-facing, but you can see how "minor fixes" sometimes necessitate bigger changes a project may not be ready to embrace yet.

Debian discusses vendoring—again

Posted Jan 13, 2021 13:03 UTC (Wed) by khim (subscriber, #9252) [Link] (5 responses)

> The issue is that following your criteria, basically no js application is suitable for inclusion.

Depends on the promises they make.

> Won't happen. An incredible amount of those js packages are like 20 lines of code, that depend on other 1-3 similarly small packages.

Which, in turn, means that any JS application is POS which can and would be owned by malicious input presented to it.

That's why all these talks about trying to fix libraries, etc is pointless. You don't need to do it.

Instead you have to think what protections you have in place for the inevitable breakage which results. How these applications are compartmentalized is important. How data is protected from them is important.

How these apps are updates is NOT important because it's just pointless waste of time to try to plug holes in the sieve.

Debian discusses vendoring—again

Posted Jan 13, 2021 13:37 UTC (Wed) by pizza (subscriber, #46) [Link] (1 responses)

> Depends on the promises they make.

That's easy. Not only do they make no promises, they explicitly disclaim warranties, including merchantability and fitness for any purpose.

Debian discusses vendoring—again

Posted Jan 13, 2021 16:09 UTC (Wed) by smoogen (subscriber, #97) [Link]

Wait you mean all those licenses and such which no one reads but says that the writer disclaims all responsibility actually mean something?

Personally I look at all these Javascript/Python/etc to being similar to having every call in libc/libm etc as separate libraries versus 'bundled' together as a single item.

Debian discusses vendoring—again

Posted Jan 15, 2021 3:37 UTC (Fri) by VolkerWeissmann (guest, #144200) [Link] (2 responses)

> Which, in turn, means that any JS application is POS which can and would be owned by malicious input presented to it.

What does POS stand for?

Debian discusses vendoring—again

Posted Jan 15, 2021 14:13 UTC (Fri) by jafd (subscriber, #129642) [Link]

A "Piece Of Excrement".

Debian discusses vendoring—again

Posted Jan 15, 2021 15:50 UTC (Fri) by MKesper (subscriber, #38539) [Link]

Pile of shit, I guess?

Debian discusses vendoring—again

Posted Jan 13, 2021 9:01 UTC (Wed) by zdzichu (guest, #17118) [Link] (1 responses)

Last sentence of you comment is blatantly false.

First, Fedora offers 13 months of support for each version.

Second, Fedora is fully fledged distribution with stable releases and various variants (for example Workstation, Server). It is NOT "a testing distribution".

I am a Fedora maintainer. I am not associated with Red Hat - most of us aren't.

Debian discusses vendoring—again

Posted Jan 13, 2021 9:54 UTC (Wed) by LtWorf (subscriber, #124958) [Link]

It was a hyperbole. Remains the issue that 13 months is a smaller number than 60.

Debian discusses vendoring—again

Posted Jan 13, 2021 10:15 UTC (Wed) by Cyberax (✭ supporter ✭, #52523) [Link] (1 responses)

Realistically Kubernetes is much better supported by upstream than Debian. Google probably has an order of magnitude more security people working on it than the whole Debian security team. Ditto for many other packages.

It's just not something that Debian should tackle. Sure, it goes against the old model of Linux distros but times change...

Debian discusses vendoring—again

Posted Jan 14, 2021 0:58 UTC (Thu) by dvdeug (guest, #10998) [Link]

Except we're not talking about Kubernetes; we're talking about GSA. GSA almost certainly doesn't have an order of magnitude more security people than Debian. And when we're done with GSA, we're going to be talking about an RPG written in an open source JS framework. Part of the problem is there is no line here; the big items may have support, but the small items are what make up most of the system, and they're endless.

Debian discusses vendoring—again

Posted Jan 13, 2021 12:06 UTC (Wed) by roc (subscriber, #30627) [Link] (1 responses)

Recently in our Rust project (https://pernos.co) we integrated https://github.com/EmbarkStudios/cargo-deny into our CI so that as soon as a CVE is published for any library our project depends on, our CI starts failing. That CVE data is managed at https://github.com/RustSec/advisory-db and could be consumed by all manner of tools.

I don't know the NPM world, but if they don't have something like this already, they need it. Distro packagers could use such a tool to detect which packages contain bundled dependencies that need updating.

Debian discusses vendoring—again

Posted Nov 6, 2023 11:43 UTC (Mon) by j0057 (subscriber, #143939) [Link]

I agree, the distro model is something from the world of C where system-provided dynamic libraries are the only way to re-use code, and even there many applications have started vendoring their dependencies.

Languages other than C have proper packaging support, and it's now up to the distros to adapt and make use of the information there is. Hunting down which application uses which library is really not that difficult, unless you're unable to tap into the metadata that a given packaging ecosystem already provides.

Debian discusses vendoring—again

Posted Jan 13, 2021 12:12 UTC (Wed) by cesarb (subscriber, #6266) [Link] (6 responses)

> The issue is that fedora offers support for 5 months and debian for 5 years,

Fedora offers support for 13 months, not 5 months.

> and if some js library turns out to have a security bug, then the security team is supposed to hunt down possibly hundreds of copies of the same library and fix the issue.

Fedora uses a trick to help with that: as explained at https://fedoraproject.org/wiki/Bundled_Libraries every package which includes a bundled library will have a "Provides: bundled(package) = version", so it's easy to find which packages have a given bundled library, and on which version.

Debian discusses vendoring—again

Posted Jan 13, 2021 18:57 UTC (Wed) by drawks (guest, #144158) [Link] (5 responses)

I recall years ago that there was some suggestion that debian could similarly use the "Built-Using:" metadata tags for indicating that a given package has some vendored code included in it. The "Built-Using:" tag already exists for tagging the handful of statically compiled packages which cannot avoid vendoring (busybox, various installer components, etc). The existing presumption is that the vendored components are available at package build time as satisfiable "Build-Depends:". The difference in the npm situation is that at build time the dependencies which are to be bundled would not be packaged separately, but would come from $ambiguous_source and the build system would have to be trusted to tag in the "Built-Using:" data.

This type of solution is a find technical solution if not 100% philosophically "clean" in the way debian intends. I would posit that such a solution would be fine for third party package providers and it would be a big step towards solving real world problems if Debian would just outline that this is a best practice for third parties and that first party debian packages would still be required to provide individually packaged packaged dependencies at build time. Softening the the limits on the number of versions of a library can be in the debian repositories would further make this a maintainable situation moving forward with a reasonable path for all various upstreams to cut between major versions of their dependencies.

at any rate thats my .02 maybe someone can tell me why I'm completely wrong :)

Debian discusses vendoring—again

Posted Jan 13, 2021 19:35 UTC (Wed) by smcv (subscriber, #53363) [Link] (4 responses)

This isn't what Built-Using is for. A new metadata field could absolutely be used to track what's vendored into each package, but Built-Using is not it.

Built-Using is for the situation when we have some code in one Debian package, and we copy that code into a different Debian package during the second package's build. For instance, if you build busybox-static while you have libc6-dev_2.31-9 installed, version 2.31-9 of libc.a gets statically linked into the busybox-static binary. Now, as long as we have that copy of busybox-static in the archive, we also have to keep a copy of the glibc_2.31-9 source package around, otherwise we'll be missing the source code for some of our binaries (which is a LGPL violation).

Which upstream packages are vendored into the source package for which Debian packages could be tracked in a decentralized way by inventing a new metadata field, vaguely similar to Fedora's use of Provides, but at the moment it's tracked centrally instead: https://salsa.debian.org/security-tracker-team/security-t...

Debian discusses vendoring—again

Posted Jan 13, 2021 20:09 UTC (Wed) by drawks (guest, #144158) [Link] (3 responses)

Your description of how it works doesn't sound too different to vendoring, especially for the go/rust type of situations where you are literally vendoring by way of statically linking. I'd be unopposed ot using a new metadata tag that is more specifically scoped for this case, but my point still stands that this is a usable mechanism in all the ways I have described.

Debian discusses vendoring—again

Posted Jan 14, 2021 17:07 UTC (Thu) by smcv (subscriber, #53363) [Link] (2 responses)

The resulting binary package has similar properties, but the way you get there is different. With a package that uses Built-Using, a no-changes rebuild will make it pick up the current versions of the packages that it was Built-Using (for example its statically linked copy of glibc, or its statically linked copy of a Go or Rust library). With a package that uses vendoring, the vendored versions of its dependencies are part of the package's source code and will not change without an update to the source package.

Debian discusses vendoring—again

Posted Jan 15, 2021 3:37 UTC (Fri) by VolkerWeissmann (guest, #144200) [Link] (1 responses)

If you have
mydependency = "1.2.3"
in your Cargo.toml, then cargo won't update it if you rebuild it.

Debian discusses vendoring—again

Posted Jan 15, 2021 4:36 UTC (Fri) by mathstuf (subscriber, #69389) [Link]

I think you want `"=1.2.3"` there for that. Without any comparison operator, you get `^` or `~` (depending on a zero major version) which means "any compatible version" in practice.


Copyright © 2025, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds