LWN: Comments on "The bogus CVE problem"

The bogus CVE problem

wtarreau — Fri, 29 Sep 2023 04:01:28 +0000

I totally agree with what was said above!

The bogus CVE problem

mathstuf — Sat, 23 Sep 2023 21:16:59 +0000

My understanding is that it is run by a military industrial complex component (or is it more the NSA/CIA security analogue?) via MITRE, not the government itself.

The bogus CVE problem

domdfcoding — Sat, 23 Sep 2023 07:31:54 +0000

What do you expect? Both the CVE program and NVD are run by the US Government. That alone should tell you not to touch it with a barge pole.

The bogus CVE problem

fung1 — Fri, 22 Sep 2023 17:06:54 +0000

I mostly run the vulnerability management team for a large ecosystem of popular free/libre open source services. We request CVE assignments as a matter of course when we determine that we're going to issue a security advisory, and use them for the purpose they were intended (tracking). We purposefully do not make up CVSS scores because the heck if I can guess whether this specific problem is critical or benign in your deployment, it all depends on how you're using our software. At least for us, CVSS is entirely pointless.

We also do not bother to dispute CVE assignments others request for bugs in our software, because it's not worth our (limited) time to do so. It's our policy that if someone has requested a CVE for one of our bugs and notified us what number got assigned, then we'll reuse that *if* we issue an advisory (instead of requesting a conflicting CVE). It's been clear for a very long time that since anyone can request and receive a CVE assignment without even talking to the people maintaining the project it's about, the mere existence of a CVE does not imply there's any sort of vulnerability, nor is there any point in wasting time trying to get them rescinded.

The bogus CVE problem

j16sdiz — Fri, 22 Sep 2023 08:54:10 +0000

> His example in that case is a double-free in curl that the project determined had a "medium" severity, while NVD scored it as "9.8 critical", as can be seen in the GitHub advisory database.

Is there anything special about this "double-free" making this just a medium?

Depends on the malloc implementation, double-free can corrupt data. In a library like curl, this can have very bad consequence.. A "high" score is well justified.

The bogus CVE problem

kunitz — Fri, 22 Sep 2023 06:46:43 +0000

I work in the financial industry and we live already in the regulated future. Basically if anything in your build has an unresolved high or critical rated CVE, you can't use it anymore. Right now we are in the process to move all our container images to Alpine, because it doesn't use all the user space tools. A normal Debian image has 70 to 100 high or critical rated CVEs.

Since there are so many software packages, nobody has the time to look at the detail of a CVE. If there is a CVE which is high or critical you have to make it disappear, regardless whether you actually use the functionality or expose the service in any way. The financial industry has the audit and governance structures to enforce such rules.

For me it looks like that dependency reduction will become very important in the industry. How open-source will deal with the requirements of the CRA will be interesting to see.

The bogus CVE problem

bearstech — Wed, 20 Sep 2023 14:21:18 +0000

I work for companies who misconduct in that way, so the problem might spread to your company without your consent. At least I got to bill them for handling that nonsense bureaucracy.

The main flow of alerts/requests I've been handling for close to a year is composed of ... requests to add anti-XSS HTTP headers. By far. At the same time I got _zero_ status requests on the Zenbleed/Inception mayhem from this summer. The cultural gap is that wide.

In my case the problem is clearly companies who want to adopt security policies but are no trained on computer security. I can't blame them except their CTO which most of the time is totally missing the point while thinking that security is just another component that need dashboards and reports, and done.

CVE ranking is conceptually wrong, a security issue must be evaluated with context and is multi-faceted, there should never be a single figure in the first place. When talking with a dev or project manager I can educate about those issues and explain them how to handle a specific security issue in context, that's often satisfying for both parties. But then it won't bubble up to the top management and against the solidified bureaucracy they call security policy.

The bogus CVE problem

wtarreau — Mon, 18 Sep 2023 12:14:17 +0000

> Most CVEs are valid.

Some ex-insiders such as Kurt Seifried and Josh Bressers in a recent talk disagreed with this suggesting that around 20% only were valid. See the link to their talk below in another comment.

The bogus CVE problem

cpitrat — Mon, 18 Sep 2023 11:20:36 +0000

But then you get to solve 2 CVEs with a single patch which is certainly something you can brag about.

The bogus CVE problem

pizza — Fri, 15 Sep 2023 13:17:19 +0000

> ... or this garbage will convince many of us that too much is too much and it's time to switch jobs,

This is the most likely outcome.

> we're still free to stop maintaining projects that take too much time under such stupid rules, and let the internet fall apart as well.

Ah yes, but if you stop maintaining it, you're engaging in willful gross negligence and will be on the hook for $very_massive liabilities. And that's where things are heading.

Damned if you do, damned if you don't.

The bogus CVE problem

wtarreau — Fri, 15 Sep 2023 11:42:37 +0000

> this will no doubt create an immense amount of work for projects, distributors, and so on, almost as if it was a jobs program.

... or this garbage will convince many of us that too much is too much and it's time to switch jobs, maybe become a european deputee of some other such dummy jobs where you can vote for random rules that morons will blindly apply. I mean, there's no reason we should be their slaves, we're still free to stop maintaining projects that take too much time under such stupid rules, and let the internet fall apart as well.

The bogus CVE problem

farnz — Fri, 15 Sep 2023 10:01:51 +0000

That sort of thing is already captured by the "Exploit Code Maturity (E)" component of the CVSS score; if the overall score was capped to 4 if E:U or E:X (i.e. no PoC available now), then people would focus on that.

Organisations that consume the CVSS metrics can do that today, of course.

The bogus CVE problem

buck — Fri, 15 Sep 2023 04:29:37 +0000

As a source of names for bugs it has served its purpose well, so not every vulnerability needs to get a vanity exploit name, and the ones that anybody cares about, you can generally remember them by number until they fade into obscurity (one hopes). The ones that are bogus or that don't merit attention? No inherent problem with them having names, just like one is free to name one's imaginary friends.

After all, remember the first of the 3 hardest problems in computer science.

As the key to databases (NVD, the CVE-named errata pages of RedHat, Debian security-tracker, etc.) with links to advisories and/or references, it's kinda useful to have them be content-addressable.

As long as one can ignore everything else, which I guess it's hard for somebody to do if a CVE is kind of like an unsealed indictment of your software product, ... hmm.

Maybe there needs to be a CVE-rank algorithm to score CVEs: If there are no vendor and/or distro and/or vulnerability-list and/or such links attached to, say, the NVD entry, then it should carry a low score and people should ignore it (should the score stay low). And if the score doesn't respond dynamically enough to reflect some really bad news? Well, if you're looking to track CVSS scores to tell you what advisories to pay attention to in your system, then I got nothing for you. But if you're only responding to some vulnerability scanner telling you which high-CVSS things you need to worry about, then it's the scanner's problem, not yours, and you can maybe go back to figuring out your most significant risks and how to balance them yourself.

Not sure if that would be very open to gaming (CVEO?) but also hard to see the point. I don't flatter myself that I'm very devious, though.

The bogus CVE problem

florianfainelli — Fri, 15 Sep 2023 03:01:18 +0000

Unfortunately the CVE circus will continue to go on:

- EU CRA: https://www.european-cyber-resilience-act.com/
- US EO 14028: https://www.nist.gov/itl/executive-order-14028-improving-...

this will no doubt create an immense amount of work for projects, distributors, and so on, almost as if it was a jobs program.

The bogus CVE problem

amworsley — Fri, 15 Sep 2023 01:22:39 +0000

I think it would be good to see some better CVSS rules established.
e.g. no CVSS 10 (or > 8?) if there is no PoC exploit code. So much time is wasted trying to see
if a vaguely described fault is actually a significant problem. If there is a PoC it really spells out
what is required and what can be done very quickly and efficiently.

The bogus CVE problem

nliadm — Thu, 14 Sep 2023 23:10:26 +0000

As an author of a container static analyzer, this is one of the reasons we purposefully do not consume NVD data directly. The CVSS scores from NVD frequently conflict with vendors and authors own severity ratings, and so users would ask us to "just fix" the score in the report. We now only consume vendor and language-authority data, and it's completely eliminated this class of problem.

I think having a process to triage reported vulnerabilities easily is key to handling this at scale. Frameworks like SSVC are great, but there's an onus on reporting tools to be able to integrate those decisions back in and not show scary red marks to people that just want a "security" toggle.

The bogus CVE problem

mcatanzaro — Thu, 14 Sep 2023 17:50:22 +0000

Honestly the current system is better than the alternative. It wasn't too long ago that LWN was running articles about how it was too hard to get CVEs assigned to real bugs due to nonresponsiveness from MITRE. Now it's easier to get the CVEs, but of course this means there are going to be more invalid CVEs.

Most CVEs are valid. But invalid CVEs are certainly annoying. Another common problem is multiple CVEs being reported for the same issue. Recently somebody found one bug in WebKit and managed to get 6 different CVEs for it by asking MITRE instead of Apple to assign them. It was a serious vulnerability and the reporter otherwise did a good job, but 6 CVEs was unimpressive. (I think I requested these be marked as duplicate CVEs, but I don't think it actually happened.)

What's especially weird is that sometimes MITRE sides with the reporter rather than the project developer even when there is no dispute. E.g. a few years ago, somebody received two CVEs for what turned out to be one bug (I think it was also WebKit bug?) and when I requested that they be combined, MITRE decided that two CVEs was correct because there were two different test cases that trigger the bug. This was mind boggling to me.

The bogus CVE problem

MarcB — Thu, 14 Sep 2023 15:54:29 +0000

Another example is a batch of four recent CVEs for Notepad++: https://securitylab.github.com/advisories/GHSL-2023-092_N...

One of them might be a real vulnerability, but very likely not a 7.8.

The other three are scored 5.5, but look much more like zeroes. Read buffer-overflows that do not cross any confidentiality or other security boundaries and do not have any way to exfiltrate the data anywhere.

The bogus CVE problem

wtarreau — Thu, 14 Sep 2023 15:39:31 +0000

I feel sorry for you, really. It must be quite frustrating to work in your company :-(

The bogus CVE problem

hkario — Thu, 14 Sep 2023 12:03:40 +0000

As much as I'd like that to happen, CVEs are baked into regulation, and government agencies and enterprises in regulated industries do spend a lot of money on software. So CVEs are here to stay.

The bogus CVE problem

smoogen — Thu, 14 Sep 2023 11:42:47 +0000

Some of the problems with CVE's is the scale of the software available now and how much is used in odd places all over the place. CVE's were designed and written around the idea that the amount of software was in the hundred's of thousands and you only had a couple million systems to alert and protect.

Some of the problems with CVE's is that the system was designed to replace at least 2 other systems which fell apart in the late 1990's where most software was closed source and unless you could prove undeniably that there was a true vulnerability (and could not be hand waived off with "you didn't do exactly what we said in this manual on Alpha Centauri protected by a cyberraptor ") it wasn't looked and or fixed.

Some of the problems is that everyone has a tendency to make anything they run into a game they can 'win' at. And when they can't win, look for ways to avoid playing anymore. This goes for core programmers, security researchers, software companies, etc. These are known problems which aren't solvable because they are 'human condition' issues (and every solution just becomes a new game to win or lose at).

Most of the issues are that the above are known, but because they are all hard to get N people to agree to even trying a solution, you end up punting the problem to the next year and maybe just add another number to the amount of possible CVE's for a year.

The bogus CVE problem

ringerc — Thu, 14 Sep 2023 09:37:06 +0000

The CVE noise is ridiculous. My org spends so much time and effort "fixing" these "vulnerabilities" that it has little left over for *actual real security efforts*.

They've decided it's too hard to classify the balance of supposed vulnerabilities, so they're just going to take the severities as fact, irrespective of where the component lives in our stack.

"Critical" vuln in a completely unrelated part of a golang executable that happens to share the same huge monorepo as a library we use? Everybody panic, rush to upgrade everything to the latest no matter what the breakage, even though we probably don't use it. Used by a 3rd pty component that didn't jump fast enough? Fork it and hack it.

Then we have all these mini forks of various 3rd party components sitting around rotting. Or they decide it's too hard to keep up with "security" for a perfectly good external component and rewrite it in-house, badly. No more CVEs because nobody's looking anymore! It's less secure, but we made the squeaky wheel go away. Go us.

They've also rolled out various mandatory and enforced static checkers and linters with centrally managed one-size-fits-all configs. The stupid things run on pull requests and they don't compare the pull request results to a scan of main with the same config and db version. So whenever the vendor or info-"security" management rolls out new config/dbs, totally unrelated changes become unmergeable. Sure you could raise a new separate PR to patch main then rebase your change. But you've got a gauntlet of Jira ticket workflow requirements, encorced pull request reviews from people who're often unavailable, slow unreliable GitHub workflows, etc. Who's got time for that? So we have PRs "fix off by one in A" that have 200 lines of completely irrelevant noise "fixing" meaningless non issues. They obscure the real changes and introduce new bugs.

Yay management of developers by non developers, and management of IT security by people who couldn't explain the difference between authentication and authorisation if you gave them a textbook first.

The bogus CVE problem

wtarreau — Thu, 14 Sep 2023 07:25:58 +0000

I also recommend the long but interesting talk from Kurt Seifried and Josh Bressers on this matter: https://opensourcesecuritypodcast.libsyn.com/episode-392-.... They've been part of this process in the past and quit it. They estimate that less than 20% of CVEs filed are relevant (I would have thought even less to be honest).

But yeah the process is totally broken and non-transparent. Other projects such as SQLite don't want to hear about CVE anymore for these reasons. In HAProxy we don't file them any more after an embargoed CVE was twitted with only the subject, but enough details to hurry up the process. Instead we now let distro package maintainers decide if it makes their life easier with or without one and leave it to them to file these. When they do, they fill them correctly because they're more accustomed to the process. In the end it's a clean and efficient sharing of the roles so it's not that bad.

One thing I've been tempted to do many times is to file one CVE per backported patch, at least to ensure that all backports land into distros, and not have to respond to questions about CVEs anymore. In our case it would cause a real burden to our package maintainers and since they're doing a great job I don't want to do that. It would have been different otherwise :-)

I tried to educate coworkers and other people about focusing on "bug fixes" and not "CVE" for backports, but it doesn't seem to take off. There's so heavy an industry hammering "CVE" all the day that they manage to make anyone believe this has anything to do with security, while it's just a business consisting in selling lists and automated tools checking for the presence of a tiny portion of bugs affecting software, and this business is so lucrative that it's almost hopeless to be able to open other people's eyes about what it really is. It's particularly annoying because real bugs are rarely fixed and CVE make lots of noise.

The bogus CVE problem

madhatter — Thu, 14 Sep 2023 07:14:54 +0000

You do make a depressingly good point about software makers, particularly commercial ones, being incentivised to handwave vulnerabilities away. Bruce Schenier wrote this pithy response in last month's Crypto-Gram newsletter, in response to a vendor quote in a vice.com article about a vulnerability found in Tetra police radios:

> Specifically on the researchers’ claims of a backdoor in TEA1, Boyer added “At this time, we would like to
> point out that the research findings do not relate to any backdoors. The TETRA security standards have
> been specified together with national security agencies and are designed for and subject to export
> control regulations which determine the strength of the encryption.”

And I would like to point out that that’s the very definition of a backdoor.

The bogus CVE problem

Trou.fr — Thu, 14 Sep 2023 07:03:02 +0000

Nvd does not assign and rate vulnerabilities, this is done by any of the 300+ CNA, as long as the product that is vulnerable is not published by a CNA itself.
AFAIK, nvd is exactly what is says, a database.
That's not to say the CVE/CVSS is broken, but it is still the only tool that allows people and organisations to track which vulnerability they should patch or they have patched.
And BTW the CVSS 4.0 https://www.first.org/cvss/v4-0/ is going to be published soon, it should fix some of the problems of v3 (including the impossibility to reach 10)

The bogus CVE problem

wahern — Thu, 14 Sep 2023 00:04:44 +0000

All this is true enough, but proving that a vulnerability is protected by an "airtight hatchway" (i.e. is unexploitable in practice) is difficult, often more difficult than identifying the vulnerability. Not only is it difficult, but implementations and vendors are highly incentivized to wave away vulnerabilities with claims that they're unexploitable. But in most cases claims are never proven systematically, rather averred. And such assertions often are merely failures of imagination--defenders don't think like attackers, even when they're being earnest and diligent.

The CVE system has flaws--serious flaws. But I'd argue that these flaws are less severe than the alternative where vendors can more easily waive off reports. Let's not forget from whence we came--a time when vendors didn't take these things seriously unless and until bugs were already being conspicuously exploited in the wild. And we still have problems with too many vendors not taking CVEs seriously. The current system is unfair to conscientious developers and maintainers whose time is wasted by self-aggrandizing bug reporters, but we would never have needed the current CVE system if such conscientious people were the majority of those shipping software.

* There's a reason we can't have nice things. * No good deed goes unpunished. * Etc, etc.

The bogus CVE problem

flussence — Wed, 13 Sep 2023 21:55:38 +0000

CVEs are exactly the same class of scam as Better Business Bureau ratings. Anyone can make them up, they have a false air of reputability, and nowadays they're more often used as a force-multiplying tool in cash shakedowns than their stated purpose.

If NVD/MITRE don't massively clean up their act following this, the next step is to start removing baked-in recognition of CVE numbers in the many tools that have it, and educating the general public that those organisations possess no legitimate authority. IMO this part of the industry is at least a decade overdue for a scorched-earth cleanup like what Let's Encrypt did to the CA aristocracy.

The bogus CVE problem

geofft — Wed, 13 Sep 2023 21:10:12 +0000

William Woodruff's post from last year ReDoS "vulnerabilities" and misaligned incentives is another good look at this problem focusing particularly on people's excitement with superlinear regular expressions, and points out a very practical issue: a lot of automated security tools will report issues if any of the dependencies you're using have unresolved CVEs. This is well-intentioned, but for things that aren't actually vulnerabilities, it leads to needless churn and panic, and for things that are so much not a vulnerability that they aren't fixed, or aren't fixed except in the current development release, or similar, it may even cause deploy tools to refuse to deploy. Or, worse, it may cause well-funded corporate users to make demands of open-source projects to backport a patch that has no need to be backported. When there's an actual bug to fix, we can just lament the open-source sustainability problem, but when there isn't, it's nothing other than rude.

I actually think that this is, essentially, a DoS vulnerability in the CVE process itself. Anyone can submit an entry into a database that gets insufficiently validated and causes other people to have to scramble and respond and maybe makes their automated systems break. That's unauthenticated input causing denial-of-service attacks!

Also it occurs to me that CVE + CVSS actually does kind of provide a solution here. The point of CVE is that it's common, i.e., if someone else discovers the same vulnerability, there's a way to ideally give them the same CVE identifier or at least mark their new one as a duplicate. It feels like we should be able to proactively mark things as CVSS score 0, and say, yes, this is an integer overflow or stack corruption or whatever, but exploiting it rather involves being on the other side of this airtight hatchway, and so we're going to mark the fact that it exists so people who find it know it was already analyzed. And in some cases it makes sense to "fix" it, as with CVE-2020-19909. It's always better to print a real error message instead of "double free or corruption (fasttop)". (But I do wonder about the incentive system, as Woodruff mentions, of having CVEs on your résumé, and whether the existence of CVEs with CVSS scores of 0 will make things better or worse.)