The US military wants to understand the most important software on Earth (MIT Technology Review) [LWN.net]

The MIT Technology Review has posted an article on a program within the US Defense Advanced Research Projects Agency to identify threats to open-source code.

The ultimate goal is to detect and counteract any malicious campaigns to submit flawed code, launch influence operations, sabotage development, or even take control of open-source projects. To do this, the researchers will use tools such as sentiment analysis to analyze the social interactions within open-source communities such as the Linux kernel mailing list, which should help identify who is being positive or constructive and who is being negative and destructive.

to post comments

The US military wants to understand the most important software on Earth (MITTechnology Review)

Posted Jul 14, 2022 18:00 UTC (Thu) by NightMonkey (subscriber, #23051) [Link] (1 responses)

Hmm... So, this will make more "lists" I can be on that I have no ability to contest? Now, the United States has 'no-fly' lists, will we soon have a 'no-code' list? And will the Pentagon decide if a developer can submit code to the Kernel, rather than the Kernel devs?

Of course, they probably won't address one of the biggest threats to 'open source code', namely, large corporations and governments who add "negative and destructive" friction onto individuals who are participating, or would like to participate, in Open Source development and communities. Can forcing engineers to run a locked-down Windows environment for 8-10 hours a day be seen as such a threat? ;) Call the Pentagon! :D

The US military wants to understand the most important software on Earth (MITTechnology Review)

Posted Jul 17, 2022 5:41 UTC (Sun) by NYKevin (subscriber, #129325) [Link]

In 2019, the US Air Force announced that it had just figured out how to launch a nuclear missile without using 8-inch floppy disks. My assumption, therefore, is that the word "understand" in the headline is, perhaps, an exaggeration, and the Pentagon actually just wants to verify that their dependencies are not full of random crap. Which is a legitimate concern, because everyone's dependencies seem to be full of random crap these days. They're couching it in terms of "threats" because that is how you get the US military to sit up and pay attention to you, not necessarily because they actually plan to put individual humans on a list or anything of that nature. I'm also rather skeptical that they'll get anything useful out of their buzzword bingo of machine learning keywords, but at least it's slightly less dumb than web3.

Source: https://www.nytimes.com/2019/10/24/us/nuclear-weapons-flo...

The US military wants to understand the most important software on Earth (MITTechnology Review)

Posted Jul 14, 2022 18:18 UTC (Thu) by aklaver (guest, #62352) [Link]

"... such as the Linux kernel mailing list,which should help identify who is being positive or constructive and who is being negative and destructive.". Well they are picking target rich environment. So does this mean rejecting patches identifies you as a security risk?

The US military wants to understand the most important software on Earth (MITTechnology Review)

Posted Jul 14, 2022 18:28 UTC (Thu) by NightMonkey (subscriber, #23051) [Link] (7 responses)

'“This subject kills me,” says d’Antoine of the quest to better understand the open-source movement, “because, honestly, even the most simple things seem so novel to so many important people. The government is only just realizing that our critical infrastructure is running code that could be literally being written by sanctioned entities. Right now.”'

This subject kills me, too. So, at a certain level, all code is a math expression (where it is either compiled or interpreted down to machine code). So, if an Iranian or North Korean, Russian soldier or Ukranian Azov Battalion member, or Israeli or Palestinian, or a clerk in the Pentagon says "1+1=2", does that change the truth of it? Other parts of the article basically hint at a worldview that colors the very openness of the development process as a threat.

Many people, myself included, appreciate the openness of F/OSS because the code doesn't care about your flag, your location, your mode of living, your love or hatred of arugula, or the color of your skin, or your place on the gender spectrum. Or even if you are a dog. :D (https://en.wikipedia.org/wiki/On_the_Internet%2C_nobody_k...'re_a_dog)

The US military wants to understand the most important software on Earth (MITTechnology Review)

Posted Jul 14, 2022 23:38 UTC (Thu) by gerdesj (subscriber, #5446) [Link] (3 responses)

Well at least the article stops short of accusing everyone of being closet commie bastards. Who on earth allowed that piece of tripe to worm its way out from the 1950s and rewrite itself in 2020s terms?

When you see that sort of nonsense coming out of a well respected org such as MIT as a puff piece then you know that you had better avoid being "undesirable" and start being a better person. This bloke: https://www.technologyreview.com/author/patrick-howell-on... is one role model for you who can trot out this sort of drivell without whincing.

I understand that one must dumb down somewhat when communicating with the hoi polloi and other generally unwashed types but this is an article that clearly explains to children that we can't trust open source code because we can read it at any time. You can't blindly go around trusting something you can delve into and review yourself. You should allow adults to do that for you.

I'm acutely aware of many of the flaws that turn up in FLOSS - I follow dozens of bugzillas etc and mailing lists that exhaustively discuss how to deliver next month's bugs effectively and on schedule. I have some insights into the sheer effort that say jra goes to to screw up my Samba experience or some of you lot do with delivering Linux and that corbet bloke and his dodgy website.

I also get to tread the Patch Wednesday (yes weds not tues - "let he who is without fear ...") treadmill with absolutely no idea what is going on but I do it anyway: yay - CVEs with serious sounding flaws and some jolly exciting write ups but I can't look at the code - its a bloody cargo cult thing. Getting to the bottom of some of the weirder corners of Windows is quite a challenge - for example: AdminSdHolder - who knew, until you knew! What a load of cobblers.

https://techcommunity.microsoft.com/t5/ask-the-directory-... - Why would you? That's wankery in action - We've bodged a solution/papered over some cracks and expect you to do some weird shit. Soz/lol, that's the thing you engage when you do things like create a service account that can only change passwords without being a domain admin. You fiddle with perms on a LDAP container object to give rights to a user type object and ADUC can't do that sort of thing (lol).

Anyway, I doubt that the US military hasn't noticed where their software is coming from nor how it is written.

The US military wants to understand the most important software on Earth (MITTechnology Review)

Posted Jul 15, 2022 12:37 UTC (Fri) by eduperez (guest, #11232) [Link]

> I understand that one must dumb down somewhat when communicating with the hoi polloi and other generally unwashed types but this is an article that clearly explains to children that we can't trust open source code because we can read it at any time. You can't blindly go around trusting something you can delve into and review yourself. You should allow adults to do that for you.

I think they are more worried about who wrote the code than who can read it:

> "The ultimate goal is to detect and counteract any malicious campaigns to submit flawed code, launch influence operations, sabotage development, or even take control of open-source projects."

The US military wants to understand the most important software on Earth (MITTechnology Review)

Posted Jul 15, 2022 19:55 UTC (Fri) by vulpicastor (subscriber, #122452) [Link] (1 responses)

When you see that sort of nonsense coming out of a well respected org such as MIT as a puff piece

The MIT Technology Review is editorially independent from other parts of MIT, so it’s inaccurate to lump it together with, say, the school’s PR department, which works for the leadership. In any case, for any sufficiently large organization, the left hand probably doesn’t know what the right hand is doing.

The US military wants to understand the most important software on Earth (MITTechnology Review)

Posted Jul 18, 2022 6:33 UTC (Mon) by ceplm (subscriber, #41334) [Link]

Besides MIT is known to do anything which brings dough to them. And usually military projects bring a lot of it. After all they brought linguistics from that impoverished liberal arts college upstream just because DOD believed in the machine translation of the Russian voice communication. Yes, it was 1960s.

The US military wants to understand the most important software on Earth (MITTechnology Review)

Posted Jul 15, 2022 13:08 UTC (Fri) by IanKelling (subscriber, #89418) [Link] (2 responses)

I didn't read the article yet, but I don't agree at all. I think your sentiment is true for most code, but there is also some very important code where it isn't true.

The US military wants to understand the most important software on Earth (MITTechnology Review)

Posted Jul 15, 2022 15:21 UTC (Fri) by NightMonkey (subscriber, #23051) [Link] (1 responses)

OK, I see your assertions. Thanks for sharing. But can you explain your reasoning behind your opinion? I'm genuinely curious where my flaw in reasoning here is, if there is one.

I think you should probably read the article before replying, though. :)

The US military wants to understand the most important software on Earth (MITTechnology Review)

Posted Jul 17, 2022 19:35 UTC (Sun) by IanKelling (subscriber, #89418) [Link]

Well, even this comment section: it doesn't encourage you to publish your name or anything about you. That is not about math. Lessig's book codev2 seems to give various examples https://commons.wikimedia.org/wiki/File:Code_v2.pdf

The US military wants to understand the most important software on Earth (MITTechnology Review)

Posted Jul 14, 2022 18:33 UTC (Thu) by Cyberax (✭ supporter ✭, #52523) [Link] (2 responses)

So... Patches delivered via ICBMs?

The US military wants to understand the most important software on Earth (MITTechnology Review)

Posted Jul 14, 2022 18:38 UTC (Thu) by amacater (subscriber, #790) [Link] (1 responses)

https://en.wikipedia.org/wiki/ICBM_address maybe? [Which in turn looks like a straight copy from the Jargon File entry].

The US military wants to understand the most important software on Earth (MITTechnology Review)

Posted Jul 14, 2022 20:07 UTC (Thu) by Cyberax (✭ supporter ✭, #52523) [Link]

Apparently, my idea is not novel. USPS has already done mail delivery via a rocket: https://www.popularmechanics.com/flight/a21601/usps-first...

The US military wants to understand the most important software on Earth (MITTechnology Review)

Posted Jul 14, 2022 18:40 UTC (Thu) by pebolle (guest, #35204) [Link] (8 responses)

> [T]he researchers will use tools such as sentiment analysis to analyze the social interactions within open-source communities such as the Linux kernel mailing list, which should help identify who is being positive or constructive and who is being negative and destructive.

Likewise, from today's Security quote of the week:
> Detecting hate speech is a good proxy for terrorist radicalisation. In 2018, we thought we could detect hate speech with a precision of typically 92%, which would mean a false-alarm rate of 8%.

Both quotes immediately triggered my "Snake oil" alarm.

(I do hope my alarm is calibrated correctly, because I find the approaches advocated in those quotes creepy beyond belief.)

The US military wants to understand the most important software on Earth (MITTechnology Review)

Posted Jul 14, 2022 22:38 UTC (Thu) by Wol (subscriber, #4433) [Link]

> Likewise, from today's Security quote of the week:

> > Detecting hate speech is a good proxy for terrorist radicalisation. In 2018, we thought we could detect hate speech with a precision of typically 92%, which would mean a false-alarm rate of 8%.

The follow-on to that is good, though ...

In 2022, now we understand the problem better, our ability to detect hate speech has gone DOWN...

Cheers,
Wol

The US military wants to understand the most important software on Earth (MITTechnology Review)

Posted Jul 15, 2022 9:55 UTC (Fri) by dottedmag (subscriber, #18590) [Link] (5 responses)

8% false-alarm (false positives) rate is... huge for such a small target.

A napkin math: if you have 1M participants, 100 participants are terrorists, and the test has 0% false negatives, then this test would drag in 100 real terrorists and 79992 falsely accused ones.

The US military wants to understand the most important software on Earth (MITTechnology Review)

Posted Jul 15, 2022 11:05 UTC (Fri) by taladar (subscriber, #68407) [Link] (3 responses)

Actually you would only get 100 terrorists if you have a 0% false negative rate which is unlikely if your false positive rate is that high.

The US military wants to understand the most important software on Earth (MITTechnology Review)

Posted Jul 15, 2022 21:10 UTC (Fri) by Wol (subscriber, #4433) [Link] (1 responses)

> unlikely if your false positive rate is that high.

Well actually, if your false positive is high, then your false negative is likely to be low ... if your false positive is that high, you'll probably get 99 out of 100 real terrorists.

I can't remember the exact terminology, but tests either tend to be very good at picking up the target, OR very good at not picking up non-targets. Of course, Sod makes it very hard to run both tests over the same dataset :-)

(We had that with CoVid - tests were either very sensitive and picked up every genuine case along with a lot of false positives, or very specific and didn't pick up false negatives but let genuine cases slip through.)

Cheers,
Wol

The US military wants to understand the most important software on Earth (MITTechnology Review)

Posted Jul 20, 2022 6:06 UTC (Wed) by riking (subscriber, #95706) [Link]

Ah, you're thinking about good tests. Machine Learning tends to be bad at both simultaneously.

The US military wants to understand the most important software on Earth (MITTechnology Review)

Posted Jul 16, 2022 13:45 UTC (Sat) by farnz (subscriber, #17727) [Link]

Depends on the details of the test. We have a 2x2 confusion matrix of test result versus real result, and the false positive rate tells us how many samples fall in one of the 4 cells of the matrix - in this case, how many are in the "test says yes" column and the "real world says no" row. This gives us a decent chance of guessing at the behaviour of the system when the "test says yes", and when the "real world says no", but we need more data to be able to say something about the behaviour of the system when the "test says no" or when the "real world says yes".

In particular, it's common to have a low false negative rate with a high false positive rate, or vice-versa, since the underlying judgement is likely to be a confidence level and a threshold; if you set the threshold low, you have very low false negative rates, but very high false positive rates, while if you set the threshold high, you get very high false negative rates, but very low false positive rates.

The US military wants to understand the most important software on Earth (MITTechnology Review)

Posted Jul 15, 2022 16:20 UTC (Fri) by NYKevin (subscriber, #129325) [Link]

Here is my favorite demonstration of this principle: https://www.youtube.com/watch?v=vBPFaM-0pI8

The US military wants to understand the most important software on Earth (MITTechnology Review)

Posted Jul 15, 2022 22:53 UTC (Fri) by gerdesj (subscriber, #5446) [Link]

> Both quotes immediately triggered my "Snake oil" alarm.

Not half. The term "sentiment analysis" is causing my left eye to twitch and a vein to throb. I may go postal soon 8)

The US military wants to understand the most important software on Earth (MITTechnology Review)

Posted Jul 14, 2022 19:58 UTC (Thu) by amarao (guest, #87073) [Link] (11 responses)

The best way to classify people is to see who is writing tabs instead of spaces. Those are very different groups with very little common ground and with almost no shared values.

The US military wants to understand the most important software on Earth (MITTechnology Review)

Posted Jul 15, 2022 4:16 UTC (Fri) by nksingh (subscriber, #94354) [Link]

Or the people who like to nybble on their bytes from the big end or the little end.

The US military wants to understand the most important software on Earth (MITTechnology Review)

Posted Jul 15, 2022 9:41 UTC (Fri) by adobriyan (guest, #30858) [Link] (4 responses)

Those preincrementing variables in for loop are very suspicious.

The US military wants to understand the most important software on Earth (MITTechnology Review)

Posted Jul 17, 2022 5:46 UTC (Sun) by NYKevin (subscriber, #129325) [Link] (3 responses)

Unfortunately, in C++, preincrement is sometimes more efficient than postincrement (usually when the variable is not a primitive type), and the sickness has spread from there back into C.

The US military wants to understand the most important software on Earth (MITTechnology Review)

Posted Jul 17, 2022 13:05 UTC (Sun) by mathstuf (subscriber, #69389) [Link]

Once C++ developers get more into the habit of `for (type x : c)`, hopefully that would stop happening :) .

The US military wants to understand the most important software on Earth (MITTechnology Review)

Posted Jul 17, 2022 22:15 UTC (Sun) by rschroev (subscriber, #4164) [Link] (1 responses)

Why do you consider preincrement a sickness? What are the arguments for preferring postincrement?

The US military wants to understand the most important software on Earth (MITTechnology Review)

Posted Jul 18, 2022 1:53 UTC (Mon) by mathstuf (subscriber, #69389) [Link]

I would guess it is mostly about code churn/noise from contributors flipping from one to the other.

The US military wants to understand the most important software on Earth (MITTechnology Review)

Posted Jul 15, 2022 10:17 UTC (Fri) by Thomas (subscriber, #39963) [Link] (4 responses)

Tabs - spaces
vim - emacs
CLI - GUI
sysvinit - systemd
case-sensitive FS - case-insensitive FS
Dark mode - bright mode

You name it, clearly hate speech and terrorists everywhere. There is a lot of intel for the military to gain.

The US military wants to understand the most important software on Earth (MITTechnology Review)

Posted Jul 15, 2022 12:46 UTC (Fri) by amarao (guest, #87073) [Link] (2 responses)

Given that literature already has a story of a devastating civil war due to endianness disagreement, we surely can expect a rather bloody uprising from Devuan extremists.

The US military wants to understand the most important software on Earth (MITTechnology Review)

Posted Jul 15, 2022 13:35 UTC (Fri) by Narusegawa (guest, #159714) [Link] (1 responses)

Extreme thoughts float in the heads of those who express them out of place.

Literary reference

Posted Jul 24, 2022 6:57 UTC (Sun) by CChittleborough (subscriber, #60775) [Link]

That civil war occurs in a satirical novel by the guy who wrote “A Modest Proposal”, an (in?)famous satire. I’m sure that amarao’s comment is also satire.

The US military wants to understand the most important software on Earth (MITTechnology Review)

Posted Jul 15, 2022 13:24 UTC (Fri) by Thomas (subscriber, #39963) [Link]

Sorry. It is of course

Dark mode - Light mode

and not Bright mode, but you got the point.

The US military wants to understand the most important software on Earth (MITTechnology Review)

Posted Jul 14, 2022 20:57 UTC (Thu) by flussence (guest, #85566) [Link]

So to enact this understanding they're going to... outsource it to some of the least understandable software on the planet? Sure, throw some Machine Laundering at it, have the computer hallucinate an interpretation that reinforces existing (and probably horrifically bigoted) biases. Nothing good will come of this.

The US military wants to understand the most important software on Earth (MITTechnology Review)

Posted Jul 15, 2022 3:40 UTC (Fri) by pabs (subscriber, #43278) [Link]

Wonder if this would have detected the influence campaign to move away from copyleft towards permissive licenses by corporations, which seems to have largely been successful unfortunately.

The US military wants to understand the most important software on Earth (MITTechnology Review)

Posted Jul 15, 2022 12:26 UTC (Fri) by clugstj (subscriber, #4020) [Link] (2 responses)

The title is purposely inflammatory. Calling DARPA "The US military" is quite a stretch. It's a research project for deity's sake. Trying to determine if code that you depend upon has been tampered with is quite understandable. The methods may be flawed, but no one should be surprised that they are looking into it.

The US military wants to understand the most important software on Earth (MITTechnology Review)

Posted Jul 15, 2022 15:34 UTC (Fri) by NightMonkey (subscriber, #23051) [Link] (1 responses)

I personally like it, and feel it is accurate and clear - which is one of the things I love about LWN. I'm a fan of returning the "Defense Department" to the "War Department", and I like clear, non-DoubleSpeak language. "The US War Department's Research Division" would perhaps be even better. :)

What is surprising is how propagandized the US and its allies are, in that we can accept DoubleSpeak almost everywhere.

DARPA is the source of many important technological innovations, no doubt. As a taxpayer, I'd prefer that we just fund the research, and the funds not have to go through the military.

And I like it when the press shows a dedication to clarity and accuracy, which is very rare today. And this is one reason I support LWN.

The US military wants to understand the most important software on Earth (MITTechnology Review)

Posted Jul 15, 2022 17:07 UTC (Fri) by Wol (subscriber, #4433) [Link]

> DARPA is the source of many important technological innovations, no doubt. As a taxpayer, I'd prefer that we just fund the research, and the funds not have to go through the military.

That's American Imperialism at work :-)

They fund lots of "military research", which has to go to American firms, and then scream blue murder when their companies are not allowed to bid for or buy up European research projects ...

Cheers,
Wol

Ethically measuring a Free/Open project's ability to withstand malicious "contributions"

Posted Jul 15, 2022 13:34 UTC (Fri) by somlo (subscriber, #92421) [Link] (1 responses)

I've been thinking about that for a while now. It's an interesting (and, if you think about it, important) question. Just not sure there's a good way to go about it ethically.

I'm reminded of the UMN debacle, as one particularly ham-handed example of how *NOT* to go about designing an experiment to answer the question...

Ethically measuring a Free/Open project's ability to withstand malicious "contributions"

Posted Jul 17, 2022 2:34 UTC (Sun) by marcH (subscriber, #57642) [Link]

> I'm reminded of the UMN debacle, as one particularly ham-handed example of how *NOT* to go about designing an experiment to answer the question...

Well it's much easier if your final goal is to actually add vulnerabilities, not publish a research paper on how it can be done. People who did the former simply did not talk about it and the vulnerabilities they added are still there. If they get caught at some later point they'll just say "oops! Sorry"; C makes deniability very easy.

I've seen a lot of comments like this one about the experiment = blaming the messenger. Even when correct, neither interesting nor relevant. I haven't read much about actual security gaps in the kernel processes. I hope I missed that.

The US military wants to understand the most important software on Earth (MITTechnology Review)

Posted Jul 17, 2022 7:57 UTC (Sun) by ssmith32 (subscriber, #72404) [Link] (1 responses)

"It’s not much of an exaggeration to say that the whole world is built on top of the Linux kernel—"

No, it's turtles, all the way down. Clearly.

The US military wants to understand the most important software on Earth (MITTechnology Review)

Posted Jul 17, 2022 12:47 UTC (Sun) by Wol (subscriber, #4433) [Link]

But they clearly don't know their hardware - even Windows runs on top of Minix nowadays ...

Cheers,
Wol