LWN: Comments on "DeVault: GitHub Copilot and open source laundering"

Not trained on MS closed source

glenn — Fri, 11 Nov 2022 22:16:17 +0000

It took some time, but there's this one: https://lwn.net/Articles/914150

It's not unthinkable that users of the tool who for commercial purposes could also be sued.

DeVault: GitHub Copilot and open source laundering

mathstuf — Tue, 30 Aug 2022 15:22:18 +0000

IIRC, search was hampered with rate limits and "must be logged in to search repo contents" since before the acquisition.

DeVault: GitHub Copilot and open source laundering

scientes — Tue, 30 Aug 2022 13:27:03 +0000

I think there is too much attention to this feature rather the freedom-stealing aspects of github.

What I noticed is that github has serious rate-limiting on their search functionality, but google's codesearch project (which was a nerdy project and the RE2 regex library was built for it) embraced these type of searches. It looks like Microsoft is hostile to technical proficiency and just wants control and classical Microsoft things.

DeVault: GitHub Copilot and open source laundering

nim-nim — Fri, 01 Jul 2022 08:32:20 +0000

It would be very hard to argue in court that code produced by an AI trained on an existing code base is anything else than a derivation of this code base, since there is no human inventivity involved to muddy up waters.

Despite years of commercial pretense to the reverse professionals know “smart” systems are no smarter than the human who coded them.

Not trained on MS closed source

nim-nim — Thu, 30 Jun 2022 13:45:46 +0000

FOSS is not the same thing as public domain, it is all copyrighted

DeVault: GitHub Copilot and open source laundering

farnz — Wed, 29 Jun 2022 11:31:39 +0000

It's also worth noting in this context that part of the reason to have vendor scripts of some form (be they AtomBIOS, ACPI or others) is that power delivery changes can't happen instantly - and different board manufacturers will have done different transient analysis to determine what their hardware can reliably support. The results of that analysis (whether it's a "rule of thumb" assessment or a proper calculation) need communicating to the driver somehow - and a small scripting language is as good as any other way to deal with it, especially since the edge cases get complex if you're doing a per-board calculation based on measuring the final system during post-manufacture testing of a board.

That said, given the quality of some vendor code, I understand Luc's reluctance to trust it - I've encountered one vendor who asserted that the CPU would detect an OUT 0xCF8, EAX instruction in userspace and then ensure that nothing else accessed PCI configuration space until the same userspace process executed either OUT 0xCFC, EAX or IN EAX, 0xCFC later, on the basis that if the CPU didn't do that, it would be possible for their userspace driver to crash. I'm not even sure how this could work under Linux…

DeVault: GitHub Copilot and open source laundering

mjg59 — Wed, 29 Jun 2022 10:31:06 +0000

The easiest way to think about Atom is like ACPI - it's a lump of bytecode that's interpreted in kernel context, and which talks to hardware the kernel could already talk to. We could rip out the ACPI interpreter in the kernel and hardcode knowledge of every singly ACPI board instead, and we'd have successfully replaced non-free code with free code. But we'd also have a kernel that didn't boot on a bunch of new systems until people had reverse engineered the relevant ACPI code and reimplemented it, and so nobody has seriously suggested doing that.

There's tradeoffs. I'd love to avoid having to rely on non-free code to make hardware work, and I'm not going to criticise people for writing drivers that do that. But the reality is that any such driver is going to work less well than a driver that uses the defined interface to call non-free firmware (in much the same way that we call into non-free UEFI code to set boot variables these days), and so there's value in the driver that calls non-free code existing, and also it's unsurprising that distros would pick the one that works with more hardware.

Luc's priorities on -radeonhd were probably based on his experience with the VIA chipsets that were extremely limited by what the BIOS permitted (and yeah it turns out that not being able to set any modes other than those that are hardcoded in the BIOS is not good!), but the outcome was also that his driver for those chipsets simply didn't work on all hardware - I had a VIA-based laptop that would just give a black screen with his driver, because the BIOS didn't match his expectations. To be completely fair, on Radeon I hit some similar constraints when I was researching reclocking the RAM for power management - the Atom scripts simply took too long, so I took out a bunch of the wait statements and hardcoded those into the kernel and it was great, and then after a couple of days of uptime the card would wedge during a reclock and also it didn't work on all hardware, so it turns out there was a reason that those were there in the first place. So I absolutely understand the desire to have native code for all of this, but also in the absence of vendors providing explicit contracts about hardware behaviour, a driver that doesn't use the defined interfaces is inherently going to break things.

DeVault: GitHub Copilot and open source laundering

flussence — Wed, 29 Jun 2022 10:12:22 +0000

That first paragraph's a letdown, but...

I learned something today. Thanks for explaining all that to my dumb ass.

This is the kind of thing that makes me keep my subscription to the site going.

DeVault: GitHub Copilot and open source laundering

Wol — Tue, 28 Jun 2022 08:00:05 +0000

(And to gspr)

Oh the joys of the ambiguity of English ...

As I read it "copyright attaches to the model" - in other words there is no copyright *in* the model. But if there is copyright in the *original*, then that applies *to* the model as well ...

I think we're talking at cross purposes ... :-)

Cheers,
Wol

DeVault: GitHub Copilot and open source laundering

gspr — Tue, 28 Jun 2022 06:37:56 +0000

By this logic, unlicensed redistribution of copyrighted images, video and audio are fine as long as the material is recompressed. It's just math!

Clearly absurd.

DeVault: GitHub Copilot and open source laundering

hummassa — Tue, 28 Jun 2022 01:26:16 +0000

I will offer an informed alternative to your pov, Wol:

If a mathematical model produces as its output a perfect reproduction of an copyrightable and copyrighted work (something novel produced by the human mind), then said mathematical model is nothing but a copying apparatus. There is no difference between the neural model of Copilot and a big HD containing all the works it's seen, just well indexed. The output is just a copy of the copyrighted work, subject to the same protections under the laws and treaties.

Not trained on MS closed source

bluca — Mon, 27 Jun 2022 18:31:00 +0000

Or more accurately, it is trained on publicly (legally) available corpora, as the law requires to be excepted from copyright restrictions

DeVault: GitHub Copilot and open source laundering

Wol — Mon, 27 Jun 2022 16:51:40 +0000

Are you talking about a real-world model - like you made an aircraft model from a photo of an aircraft? In that case it's a physical object and patent rules should apply, or design rules, and NOT copyright rules.

If, however, you're referring to a model like most people here are - computer science, otherwise known as maths - then equally copyright should NOT apply, because it's maths. Or "sweat of the brow". Or a whole other bunch of doctrines that lawyers do their best to mis-understand but that state quite clearly it is not copyrightable material.

Cheers,
Wol

Not trained on MS closed source

excors — Mon, 27 Jun 2022 14:44:36 +0000

I think there are significant non-copyright reasons to exclude non-public code. E.g. The Copilot FAQ says:

> Because GitHub Copilot was trained on publicly available code, its training set included public personal data included in that code. From our internal testing, we found it to be rare that GitHub Copilot suggestions included personal data verbatim from the training set. [...] We have implemented a filter that blocks emails when shown in standard formats, but it’s still possible to get the model to suggest this sort of content if you try hard enough. We will keep improving the filter system to be more intelligent to detect and remove more personal data from the suggestions.

"rare" != "never", and if someone stores sensitive personal data in a private GitHub repository then they absolutely don't want Copilot to reveal that information publicly to anyone who tries hard enough.

I expect the same applies to other confidential information, like yet-to-be-announced product names that companies might store in private repositories, or secret keys, or algorithms that they're protecting as trade secrets, etc.

Since the Copilot training data apparently includes public repositories even if they have a restrictive license, but excludes private repositories even if they have a very permissive license, it sounds like GitHub is confident that there are no copyright issues but is concerned about those other privacy issues.

Not trained on MS closed source

geert — Mon, 27 Jun 2022 14:26:28 +0000

"trained on A, including B", does not mean that it was trained on B only.
So it may have been trained on any source code that is publicly available; we don't know what exactly, the description in the FAQ is very vague (deliberately?).

FWIW, it might have been trained on whatever proprietary code that was ever leaked to the Internet, which might even include the sources of some version of Microsoft Windows ;-)

DeVault: GitHub Copilot and open source laundering

anselm — Mon, 27 Jun 2022 14:18:26 +0000

For the purposes of determining whether X is a derivative work of Y, the judge looks at X, Y (its contents, not its license), and the copyright statute. Nothing else.

I think it would still be of some interest whether X resulted from Y through “cp Y X” or through a query to Copilot whose model was trained on Y. In the first case, X is pretty clearly a derived work of Y. In the second case, Microsoft, at least, would probably like to claim it isn't.

Not trained on MS closed source

nim-nim — Mon, 27 Jun 2022 12:45:01 +0000

Or more accurately, they are *very* sure of the (il)legality of their approach and only dare to use it against projects that they feel won’t fight back.

Microsoft has access to plenty enough of proprietary code to train a model on, that they chose to use other people’s code instead says volumes.

DeVault: GitHub Copilot and open source laundering

mathstuf — Mon, 27 Jun 2022 09:22:24 +0000

Eh. `git-remote-hg` exists and works well enough that it's the only way that I interact with Mercurial repos at least.

Not trained on MS closed source

NYKevin — Mon, 27 Jun 2022 05:41:28 +0000

GitHub does not require uploaded code to be FOSS, as long as it allows the use of GitHub's "fork" button (loosely equivalent to git clone), and one or two other pieces of site functionality. In theory, they could have limited their training set to only include FOSS repositories, and not proprietary or no-license repositories (in most jurisdictions, no license means the same thing as "all rights reserved").

But I couldn't find any statement in their FAQ one way or the other - it just refers to "public repositories on GitHub," a category including both FOSS and proprietary code. It's entirely possible that they are using all of that code, and IMHO that seems like the most straightforward way to read the sentence (which doesn't mean that it is the intended meaning, of course).

Not trained on MS closed source

bluca — Mon, 27 Jun 2022 00:31:42 +0000

Exactly which proprietary stuff is publicly available and wasn't used?

Not trained on MS closed source

LtWorf — Sun, 26 Jun 2022 22:27:38 +0000

There's plenty of proprietary stuff on github that they didn't dare to use.

I think this alone shows that they are not very sure about the legality of what they are doing, but trust that developers won't be able to do anything about it (unlike the paying customers).

DeVault: GitHub Copilot and open source laundering

sjj — Sun, 26 Jun 2022 20:16:01 +0000

Isn't 20 years enough to carry a grudge? Those middle managers have moved on a long time ago already.

DeVault: GitHub Copilot and open source laundering

mjg59 — Sun, 26 Jun 2022 19:40:50 +0000

> the R200/(reverse-engineered)R300 driver also used to be blob-free

The DRM side of the R2/3/400 driver always required a firmware blob - but for a long time it was just embedded inside the kernel driver, so wasn't user visible. My recollection (which seems to be supported by the driver, but it's been a long time since I looked at this properly so I could be wrong) is that a bunch of the 2D acceleration in that driver depended on DRM, so effectively the 2D driver also had a blob dependency if you wanted it to work properly.

The difference between -radeonhd and -ati as far as reliance on firmware goes was that the defined interface to various pieces of card functionality was to execute interpreted scripts present in the card flash. These scripts didn't do anything that the driver couldn't, so you could absolutely reimplement that functionality in the driver - the problem is that card vendors used these scripts as a way to abstract hardware differences (eg, using RAM from different vendors with different timing constraints), and ignoring Atom would mean having to have card-specific data in the driver before that card would work correctly. A hybrid approach is to use Atom for data but not for code, but even then there are still risks due to the fact that the defined interface is the scripts and not the data tables. A card vendor could modify the way the script interpreted the tables (or even hardcode stuff directly into the script) and again you'd need card-specific knowledge to avoid that. -radeonhd spent a while trying to avoid executing any Atom code, but effectively relied on it anyway - it couldn't program the card from cold and so depended on the system firmware having executed the scripts before it ran. In any case, support for executing Atom code (including running the ASIC init function) was added to -radeonhd by September of 2007.

Looking at the initial commits to support r500 in the -ati driver, I think the only time it would ever call int10 is if the card was entirely uninitialised. -radeonhd would do exactly the same if it was configured without support for doing Atom-based init.

DeVault: GitHub Copilot and open source laundering

Cyberax — Sun, 26 Jun 2022 18:49:02 +0000

> Not many people may remember this now, but the R200/(reverse-engineered)R300 driver also used to be blob-free. Strangely that stopped being the case after AMD took over, even though it was feature-complete.

Because it made no sense to reimplement the critical power management and link training code multiple times, instead of doing it once in AtomBIOS.

DeVault: GitHub Copilot and open source laundering

flussence — Sun, 26 Jun 2022 10:00:23 +0000

I'm still a bit distrustful of them after what they did to undermine the xf86-video-radeonhd project.

Rumour has it that AMD middle management wanted the FOSS option they were due to announce to be kept slightly inferior to fglrx for Reasons, and having an independent effort that didn't depend on firmware blobs or the decrepit x86-only int10 VBIOS like that did was severely embarrassing them.

Not many people may remember this now, but the R200/(reverse-engineered)R300 driver also used to be blob-free. Strangely that stopped being the case after AMD took over, even though it was feature-complete.

DeVault: GitHub Copilot and open source laundering

bluca — Sun, 26 Jun 2022 09:46:07 +0000

The TDM provisions in the copyright directive apply only for legally accessible corpora. A stolen body of code cannot be data mined legally just because it's available, it needs to be legally available.

DeVault: GitHub Copilot and open source laundering

gspr — Sun, 26 Jun 2022 09:40:48 +0000

> It's not clear to me whether this claim is actually correct. A model is ultimately "just" a big bag of statistical information, and I honestly don't know whether (US) copyright law attaches to such things in the first place, but I'm skeptical (see e.g. Feist v. Rural).

Surely it does apply in one extreme, namely that of a really good model! If I take a copyrighted picture and create a model that very accurately reproduces said picture, I don't think it's very unlikely that my model runs afoul of the original's copyright.

In the other extreme—that of a really terrible model—it probably doesn't, but we probably shouldn't write off the models in-between those extremes.

DeVault: GitHub Copilot and open source laundering

gdt — Sun, 26 Jun 2022 03:22:32 +0000

The deeper point about differing copyright laws is not so much interpretation of copyright licenses, but the distinction between "fair use" and "fair dealing".

Copilot claims its actions are fair use, and therefore the license is irrelevant. However in fair dealing jurisdictions Copilot's use of the program source must either meet the copyright license or one of the black-letter list of allowed uses in the fair dealing exceptions of that juridiction's copyright law.

DeVault: GitHub Copilot and open source laundering

Cyberax — Sun, 26 Jun 2022 00:23:15 +0000

ATI had closed the driver for R300 video cards, even though the driver for R200 was open. The opened up only after being acquired by AMD.

DeVault: GitHub Copilot and open source laundering

salimma — Sun, 26 Jun 2022 00:11:11 +0000

I'm surprised Mercurial is not mentioned. Sourcehut supports it, and GitHub doesn't, so merely by using it you set up some barrier against people accidentally uploading a fork to GH.

It will probably also curtail your network effect even further though.

DeVault: GitHub Copilot and open source laundering

salimma — Sun, 26 Jun 2022 00:07:59 +0000

> The GPL2 didn't make nVidia *or* AMD play nice with Linux

Any specific example for AMD here? They seem to be much better citizens when it comes to the GPL, at least compared to nVidia (and even nVidia is finally open sourcing kernel drivers)

DeVault: GitHub Copilot and open source laundering

Vipketsh — Sat, 25 Jun 2022 21:27:31 +0000

I think rgmoore and NYKevin answered that question pretty convincingly in their replies to my comment above: if you believe that your code exists in something it shouldn't you argue that and courts don't care how it got to where it shouldn't have, only if it is there or not. Perhaps I can put it another way: what the AI model produces doesn't matter, only where that output ends up in.

Even so, I still think that the legal status of the "AI model" warrants further introspection. When talking about open source code the "AI model" isn't much of a consideration because at worst it stores code in some cryptic way that is available in a much more easier digested form. But that changes a whole lot if the training material is some propriety code stolen from somewhere. If your "AI model" is able to reproduce the stolen code verbatim (or sufficient parts for copyright to apply) and training of the "AI model" is "give an exception on copyright rules when doing text and data mining, for any purpose" (bluca's words) that should mean that this trained "AI model" is fully legal and thus a legal distribution mechanism for the stolen code. Surely there is a legal principal to prevent things to work out this way ? At what point does an "AI model" turn into a "distribution mechanism" ?

DeVault: GitHub Copilot and open source laundering

khim — Sat, 25 Jun 2022 17:05:33 +0000

> But that applies only if the broad and narrow ruling turn out the same way.

Where does that idea comes from? Narrow ruling is used precisely to ensure the possibility of broad ruling (made in a different case by a different judge later) to proclaim the opposite outcome!

> If the broad and narrow grounds for the ruling have opposite results, the judge has to go based on which one seems to be a better reading of the law and situation, not just on narrow versus broad.

Narrow is almost always better. Because, well, it's narrow. It describes the situation more precisely. The only save you can have is to proclaim that narrow reading is so narrow it's not applicable to your case at all.

That often happens with patents (judge is presented with half-dozen of patents which can be, theoretically, be treated as prior art and eliminate the patent completely, but 9 times out of 10 judge doesn't do that, but only just proclaim that yes, patent is still valid, just not applicable for your case).

> That's the way these things usually go.

I would say it's the way these things usually don't go. 99% of time decision doesn't reach high enough courts to decide anything definitively. Usually it takes dozens of cases and decades of litigation for that to happen.

DeVault: GitHub Copilot and open source laundering

eduperez — Sat, 25 Jun 2022 14:23:39 +0000

> Training of the model is not done under the terms of any license. It's done under the copyright directive laws, which give an exception on copyright rules when doing text and data mining, for any purpose. I don't see how there's any 'legal mush'.

I do not think anybody is arguing about the model training, but the output from the model once it has been trained.

If the model produces an output that is a verbatim copy of some GPL'd code (see https://twitter.com/mitsuhiko/status/1410886329924194309 for an example), is that code free now, just because it was produced by some AI? Or is it still protected by the GPL, because it is a derivative work? When can we consider that the output has been produced by the AI, and when can we consider it is still a derived work? This is the legal mush.

DeVault: GitHub Copilot and open source laundering

Wol — Sat, 25 Jun 2022 11:47:47 +0000

> Quite. Also, many putative authorities on the GPL seem to forget that there are many legal systems.

And far too many authorities read what they want to see, not what's actually there. I've sure been guilty of that. I think my knowledge of the GPL now is pretty good, precisely because I've had plenty of people call me out on my mistakes.

How many "experts" have NOT been through that learning experience? The majority of them?

Cheers,
Wol

Not trained on MS closed source

bluca — Sat, 25 Jun 2022 09:18:09 +0000

What lawsuits?

DeVault: GitHub Copilot and open source laundering

NYKevin — Sat, 25 Jun 2022 06:00:00 +0000

Clarification: Copyright is path-independent, provided that a path actually exists. In theory, you do have to show that the defendant had access to the copyrighted work, and that this somehow affected the contents of the allegedly infringing work. But in practice, this is a very weak requirement. You don't have to make specific claims about exactly what the defendant did or the path they took from A to B. You just have to show that the path probably exists. OTOH, if the defendant can plausibly argue lack of access (e.g. "Your code was proprietary, and I couldn't read it even if I wanted to"), then that is a valid defense to copyright infringement (provided, of course, that the finder of fact believes this argument).

DeVault: GitHub Copilot and open source laundering

gerdesj — Sat, 25 Jun 2022 01:00:53 +0000

"A lot of engineers have very strange ideas about how the GPL works."

Quite. Also, many putative authorities on the GPL seem to forget that there are many legal systems. If you are going to dive in and be authoritative on he GPL then you really should present an argument that works for all legal systems that the GPL attempts to work within. Quite a job!

Legal is as legal does: Some legal systems have a concept of "reasonable" or what a "reasonable" person would do and I think that is what the GPL is riffing off. There's also the concept of being able to "quietly enjoy [something]". I'm a Brit. so my local legal system informs my knowledge here. Not all legal systems work like that.

I think it is fair to say that we all have strange ideas about how the GPL works. There's no need to call out end users.

DeVault: GitHub Copilot and open source laundering

mathstuf — Fri, 24 Jun 2022 22:52:28 +0000

> What was that about the ?Pennsylvania legislature trying to define "pi = 3"?

Hey, PA has its problems. But this one came from Indiana. Luckily there was a school teacher that was there to help teach some math.

https://www.straightdope.com/21341975/did-a-state-legisla...

Not trained on MS closed source

glenn — Fri, 24 Jun 2022 22:46:49 +0000

Maybe we can take Copilot seriously once Microsoft pledges to indemnify their users from lawsuits related to the code Copilot generates.