Development quote of the week [LWN.net]

To sum up: "by using sixteen copies of our our massive language model, whose training data includes every version of GCC ever released, a warehouse full of GPUs, all the public code in the world and having it autocorrect itself by testing its output against GCC, we managed to make a C compiler that self-reports that it mostly works. This took two weeks and cost $20,000 and, gosh, I have so many feelings."

— Mike Hoye

to post comments

Read the whole thing!

Posted Feb 15, 2026 15:46 UTC (Sun) by mmarsh (subscriber, #17029) [Link] (54 responses)

This excerpt is the tiniest tip of the most delightful iceberg.

Read the whole thing!

Posted Feb 15, 2026 17:01 UTC (Sun) by malmedal (subscriber, #56172) [Link] (53 responses)

I found it rather silly, he's saying it's not more impressive than using cp. But the new compiler is in Rust, and to the best of my knowledge cp is not currently capable of replacing c-code with equivalent rust...

I'd be interested in an analysis of how the architecture of the new compiler compares to other known compilers, does it use RTL like gcc? Or whatever is equivalent in LLVM? tcc perhaps?

Read the whole thing!

Posted Feb 15, 2026 18:30 UTC (Sun) by excors (subscriber, #95769) [Link] (52 responses)

The Anthropic blog post / press release says the system was prompted to "have an SSA IR to enable multiple optimization passes". But it also says "Even with all optimizations enabled, it outputs less efficient code than GCC with all optimizations disabled", so it may have the rough shape of an optimising compiler but it doesn't actually work as one.

The post also says "I tried (hard!) to fix several of the above limitations but wasn’t fully successful. New features and bugfixes frequently broke existing functionality" so this codebase is not the first step towards making an adequate compiler, it's a dead end. (At least with current AI tools; if we give them another hundred billion dollars then maybe the next tools will be good enough to make us all redundant, and if not then just another hundred billion might do it.)

I can't find any reliable benchmarks or analysis of their compiler, though. It looks like https://harshanu.space/en/tech/ccc-vs-gcc/ has probably genuine data showing an AI-generated SQLite benchmark where most queries were ~6x slower than GCC -O0, and one particular query was 158,000x slower, but I don't believe its explanation of that huge slowdown (it says poor register allocation and instruction cache pressure, giving a ~12x slowdown which is inexplicably multiplied over a billion iterations; maybe that explanation was AI-written too). I'd guess the compiler must actually be miscompiling the code and performing the wrong computation, but I can't be bothered to try reproducing it myself.

Read the whole thing!

Posted Feb 15, 2026 20:13 UTC (Sun) by malmedal (subscriber, #56172) [Link] (51 responses)

The way I'm reading the blog, I don't believe the point was to create a c-compiler, rather it was to find out how to use Claude
to create large programs.

In *that* they have succeeded rather impressively. Four years ago AI could maybe complete a line when you pressed tab, now the limit of what it can coherently create appears to be around 100 000 lines.

The blog post will be useful for Anthropic's customers while making their own programs, not so much for people who want a new c-compiler.

Read the whole thing!

Posted Feb 15, 2026 20:44 UTC (Sun) by mb (subscriber, #50428) [Link] (50 responses)

>I don't believe the point was to create a c-compiler, rather it was to find out how to use Claude
>to create large programs.

Yes. But has Claude actually *created* a large program?

I do not think it created a large program.
It made a large program do similar things than an already existing large hand crafted program.
That is a very different thing from creating a large program from scratch.

In real world engineering this is pretty much useless, unless they get GCC out of the loop.
Because it effectively means I have to first develop a program so that the AI can generate a program doing the same thing.
Might be useful in corner cases like taking a proprietary program and making a free version of it. But it doesn't solve most typical engineering problems.

But I'm still impressed by the generated code itself.
These tools are pretty useful these days for certain tasks.
This is really impressive and this is a thing nobody could imagine a couple of years ago.

Read the whole thing!

Posted Feb 15, 2026 21:38 UTC (Sun) by malmedal (subscriber, #56172) [Link] (41 responses)

> In real world engineering this is pretty much useless, unless they get GCC out of the loop.

No. This teaches us where the limits of Claude's abilities currently are. It shows what techniques are needed to reach this limit. Additionally the post also explains what Claude couldn't accomplish.

All this is useful for people who are developing software with Claude in general, if they are stuggling to make progress on a 200 000 line program it is helpful to know it is beyond what Anthropic's employee could do, if they are struggling on a 50 000 line program it is helpful to know that effort spent on making the spec tighter is likely to succeed.

Read the whole thing!

Posted Feb 15, 2026 21:55 UTC (Sun) by mb (subscriber, #50428) [Link] (40 responses)

These real world engineers don't have a fully functional program in their prompt loop.
Therefore this doesn't show the real world limits of Claude.

It's a very interesting academic experiment that is orthogonal to real world engineering.

Read the whole thing!

Posted Feb 15, 2026 22:22 UTC (Sun) by malmedal (subscriber, #56172) [Link] (39 responses)

> These real world engineers don't have a fully functional program in their prompt loop.

The claim is that they got to 99% pass rate before using gcc.

Let me quote the relevant text:

> When there are many distinct failing tests, parallelization is trivial: each agent picks a different failing test to work on. After the test suite reached a 99% pass rate, each agent worked on getting a different small open-source project (e.g., SQlite, Redis, libjpeg, MQuickJS, Lua) to compile.

> But when agents started to compile the Linux kernel, they got stuck. Unlike a test suite with hundreds of independent tests, compiling the Linux kernel is one giant task. Every agent would hit the same bug, fix that bug, and then overwrite each other's changes. Having 16 agents running didn't help because each was stuck solving the same task.

> The fix was to use GCC as an online known-good compiler oracle to compare against. I wrote a new test harness that randomly compiled most of the kernel using GCC, and only the remaining files with Claude's C Compiler.

So according to the author, gcc was used to allow the project to finish in a reasonable time, not to directly help on quality. (Obviously it is quite likely that this actually ended up helping on quality even if that was not the intention)

In any case, if the compiler managed to compile sqlite, redis etc. before getting help from gcc it is already very impressive.

Read the whole thing!

Posted Feb 16, 2026 7:18 UTC (Mon) by mb (subscriber, #50428) [Link] (38 responses)

The test suite means there already is gcc in the loop.

Read the whole thing!

Posted Feb 16, 2026 9:53 UTC (Mon) by malmedal (subscriber, #56172) [Link] (37 responses)

> The test suite means there already is gcc in the loop.

No, the gcc test suite was written to help human programmers develop gcc. Saying it *is* gcc is just silly.

If you want your own c-compiler you can give the AI the gcc test suite.

If you want a compiler for your own language you need to write the tests yourself, this is less work than writing both.

Read the whole thing!

Posted Feb 16, 2026 12:10 UTC (Mon) by excors (subscriber, #95769) [Link] (32 responses)

> If you want your own c-compiler you can give the AI the gcc test suite.

Specifically, if you want your own very inefficient C compiler that mostly passes the provided test suite but has loads of bugs in areas that weren't tested and is also unmaintainable (given it has reached the AI system's limits, and no human understands the design or the code).

(There are many examples of bugs at https://github.com/anthropics/claudes-c-compiler/issues/232 . And it's trivial to find more: my first try was `int main() { return @; }` which CCC happily accepts and compiles into a program that returns an arbitrary value.)

> If you want a compiler for your own language you need to write the tests yourself, this is less work than writing both.

I'm not sure that's obviously true. GCC's test suite has many thousands of tests for specific regressions and for new features where developers anticipated a risk of regressions - you can't write those tests until you've already got a mostly-working compiler and understand where the tricky edge cases are. If all you have is a language specification, you don't know enough yet to build a test suite, and the best way to gain that knowledge is to write the compiler yourself and think about what might go wrong.

(Never mind that you can't even write a good language specification in isolation - that too has to be an iterative process with compiler developers, making tradeoffs between the language's expressivity and the compiler's complexity and performance, informed by a deep understanding of how the compiler works.)

And GCC's test suite is still hugely inadequate, because you need to write test cases for all the obvious things that human developers would naturally implement right but an AI that's overfitting on test pass rate wouldn't. (Presumably there was no test that unknown characters like `@` trigger an error because nobody implementing a lexer would be dumb enough to just skip over unknown characters). If you care about performance then you'll need to somehow test all of that too, else the AI is going to e.g. implement all your SIMD intrinsics with plain C arithmetic (as CCC does) and pretend that that's acceptable. And you need tests for diagnostics and command-line options and debug info (all of which CCC gets wrong) and absolutely everything else, because you can't assume anything will be implemented sensibly unless it's carefully verified from outside the AI system.

Building such a comprehensive test suite from scratch, with just a language specification and no understanding of how the compiler is going to work internally, sounds like an even harder task than writing your own compiler.

Read the whole thing!

Posted Feb 16, 2026 13:17 UTC (Mon) by malmedal (subscriber, #56172) [Link] (31 responses)

> There are many examples of bugs

Obviously, but you seem to be missing my explanation earlier: The point of the project is not to produce a c-compiler, it is to find out what the limits of the specific model "Claude Opus 4.6" are.

In fact let me just give you a direct quote:
> This project was designed as a capability benchmark. I am interested in stress-testing the limits of what LLMs can just barely achieve today in order to help us prepare for what models will reliably achieve in the future.

Please just read the article instead of denigrating it based on hearsay.

Read the whole thing!

Posted Feb 16, 2026 18:22 UTC (Mon) by excors (subscriber, #95769) [Link] (30 responses)

> you seem to be missing my explanation earlier: The point of the project is not to produce a c-compiler, it is to find out what the limits of the specific model "Claude Opus 4.6" are.

My point is that the project did a poor job of finding and reporting any meaningful limits, because they published very little analysis of the code that was generated, and my own basic analysis suggests the code is far more useless than they imply. Their headline claim is that they autonomously created a "fully functional" C compiler from scratch for $20K that can successfully build Linux on multiple platforms, which sounds impressive, like they've independently replicated GCC or Clang (or at least TCC); but that's very misleading.

They later say the process requires "high-quality compiler test suites" (from GCC etc), and while trying to build Linux they had to use GCC to compile the source files they didn't support yet, and their demo still required GCC's assembler and linker, which contradicts the "from scratch" and "clean-room" claims. They may not be directly copying GCC's code, but their development strategy is heavily dependent on GCC.

They provide no analysis of how well their compiler generalises to C code that wasn't part of its test suites (and it appears the answer is, not very well). One of the basic ideas of machine learning is that you need separate data sets for training and evaluation, to help detect overfitting; but they don't do that kind of evaluation here, they just report a list of projects that were used as inputs by the AI system while it was writing the compiler, many of which revealed new compiler bugs. Apart from Linux and Doom, they don't even independently verify the AI system's claim that it now builds all those projects successfully.

They say "the generated code is not very efficient", with no benchmarks to show how inefficient, and no investigation of why. The compiler has lots of optimisation passes - why don't they work? Is it a few minor bugs that could be fixed with just a little more time? Is it using completely the wrong architecture and it'll never work? Or is it too difficult for anyone to figure out what's going on?

They say "the best way to understand what language models can do is to push them to their limits, and then study where they start to break down", but they don't do any of that study here. They have just a few bullet points about where it breaks down and don't dig any deeper, and the rest of the article is promoting their successes and promising it'll be even better in the future. (And then, because he's an AI safety researcher, a brief mention that maybe someone should think about the consequences of this technology but it'll probably be okay.)

I accept they've shown they're able to autonomously produce a large amount of plausible-looking code that mostly passes a given test suite and is otherwise very poor quality; but that's not the claim they made, and it doesn't seem like a useful limit to find. What would be useful is knowing the limit to producing code that is adequately correct and efficient and maintainable and solves a novel problem and doesn't depend on a perfect test suite and so on, and they haven't shown that.

Read the whole thing!

Posted Feb 16, 2026 19:23 UTC (Mon) by malmedal (subscriber, #56172) [Link] (29 responses)

Are we reading the same article? The one I'm reading is:

https://www.anthropic.com/engineering/building-c-compiler

The first sentence of that is:

> I've been experimenting with a new approach to supervising language models that we’re calling "agent teams."

Later he writes:

> The compiler is an interesting artifact on its own, but I focus here on what I learned about designing harnesses for long-running autonomous agent teams

He gives an overview of the progress from earlier models:

> Previous Opus 4 models were barely capable of producing a functional compiler. Opus 4.5 was the first to cross a threshold that allowed it to produce a functional compiler which could pass large test suites, but it was still incapable of compiling any real large projects. My goal with Opus 4.6 was to again test the limits.

He does not exaggerate the utility of the compiler:

> The resulting compiler has nearly reached the limits of Opus’s abilities. I tried (hard!) to fix several of the above limitations but wasn’t fully successful. New features and bugfixes frequently broke existing functionality.

The article is useful for people who are using LLMs to write software, describing an approach that produced a large complicated result.

The article is very much NOT saying "look at my c-compiler", read the article to get an idea of how Claude can be used in large projects, do not read the article expecting a gcc replacement.

Read the whole thing!

Posted Feb 16, 2026 20:34 UTC (Mon) by excors (subscriber, #95769) [Link] (28 responses)

Same article but with a different emphasis. Even before the first sentence, the headline starts with "Building a C compiler". The introduction says it "produced a 100,000-line compiler that can build Linux". The embedded video says "We asked Claude to build a C compiler from scratch. A project that would take a small team months ... We tested it on the Linux kernel. It works. Zero manual coding. A fully functional compiler". I agree the article is mostly discussing Claude, not the compiler itself, but it starts by promoting Claude as having successfully built a real C compiler and doesn't even mention the caveats until much later.

> The article is useful for people who are using LLMs to write software, describing an approach that produced a large complicated result.

But the approach produced a large complicated result that doesn't work. It was told to produce an optimising C compiler, and it didn't - it produced something that can very poorly compile a small set of C programs and fails when straying beyond its test suite. I assume most people who are seriously using LLMs to write software want to write software that works properly and matches their specification, which is why I struggle to see why this experiment is useful except as a warning that this approach failed and that an optimising C compiler is far beyond Claude's capability.

Read the whole thing!

Posted Feb 16, 2026 21:34 UTC (Mon) by malmedal (subscriber, #56172) [Link]

Quite fascinating how different our underlying assumptions are, you seem affronted by the AI using the gcc test suite.

To me that is the impressive bit: Claude can now follow such a long and detailed spec on a 100 000 line block of code, this is an amazing advance, back in the old days of November 24th last year we could only dream of having such a capability.

Look at how he is denigrating old Claude:

He is saying "barely capable of producing a functional compiler", like of course it could do that, it's basically 2025 table-stakes. It wasn't even worth his time to write a blog-post about it.

Read the whole thing!

Posted Feb 16, 2026 22:00 UTC (Mon) by Cyberax (✭ supporter ✭, #52523) [Link] (26 responses)

It's like saying: "Hey, this spaceship that can fly to Pluto and back in a week is trash because it doesn't have heated driver seats and large-size soda cupholders". The fact is, AI agents were able to autonomously create something that would have taken many months for a human to do.

Was it perfect? Hell no. But just 10 years ago I would have confidently put anything like this into the "impossible" bin.

Read the whole thing!

Posted Feb 16, 2026 23:22 UTC (Mon) by excors (subscriber, #95769) [Link] (25 responses)

I dunno, to me it seems more like they made a spaceship that could fly to the Moon but all the crew died. Then they made one that could fly to Mars but all the crew died. Now they've got to Jupiter but all the crew died, and they're promising they'll soon be getting to Pluto and then to the Andromeda galaxy. And they're getting loads of hype and public support and money because they're making ever more sophisticated spaceships that were previously thought impossible, and I can see that they are making those spaceships, but it's strange how few people seem to be bothered that all the crews are still dying.

LLMs used to only generate small programs that were full of bugs, and now they can generate much larger programs that are full of bugs, and I don't understand why people don't care about the bugginess as if that's a trivial detail.

Read the whole thing!

Posted Feb 16, 2026 23:36 UTC (Mon) by malmedal (subscriber, #56172) [Link]

> LLMs used to only generate small programs that were full of bugs, and now they can generate much larger programs that are full of bugs, , and I don't understand why people don't care about the bugginess as if that's a trivial detail.

The LLM that can generate large programs with many bugs can also generate smaller programs with low bug-density. The things fall apart rather spectacularly at the edge, but if you step some ways back they are good.

Read the whole thing!

Posted Feb 16, 2026 23:58 UTC (Mon) by Cyberax (✭ supporter ✭, #52523) [Link] (23 responses)

Well, yes. But the ship got there and back! In one week!

> LLMs used to only generate small programs that were full of bugs, and now they can generate much larger programs that are full of bugs, and I don't understand why people don't care about the bugginess as if that's a trivial detail.

LLMs for programming are two years old at this point. Of course we're at the stage where everything experiences rapid unscheduled explosions all the time because we don't yet have good practices or experience for large-scale LLM use. So we're getting hilarious failures, and we'll probably get a fair share of outright tragic disasters.

But this is what happens with _every_ significant new technology, starting from trains and steam engines. A correct response here is to learn it, adapt to it, and start utilizing it. Or maybe find a niche where you can avoid it.

Read the whole thing!

Posted Feb 17, 2026 8:57 UTC (Tue) by taladar (subscriber, #68407) [Link] (22 responses)

The difference is that LLMs very much haven't improved in about two years of the 4 years they have been around. If anything they have been getting worse in the last few months despite more and more effort being devoted to them.

Read the whole thing!

Posted Feb 17, 2026 20:04 UTC (Tue) by Cyberax (✭ supporter ✭, #52523) [Link] (21 responses)

Whut? This is demonstrably not true. LLMs improved by several orders of magnitude on LLM benchmarks. Coding agents moved from barely completing Leetcode problems to writing a full C compiler (with caveats, but still).

We might not get full AGI, but even the current state of LLM development already means that the software development is going to be transformed fundamentally.

We now have a completely new building block that can reasonably perform the "do what I mean" operation. That can be integrated into different workflows.

Read the whole thing!

Posted Feb 17, 2026 22:11 UTC (Tue) by Wol (subscriber, #4433) [Link] (1 responses)

> Whut? This is demonstrably not true. LLMs improved by several orders of magnitude on LLM benchmarks. Coding agents moved from barely completing Leetcode problems to writing a full C compiler (with caveats, but still).

How much effort do you need to write the prompts though? Is it worth it?

Cheers,
Wol

Read the whole thing!

Posted Feb 18, 2026 9:53 UTC (Wed) by kleptog (subscriber, #1183) [Link]

> How much effort do you need to write the prompts though? Is it worth it?

You only need to write the prompts once though? I have no idea what prompts the GitHub Copilot coding assistant in VSCode uses, and I don't need to care either.

An LLM is a very low-level tool. It will get wrapped up in more and more layers of abstraction to hide more an more details. You're not going write prompts, you're going to solve problems. Which will involve some typing sure, but it won't be prompts.

In any case, LLMs still require more evolution. Separation of commands & data needs to happen at some point.

BTW, LLMs can assist you in writing prompts.

Read the whole thing!

Posted Feb 18, 2026 11:09 UTC (Wed) by paulj (subscriber, #341) [Link] (17 responses)

> writing a full C compiler

No it did not. Come on.

Probing the space of functions to construct an f(x), when you are given a really good oracle for f(x) PLUS your probing algorithm contains *all the features* of f(x) encoded into it can not be considered equivalent to the task of "writing f(x)".

Normally in programming, "writing f(x)" entails discovering the problem space to begin with, recursively constructing further meta-functions to apply suitable structure to the input domain AND output domain (i.e., you must actually /define/ what x is and /should be/, and you must define what is sensible to output). Recreating an f(x) under an existing high-quality oracle for f(x) does not need an "AI".

Read the whole thing!

Posted Feb 18, 2026 12:02 UTC (Wed) by malmedal (subscriber, #56172) [Link] (15 responses)

> Probing the space of functions to construct an f(x),

Not a good description of what's going on, but more importantly you're missing what's interesting about this. From the blog post:

Several earlier models also could produce a functional compiler, but that was not worth posting about at the time.

Unlike your implication that it's some kind of cheat to follow a large spec this is actually the impressive thing.

All LLMs lose coherence and just produce a garbled mess when the task is too big.

This is a demonstration that latest Claude has pushed the limits to produce a coherent 100 000 lines block of code. More than anyone else has been able to show.

Read the whole thing!

Posted Feb 18, 2026 12:13 UTC (Wed) by paulj (subscriber, #341) [Link] (14 responses)

That's great, but it still did not write a compiler.

Read the whole thing!

Posted Feb 18, 2026 12:33 UTC (Wed) by paulj (subscriber, #341) [Link] (11 responses)

Oh, and I think it's likely a spectacularly inefficient way to use an oracle of f(x) to reconstruct f(x). I strongly suspect there are likely much more energy efficient algorithms to do this - particularly once you properly account for the energy that was used to train Claude, and encode f(x) into it (which is NOT included in the reported $20k + 2 weeks).

Read the whole thing!

Posted Feb 18, 2026 12:33 UTC (Wed) by paulj (subscriber, #341) [Link] (10 responses)

Sorry, reconstruct a _poor_ approximation of f(x).

Read the whole thing!

Posted Feb 18, 2026 13:05 UTC (Wed) by malmedal (subscriber, #56172) [Link] (9 responses)

> reconstruct

You keep obsessing about this word, but if you have a novel idea the AI can construct a new thing for you. It is not limited to making copies of existing functionality.

Read the whole thing!

Posted Feb 18, 2026 14:04 UTC (Wed) by paulj (subscriber, #341) [Link] (8 responses)

> if you have a novel idea the AI can construct a new thing for you.

I.e., you use the AI as a tool to extract useful features, to apply transforms of your idea from how you can express it into another domain (the AI maps features it extracts from your input onto an assembly of features in its data-set).

You're not disagreeing with me, there.

> It is not limited to making copies of existing functionality.

I'm curious how well you know how AI works? Also, I'd ask you to reconcile this point with your previous one. Is the AI itself coming up with new things or not?

Read the whole thing!

Posted Feb 18, 2026 14:22 UTC (Wed) by paulj (subscriber, #341) [Link]

Oh, and where it comes up with something new, what aspect of the "AI" is there to check this "something new" makes sense - rather than just the stochastic "pasting together" of features, at a rate governed by some 'heat' variable?

Read the whole thing!

Posted Feb 18, 2026 15:48 UTC (Wed) by malmedal (subscriber, #56172) [Link] (6 responses)

> I'm curious how well you know how AI works?

I have implemented a toy one myself, yes. This is why I am so impressed by all the complexity ultimately coming from dot product -> softmax -> another matrix multiplication.

I do suspect you don't though, your statements about "reconstruct f(x)" make absolutely no sense, calling inference some kind of optimization or reconstruction simply is not correct.

> Also, I'd ask you to reconcile this point with your previous one. Is the AI itself coming up with new things or not?

At training time the AI is learning high-level abstract concepts from the training data, at inference time it is applying these abstract concepts to produce output.

The AI may well have learned some abstract concept that no human has, which I would count as novel. It may also apply two or more abstract concepts together in a way that no human has done, quite a few Nobel prizes have been given to the first guy to apply an existing abstract concept to a new problem.

What the AI is *not* doing is "only regurgitating training data", if you don't believe me and also have a some millions of dollars to spend you should soon be able to take the training corpus of one of the open models, exclude all c-compilers and train your own model, which will be able to produce a c-compiler given the spec.

Read the whole thing!

Posted Feb 18, 2026 16:21 UTC (Wed) by paulj (subscriber, #341) [Link] (5 responses)

> This is why I am so impressed by all the complexity ultimately coming from dot product -> softmax -> another matrix multiplication.

I agree, it is impressive. But it's also no more than that.

> "reconstruct f(x)" make absolutely no sense, calling inference some kind of optimization or reconstruction simply is not correct.

The overall problem, in this case, was to reconstruct f(x). I wasn't saying LLM inference (or whatever compounded construction of LLMs is involved) is some kind of "reconstruct f(x)" algorithm (that is indeed meaningless) - the problem the "AI" was given was "reconstruct f(x), using these oracles".

> At training time the AI is learning high-level abstract concepts from the training data,

For a value of learning => building and whittling away at recursive feature vectors that are collective properties of the training data. It's not "learning" in an "intelligent" way, where it can incrementally train itself as it produces output. The feedback loop is still quite static, and there's still a big, hard split between the trained weights and the context window (in all the LLM-based AI stuff I've looked at anyway).

> What the AI is *not* doing is "only regurgitating training data",

I didn't say "regurgitating training data", I wrote pasting together features (I should add, a recursive series of those, I did write "recursive" wrt "features" a few times before).

> if you don't believe me and also have a some millions of dollars to spend you should soon be able to take the training corpus of one of the open models, exclude all c-compilers and train your own model, which will be able to produce a c-compiler given the spec.

I assume you're still including compilers of many other languages, and also C code bases that include functional unit tests in C though.

It is impressive, but it's not really doing anything itself. I'm sure it's useful, certainly to some. I don't really find it useful myself - except as a better search engine to get pointers to libraries/APIs and syntactic snippets to use as an /example/, for me to then work from. Anytime I've tried in the best to use AI for anything beyond that, it just ends up being wrong (sometimes subtly) and wastes more of my time than it saves (never mind the energy). By the accounts of others here, the "C compiler" that was produced very much fits in that mould.

Also, I am pretty sure current code-gen "AI" use is massively under-priced, and the prices are going to sky rocket. It is not wise to become dependent on it, before one knows the true cost.

Read the whole thing!

Posted Feb 18, 2026 16:23 UTC (Wed) by paulj (subscriber, #341) [Link] (1 responses)

Oh, as a test, if an "AI" can create a new language, a new language with features that solve problems that other languages do not, that would show it has learned something and has some ability to create new things.

Read the whole thing!

Posted Feb 18, 2026 18:10 UTC (Wed) by Cyberax (✭ supporter ✭, #52523) [Link]

Here you go: https://github.com/AvitalTamir/sever

Read the whole thing!

Posted Feb 18, 2026 18:53 UTC (Wed) by malmedal (subscriber, #56172) [Link] (2 responses)

> I wrote pasting together features (I should add, a recursive series of those, I did write "recursive" wrt "features" a few times before).

You're doing some annoying semantic contortions here, "feature", and "concept" have reasonably clear meanings in English, "feature" clearly does not fit, so instead of "concept" you invent the term "recursive feature" instead. Normal definition of "feature" in machine learning is rather a group of neurons working to together to activate based on something. This something can be e.g. a letter in OCR or it can be a concept in an LLM.

The refusal to use "concept" is extra weird given the history of the attention mechanism and why it was developed. The original use-case was for machine-translation. To do a reasonable translation from one language to another you do need to encode the concepts expressed in the source text and then render the same concepts in the destination language. The attention mechanism does this very well, it was a major advance in state of the art when it arrived.

> I assume you're still including compilers of many other languages, and also C code bases that include functional unit tests in C though.

Don't think that's necessary. People are experimenting with new languages which the AI has not seen, but which makes it convenient to express the concepts it has learned.

> prices are going to sky rocket.

maybe, maybe not. Small model quality has also improved a lot, so quite a few individuals can afford to buy hardware good enough to run this thing:

https://huggingface.co/Qwen/Qwen3-Coder-Next

Compared to latest Claude it is quite poor of course, but it compares favorably to the models Karpathy was using when he coined the term "vibe-coding".

There are many providers that are selling inference based on open-weights models these appear to be profitable.

Read the whole thing!

Posted Feb 19, 2026 13:27 UTC (Thu) by taladar (subscriber, #68407) [Link] (1 responses)

To me it sounds like you are arguing on two entirely different levels of abstraction. While you are insisting on bringing implementation details of LLMs into the discussion the person you are talking to looks at it as a black box that has a certain output and is talking about "features" of the output, not some sort of low-level transformer implementation detail.

Read the whole thing!

Posted Feb 19, 2026 14:20 UTC (Thu) by malmedal (subscriber, #56172) [Link]

My preferred term is "concept" as it is normally used in English. Paul insists strongly it's not "concept", but "feature" with "recursive" in front. His use of the term is not standard usage in English in general, and not correct in the specific way it is used when discussing neural network implentations.

Read the whole thing!

Posted Feb 18, 2026 12:49 UTC (Wed) by malmedal (subscriber, #56172) [Link] (1 responses)

> but it still did not write a compiler.

It's clearly not made a byte by byte copy of anything, so it must be working at some level of abstraction.

To answer whether it did or not we need to know what this abstraction-level is for this specific model and whether it's high enough to qualify as original work. Unfortunately this is not available for Claude, but there are a few fully open models so if you care about the accuracy of your statement you can study those and extrapolate.

Read the whole thing!

Posted Feb 18, 2026 14:16 UTC (Wed) by paulj (subscriber, #341) [Link]

> It's clearly not made a byte by byte copy of anything, so it must be working at some level of abstraction

Yes, it's an impressive level of abstraction for a tool. It's extremely impressive that - with the aid of absolutely massive levels of computation, and gargantuan amounts of human-curated training data + a fair amount of post-hoc human tweaking and guidance on training - we can get banks of computers to normalise out text and code, and the labels found with them ("Here's a {linked-list, B-tree, HTML parser, ....} in {C, Python, Rust, ...}" and extract out numerous features as these sophisticated recursive-meta-vectors, encoding not just their details but even equivalences between text/code, at a variety of levels.

It is amazing. And it's useful for some stuff.

But the hype and AI-fanboyism far, far, far exceeds what these devices are and what they are capable of.

Read the whole thing!

Posted Feb 18, 2026 13:33 UTC (Wed) by excors (subscriber, #95769) [Link]

Also in this case the domain of f(x) is not all conforming C programs, the domain is the test suites of the C programs that were in the feedback loop while developing the compiler. Loads of features that weren't covered by those test suites are broken: there weren't tests for syntax errors (so it mostly ignores them) or undefined macros (it hardcodes many from std headers), performance (the optimiser doesn't work), correctness of _Atomic (it's often non-atomic), command line options (many have no effect, `ccc-x86 -mlittle-endian` switches to Aarch64 mode because the tests only used that option with that target), etc.

This isn't just the minor bugs you'd expect in any new project, it's ignoring the C standard and not bothering to properly implement anything its limited test suite lets it get away with, which isn't how you make a usable compiler. (I've hand-written my own compiler for one specific target program, not aiming for full support of the language, and that certainly makes the task a lot easier. And at least I made it detect and fail on unsupported constructs, not just silently miscompile.)

Their compiler does kinda work in some cases, so it's not nothing, but I think it's far from a "full C compiler".

Read the whole thing!

Posted Feb 18, 2026 12:15 UTC (Wed) by taladar (subscriber, #68407) [Link]

Going from some number like 69% to 71% when investing 10000 times more effort is not "an improvement by an order of magnitude".

Anything coding agents produce is still buggy, unmaintainable garbage just like it was a few years ago.

Read the whole thing!

Posted Feb 16, 2026 17:29 UTC (Mon) by mb (subscriber, #50428) [Link] (3 responses)

>Saying it *is* gcc is just silly.

Certainly the gcc test suite *is* (part of) gcc. Obviously.
It's a very important part of gcc.
Test suites of big programs typically contain a huge collection of many test cases for mistakes done by humans during the decades of development. This makes them extremely valuable. The gcc test suite is probably worth dozens of millions of dollars of developer time.

If a machine model is able to generate a new working program from an existing test suite that tests an existing program it shows me one thing:

gcc's test suite is extremely well developed and it covers many many edge cases of a real world C compiler.
This work was done by humans. Without it ccc would not be possible.

If you build a machine model loop which iteratively improves the output by applying the gcc test suite as a cost function, then this human created test suite is an essential part of the process.

>If you want a compiler for your own language you need to write the tests yourself, this is less work than writing both.

In the real world tests and program code come hand in hand.
I have never seen pure test driven development to actually work in practice for complex programs.
I really do not believe it's possible to develop a C compiler test suite from scratch without having a C compiler to test the test suite.

This is all really impressive and a really interesting milestone for machine learning and maybe for technology as a whole.
But this still is "only" a really impressive replication of pre-existing human work. It didn't produce a new product.

Read the whole thing!

Posted Feb 16, 2026 18:51 UTC (Mon) by malmedal (subscriber, #56172) [Link] (2 responses)

> But this still is "only" a really impressive replication of pre-existing human work. It didn't produce a new product.

Difficulty is other way around, this is *more* impressive than making a new product. If you bother to read the original article from Anthropic, they actually tried this project with earlier models. Earlier attempts were able to produce a compiler, but not one that could compile large existing programs.

Quite a few C-compilers have been written by a random single person working alone. I am not aware of a single person who has written a compiler that can can compile a Linux kernel and have it actually boot.

A greenfield compiler like tcc, sdcc etc. can just say "whatever, this is the way this dialect works", which makes things enormously easier.

Read the whole thing!

Posted Feb 16, 2026 21:00 UTC (Mon) by dskoll (subscriber, #1630) [Link] (1 responses)

I am not aware of a single person who has written a compiler that can can compile a Linux kernel and have it actually boot.

Fabrice Bellard did that. He even made a bootloader that compiles the Linux kernel from source and then boots it.

This was back in 2004. I don't know if his tiny C compiler can compile a modern Linux kernel. Probably not, would be my guess.

Read the whole thing!

Posted Feb 16, 2026 21:46 UTC (Mon) by malmedal (subscriber, #56172) [Link]

Had completely forgotten, I've even used it myself.

Read the whole thing!

Posted Feb 16, 2026 7:28 UTC (Mon) by zdzichu (subscriber, #17118) [Link] (7 responses)

99.9% of programming is creating something similar to already existing code. This is where Claude excels. There remaining 0.1% of actual thinking still belongs to (senior) developers.

Read the whole thing!

Posted Feb 16, 2026 10:23 UTC (Mon) by taladar (subscriber, #68407) [Link] (6 responses)

Similar yes, not identical though in this way. In practice Claude (or any other LLM) does not excel at anything other than wasting time and money.

Read the whole thing!

Posted Feb 16, 2026 20:21 UTC (Mon) by Cyberax (✭ supporter ✭, #52523) [Link] (5 responses)

It's really fun looking at people saying: "In practice, cars just waste everyone's time, they can't even go up a dry stream, like my trusty horse can! Cars will never amount to anything"

Read the whole thing!

Posted Feb 17, 2026 9:43 UTC (Tue) by taladar (subscriber, #68407) [Link] (4 responses)

It is really interesting how all the LLM proponents are trying to associate their failing technology with something that has been successful, completely omitting that for any successful technology there are probably dozens or hundreds of technologies that looked promising at first glance but never amounted to anything in the long term.

Read the whole thing!

Posted Feb 17, 2026 20:06 UTC (Tue) by Cyberax (✭ supporter ✭, #52523) [Link] (3 responses)

Me? I'm not an LLM proponent. I barely use agents, and my AI use is mostly for "boring" parts of coding like creating tests.

> completely omitting that for any successful technology there are probably dozens or hundreds of technologies that looked promising at first glance but never amounted to anything in the long term.

Can you name something that had a similar impact in software?

Read the whole thing!

Posted Feb 18, 2026 12:20 UTC (Wed) by taladar (subscriber, #68407) [Link] (2 responses)

OOP, design patterns, dynamic typing, IDE generated boilerplate,...

There are literally dozens of technologies in software development that looked like a productivity boost (or as if they would allow lower skilled workers to perform with the same productivity) that turned out to be a productivity reduction in hind sight when looking at it from 10 or 20 years after they were introduced.

And the "impact" of LLMs on software development is incredibly over-stated, unless you mean the reduction in productivity from having to deal with AI bug reports or pull requests.

Read the whole thing!

Posted Feb 18, 2026 18:13 UTC (Wed) by Cyberax (✭ supporter ✭, #52523) [Link] (1 responses)

Sorry, nowhere close to the current level of impact. If anything, I would put the proliferation of open source code in late 2000-s and Github.

Read the whole thing!

Posted Feb 19, 2026 13:31 UTC (Thu) by taladar (subscriber, #68407) [Link]

Where is the positive impact that AI has had on anything at all right now?

I have yet to see any.

It had huge amounts of negative impact on many things, particularly anything spam/slop related and the upcoming financial bubble will likely be bigger than anything in our society since software was first invented, true. But it is hard to argue that it has even done one major thing that could be considered positive.

Reinventing SCP, extremely poorly and inefficiently

Posted Feb 16, 2026 17:52 UTC (Mon) by paulj (subscriber, #341) [Link]

So 2 weeks and $20k to invent an *absolutely terrible* way of transferring something across the internet, by wasting loads of energy and time in roughly extracting the source training material from an LLM, instead of just copying it directly.

What a shite future we are marching into.