parallel linking still wanted

Posted Feb 4, 2025 19:13 UTC (Tue) by khim (subscriber, #9252)
In reply to: parallel linking still wanted by iabervon
Parent article: GNU Binutils 2.44 Released

> It wouldn't have to be backwards-incompatible or a change to the language.

It would, most definitely, be both.

> Currently, if you just give a bunch of C/C++ files to the compiler at once, it compiles each of them and links all of them into a single file, which is what you'd want.

Sure, but that usecase sidesteps the critical question that cripples the whole thing: where are how intermediate files should live?

If you specify bunch of C/C++ files then the answer to that question is “nowhere”. Thus wouldn't work for incremental compilation and compiling everything from scratch, all the time… is very inefficient. Worse than what we have today.

> if you used a new command line option to give it a persistent storage location to be a cache

Yes, but then it stops being a “C/C++ language” and turns into “GCC language”, “clang language”, “MSVC language”.

Turbo Pascal and (later) Java solved that problem by fiat: all the compiled files go in the directory specified by this compiler option and names are the same as names of source files. Bam. Done. People may like it, people may hate it… but they have to accept it.

C++… they spent decade or so deciding what to do about that… and in the end excluded that from the standard.

And that made “standard C/C++ modules” completely useless. They couldn't be used without reading extra documentation from the compiler… and, of course, compilers couldn't agree on what they want (or else there would have been easy to include that into the standard)… nothing works. Still, even five years standard was approved.

> there's no reason that handling this usage effectively would interfere with existing usage around compiling single files and linking as a separate command

I'll believe that when I'll see that. So far we have two classes of languages:

Languages that support compilation of single files and linking as a separate command… and then compiling many files efficiently becomes more-or-less impossible and unfeasible.
Language that can compile many files simultaneously using many cores (starting from Turbo Pascal, but also including Java, Haskell, etc) – but then they need a lot of kludges to work with “compile single files and link them as separate command”, even if such usecase is possible at all (it's not possible to compile some programs written in Turbo Pascal or Java if you attempt to compile these files separately).

C++ promises to do both… we'll see if it would manage to pull that off or if it would become a legacy-nobody-cares-about before that'll happen.

parallel linking still wanted

Posted Feb 4, 2025 21:19 UTC (Tue) by mathstuf (subscriber, #69389) [Link]

> C++… they spent decade or so deciding what to do about that… and in the end excluded that from the standard.

Yeah, the ISO C++ standard is allergic to saying things like "source code lives in files" (or that code lives in libraries for that matter: the standard only speaks of programs), so it blocks itself from specifying such things.

> And that made “standard C/C++ modules” completely useless. They couldn't be used without reading extra documentation from the compiler… and, of course, compilers couldn't agree on what they want (or else there would have been easy to include that into the standard)… nothing works. Still, even five years standard was approved.

Eh, it certainly made it hard for build systems, but that's been mostly solved now. The next step is to use the support the compilers now provide to be able to compile modules correctly to flush out bugs in the compiler module implementations and to gather numbers on actual module usage to discover things like whether "large" or "small" modules are "better".

But if you're using something like autotools (which is unlikely to ever support them), Bazel (which has an open PR), or Meson (which I'm sure will…someday), yes, modules are "nowhere" for all practical purposes.