Mold (linker) 1.0 released [LWN.net]

Mold (linker) 1.0 released

Posted Dec 15, 2021 18:16 UTC (Wed) by donbarry (guest, #10485) [Link] (10 responses)

AGPL v3 license.. An .... interesting choice.

The Github LICENSE page explains why -- it's not out of a love for free software:

"I'm looking for a sponsor who wants to purchase the copyright of this work and relicense it under a more liberal license such as the MIT license. For now, mold is released under the GNU AGPL v3."

Mold (linker) 1.0 released

Posted Dec 15, 2021 18:40 UTC (Wed) by yodermk (subscriber, #3803) [Link] (7 responses)

I don't see why that would be a problem in practice. It's not like anyone is interacting with a linker over a network. So it's basically like GPL. And I don't see any problem with an executable program being GPL. Kind of restrictive for a library though.

Mold (linker) 1.0 released

Posted Dec 15, 2021 19:20 UTC (Wed) by donbarry (guest, #10485) [Link] (4 responses)

That's true, and in that sense it is a real contribution to the community. But what rankles is using the AGPLv3 as something of perceived poison pill, even though it really isn't in this case, with the goal being a payout -- cough -- I mean a permissive license.

Not exactly in line with the whole ethic of the license itself.

Mold (linker) 1.0 released

Posted Dec 15, 2021 19:32 UTC (Wed) by mathstuf (subscriber, #69389) [Link] (1 responses)

I guess I see it as analogous to freedom zero. The FSF doesn't have a say in *how* the AGPLv3 gets used just like a developer can't really say that $the_illuminati (or whatever boogeyman might be at hand) can't use $my_project for $evil. Sure, you can name and shame, but putting such terms into the license itself isn't really productive in the long run. At least here in this case, we (humanity) get a fast, FOSS linker that pushes boundaries in the meantime.

Mold (linker) 1.0 released

Posted Dec 15, 2021 20:58 UTC (Wed) by atai (subscriber, #10977) [Link]

Then FSF did say license cannot restrict use, in regard to some "hacker activism" license proposed to say doing evil not allowed

Mold (linker) 1.0 released

Posted Dec 15, 2021 19:40 UTC (Wed) by NYKevin (subscriber, #129325) [Link]

That's capitalism. If we all lived in a magical communist utopia, then these licenses would not need to exist in the first place. If the FSF did not want people to use these licenses in this fashion,* then I think they should have more carefully considered the incentives that were likely to result from their existence.

* I am rather skeptical that this is how the FSF feels, however, because RMS endorsed the practice of selling exceptions to the GPL all the way back in 2010: https://www.fsf.org/blogs/rms/selling-exceptions. So it may simply be that they do not agree with you, and would see nothing wrong with this practice anyway.

Mold (linker) 1.0 released

Posted Dec 15, 2021 20:16 UTC (Wed) by rvolgers (guest, #63218) [Link]

There are plenty of companies which use GPL for their open source release and sell alternative license options for money.

If anything, this is more public-spirited than that approach.

People like the GPL because it enforces giving more rights to users. But that also means people and companies may not like the GPL because it leaves fewer exclusive rights for them to sell for money.

There is nothing wrong with recognizing there are potential downsides to the GPL and that companies are willing to pay to avoid them.

Mold (linker) 1.0 released

Posted Dec 15, 2021 21:03 UTC (Wed) by atai (subscriber, #10977) [Link]

Putting a non cloud related program under AGPL may make sense, for example mupdf. Because it is not clear if someone else may use a non cloud intended program on the cloud or over the network someday

Mold (linker) 1.0 released

Posted Dec 21, 2021 17:35 UTC (Tue) by fw (subscriber, #26023) [Link]

It's not like anyone is interacting with a linker over a network.

Anyone using some sort of build daemon to build C/C++ software is doing that. But as usual with the AGPL, it's a very complicated question whether that triggers any of the additional requirements over the GPL.

Mold (linker) 1.0 released

Posted Dec 16, 2021 22:44 UTC (Thu) by jengelh (guest, #33263) [Link] (1 responses)

>“I'm looking for a sponsor who wants to purchase the copyright of this work and relicense it under a more liberal license”

Until someone turns the game around and buys the copyright (just) to keep it under AGPL.

Mold (linker) 1.0 released

Posted Dec 17, 2021 7:32 UTC (Fri) by pabs (subscriber, #43278) [Link]

Apparently the cost is on the order of millions, I don't think the copyleft loving community could/would cough up that much just for a faster linker, for eg the Software Freedom Conservancy's fundraiser is only expected to make a few hundred thousand USD.

Mold (linker) 1.0 released

Posted Dec 15, 2021 20:20 UTC (Wed) by cyperpunks (subscriber, #39406) [Link] (1 responses)

Why not work on LLD instead? It goes in circles around gold and ld already:
https://lld.llvm.org/

Mold (linker) 1.0 released

Posted Dec 15, 2021 20:34 UTC (Wed) by jhoblitt (subscriber, #77733) [Link]

The primary author worked on lld and claims that mold is significantly faster. https://github.com/rui314/mold#mold-a-modern-linker

I have never used mold and can't attest to its performance.

Mold (linker) 1.0 released

Posted Dec 15, 2021 21:10 UTC (Wed) by syrjala (subscriber, #47399) [Link] (3 responses)

So after gold is left rotting for long enough we end up with mold?

Mold (linker) 1.0 released

Posted Dec 15, 2021 22:03 UTC (Wed) by atai (subscriber, #10977) [Link]

can gold be revitalized?

Mold (linker) 1.0 released

Posted Dec 16, 2021 6:57 UTC (Thu) by LtWorf (subscriber, #124958) [Link] (1 responses)

I remember gold, it was supposed to be the new standard and replace the gnu one… what happened to it?

Mold (linker) 1.0 released

Posted Dec 21, 2021 0:21 UTC (Tue) by flussence (guest, #85566) [Link]

gold had significantly higher memory usage than GNU ld, didn't run all that much faster in the places I used it hoping for a speedup, and was sometimes buggy in ways that would manifest as its output binaries malfunctioning late into runtime which was a nightmare to track down.

And above all, we're talking about it in past tense because it's another abandonware Google product. Not a good look for an essential toolchain component.

Mold (linker) 1.0 released

Posted Dec 15, 2021 21:11 UTC (Wed) by ballombe (subscriber, #9523) [Link] (6 responses)

Does it speed up LTO ?
Now that would be something!

Mold (linker) 1.0 released

Posted Dec 15, 2021 21:59 UTC (Wed) by Vipketsh (guest, #134480) [Link] (5 responses)

LTO is predominantly a compiler thing and is thus limited by the speed of the compiler, not the linker.

Mold (linker) 1.0 released

Posted Dec 16, 2021 14:08 UTC (Thu) by ldearquer (guest, #137451) [Link] (4 responses)

Can you explain that to ignorants like me? Isn't LTO meant to be Link Time Optimization?

Mold (linker) 1.0 released

Posted Dec 16, 2021 14:23 UTC (Thu) by Vipketsh (guest, #134480) [Link] (3 responses)

In LTO, at some point during the compile, an internal representation of the program is written to the output object instead of machine code. During linking, the linker looks at all the objects and passes the compiler's internal representation back to the compiler which then generates the real object data that is returned to the linker to write into the output.

In the BFD linker this magic occurs with a plugin to the linker to find the LTO data, call the compiler and assemble the results.

Mold (linker) 1.0 released

Posted Dec 16, 2021 19:44 UTC (Thu) by JoeBuck (subscriber, #2330) [Link]

Right, but with a structure like mold's, the recompilation steps could be run in parallel. But yes, the recompiles could still be the bottleneck.

Mold (linker) 1.0 released

Posted Dec 17, 2021 10:11 UTC (Fri) by ldearquer (guest, #137451) [Link] (1 responses)

Thanks for the explanation.
Therefore if you don't pass the lto flag to the compiler stage, you don't get real LTO? Because it would write normal output object.
I need to go and read more about this, cause so far I always assumed that you just compile all .o files normally and then simply pass the -flto flag to the linker (GCC) :(

Mold (linker) 1.0 released

Posted Dec 17, 2021 11:26 UTC (Fri) by Vipketsh (guest, #134480) [Link]

That's correct: you need to pass the lto flag when compiling your source. You are well advised to pass all your other compilation flags used for source compilation when doing the LTO link too (-f*, -m* & possibly others).

The gcc manual has a decent explanation to start from: https://gcc.gnu.org/onlinedocs/gcc-11.2.0/gcc/Optimize-Op...

Mold (linker) 1.0 released

Posted Dec 15, 2021 21:18 UTC (Wed) by Karellen (subscriber, #67644) [Link] (3 responses)

From the README.md:

It is sometimes very hard to pass an appropriate command line option to cc to specify an alternative linker. [...]

For some reason, I thought that setting the "LD" environment variable would work for most GNU build tools. But checking the GNU make Implicit Variables documentation, although there exists CC & CCFLAGS, and CXX & CXXFLAGS, and similar (but not always exact) snowclones of tool & toolFLAGS variables for AR, AS, CPP, FC, CO, LEX, YACC, PC and LINT, and there is an LDFLAGS, there is no LD equivalent!

What a bizarre inconsistency.

Mold (linker) 1.0 released

Posted Dec 15, 2021 21:31 UTC (Wed) by Bigos (subscriber, #96807) [Link] (1 responses)

The problem is, you rarely execute the linker directly. Most often you do it using the compiler frontend, such as gcc, g++, clang, etc. Hence most often you link using CC or CXX (at least in C/C++ land).

And these frontends had to learn alternate linkers on their own using command line parameters. This is how -fuse-ld=$LINKER came to be. Unfortunately, for whatever reason (various linkers not being completely compatible?), the $LINKER parameter, at least for gcc, is restricted to a list of linkers, mold not being one of them. Also note that $LINKER is not the executable name, it is ld.$LINKER instead (like ld.gold).

https://gcc.gnu.org/onlinedocs/gcc-11.2.0/gcc/Link-Option...

Mold (linker) 1.0 released

Posted Dec 16, 2021 14:44 UTC (Thu) by jrtc27 (subscriber, #107748) [Link]

And it took an eternity for GCC to add LLD to the list of linkers. Meanwhile Clang just lets you put whatever you like, and also lets you give an absolute path to the linker rather than letting it use fragile magic to find the linker (which all too often leads to GCC finding the wrong one when cross-compiling, or doesn't find one at all because it has very rigid views on where linkers can exist).

Mold (linker) 1.0 released

Posted Dec 15, 2021 22:12 UTC (Wed) by madscientist (subscriber, #16861) [Link]

The contents of the manual are the MINIMUM set of variables that are assigned. The docs make this pretty clear where it says: Here is a table of some of the more common variables. If you want to know the full set of default variables you can ask make to show them to you:

$ make -pf/dev/null | grep ^LD
make: *** No targets.  Stop.
LD = ld

As already mentioned, though, it's very rare that anyone actually uses ld directly. These days, unless you're trying to create a bare executable, you pretty much have to use the compiler's front end because it will add important linker options and internal (but critically important) libraries. Indeed, most of make's built-in rules don't use $(LD) they use $(CC) (or the equivalent for other languages).

Mold (linker) 1.0 released

Posted Dec 15, 2021 21:57 UTC (Wed) by Vipketsh (guest, #134480) [Link] (5 responses)

I didn't get why a new linker built for speed was so important, but after looking at the README here, I sort of understand:
Program (linker output size)
Chrome 96 (1.89 GiB)
Clang 13 (3.18 GiB)
Firefox 89 libxul (1.64 GiB)

So, programs these days weigh in at multiple gigabytes. Anyone know what the breakdown of those gigabytes are ? Surely it is not all machine code...

Mold (linker) 1.0 released

Posted Dec 15, 2021 22:50 UTC (Wed) by mathstuf (subscriber, #69389) [Link] (1 responses)

With templates and, more generally, monomorphization that happens, libraries can be *quite* large. Without optimization, some projects I work on have multi-hundred MB *object* files, nevermind libraries. Given how much just raw code that gets dumped out these days from DOM specs and other API IDL files for modern browsers…I'm not really all that surprised here. Optimization helps *a lot* by eliding lots of symbol names and inlining only-visible-inside-the-TU instantiations, but that's not what developers are necessarily working on all the time.

Mold (linker) 1.0 released

Posted Dec 17, 2021 22:15 UTC (Fri) by bartoc (guest, #124262) [Link]

yeah, although template spew thats actually duplicate is eliminated by the linker when you actually assemble the executable.

Still, its easy (at least in c++) to write templates that spew out a ton of nearly identical code where a C programmer would have just used some form of virtual dispatch. You can get template spew without even changing the types on the template instantiation too, which is fun.

Mold (linker) 1.0 released

Posted Dec 16, 2021 0:27 UTC (Thu) by rillian (subscriber, #11344) [Link] (2 responses)

Well, most of it is probably debug tables. You'll note the release downloads are ~10% that size.

These are also huge programs. According to tokei on today's main branches:

firefox 37M lines of code (3M C, 6M C++, 4M headers, 8M Javascript, 3M Rust)
chromium 36M lines (14M C++, 5M headers, 4M Javascript)
linux 32M lines (21M C, 7M headers)

Browsers are effectively an entire userspace unto themselves, so it's not surprising builds are the same size as a minimal desktop image.

Otherwise, what mathstuf said. Link times on large C++ applications can be a real bottleneck, especially for incremental development. Linux has the advantage of being in C, and more modular.

Mold (linker) 1.0 released

Posted Dec 16, 2021 12:47 UTC (Thu) by cesarb (subscriber, #6266) [Link] (1 responses)

> Linux has the advantage of being in C, and more modular.

Linux also heavily uses partial linking (ld -r), which allows a large part of the linking step to run incrementally and/or in parallel. That does not seem to be usual for other large C or C++ projects I've seen.

Mold (linker) 1.0 released

Posted Dec 16, 2021 14:28 UTC (Thu) by Vipketsh (guest, #134480) [Link]

In my 5.12 kernel tree that seems to have been changed to use the more conventional method of archives.

Even with 'ld -r' the symbol lookup does not get eliminated during the final link which is, according to the documentation here, one of the slowest part of linking, so I doubt that using 'ld -r' has a meaningful impact on linking speed.