Unix sockets

Posted Dec 26, 2024 23:07 UTC (Thu) by ibukanov (subscriber, #3942)
In reply to: Unix sockets by Wol
Parent article: Systemd improves image features and adds varlink API

The difference between the fish and, say, setuid is that evolution is blind and proceeds by killing things that do not work while randomly mutating things. So for the fish the relative position of organs did not matter while setuid was designed to address, as was pointed in another sub thread, a very particular problem. A collection of IPC services was not an option in the very early UNIX.

And again, for an individual land animal the cost of switching is impossible as it means death. Yet in IT one can implement the solution, try it, show that it works yet it is extremely hard to make others to use it even if it is beneficial.

Unix sockets

Posted Dec 26, 2024 23:54 UTC (Thu) by khim (subscriber, #9252) [Link] (7 responses)

> The difference between the fish and, say, setuid is that evolution is blind and proceeds by killing things that do not work while randomly mutating things.

Software development works in very similar way. The fact that in software development mutations are not random is compensated by the fact that we may only try maybe dozen or two dozen of ways in a very short time while evolution can try billions of combinations for millions of years.

> And again, for an individual land animal the cost of switching is impossible as it means death.

Same with software. Almost all attempts to “radically redesign” something leads to a stillborn creation like Workplace OS or BB10.

> Yet in IT one can implement the solution, try it, show that it works yet it is extremely hard to make others to use it even if it is beneficial.

The reason is the exact same as with animals: intermediate stages where two solutions to the same problem are implemented (like Lungfish) need very special and “strange” environment to survive.

And without such environments it's not possible to replace things piecemeal.

Essentially after software passed the threshold of development times measured in thousands of man-years we entered the exact same situation as with evolution: any small, self-contained element can be rewritten… pretty radically… feathers are very different from fish scales. But change that effects bazillion other pieces? Fuhgeddaboudit.

CP/CMS is an ancient history, but driver letters would be forever with us in all descendants of that OS.

The only way to fix something is to build a replacement from scratch… and if we are talking about something “big” that usually fails.

Unix sockets

Posted Dec 27, 2024 10:05 UTC (Fri) by ibukanov (subscriber, #3942) [Link] (6 responses)

> Almost all attempts to “radically redesign” something leads to a stillborn creation like Workplace OS or BB10.

Those were not stillborn. They worked for a small group of people and were beneficial for them. Yet others were not interested. Or consider Google Fibers described in another thread. They were beneficial for Google and used on a massive scale there.

An evolutionary analogy is if beneficial mutations spanning huge number of genes can regularly happens, but then disappear without traces. This just does not happen.

In general according to our understanding the observed evolution in Nature is a random process *without memory* in a sense that a rate of mutations and their direction does not depend on the previous mutation history. Software mostly designed and clearly its process depends very much on success or failure of the previous steps.

Unix sockets

Posted Dec 27, 2024 11:15 UTC (Fri) by khim (subscriber, #9252) [Link]

> They worked for a small group of people and were beneficial for them.

Who benefited from them? Workplace OS was dead before anything was released on it's base. To anyone! Development team was disbanded even before the only product based it was released.

> Or consider Google Fibers described in another thread. They were beneficial for Google and used on a massive scale there.

Just like Trilobites once dominated the world. They were Apex predators, back then. And then disappeared without leaving any descendants behind.

> Software mostly designed and clearly its process depends very much on success or failure of the previous steps.

So 99% like a biological evolution then? Why do you think the remaining 1% is so important.

> In general according to our understanding the observed evolution in Nature is a random process *without memory* in a sense that a rate of mutations and their direction does not depend on the previous mutation history.

They don't depend on the “mutation history” but they definitely depend on the result of that process. Same with software development. We may debate about how ABCDE…Z mutated into QWERTY, but that's only of interest for some hobbysts and historicians. The others are confronted with the fact that most keyboards put keys in the QWERTY order and not in the ABCDE…Z (or, in some case, the opposite, TI Nspire, an TI-92 descendant, specifically doesn't use QWERTY to avoid being classified as “computer”).

Evolution of biological species and software development are closer than you think. Even situation when someone's pours billions on a development of something that would, most likely, never go anywhere (like Google currently does with Fuchsia) have a close analogue in a biological world.

Unix sockets

Posted Dec 27, 2024 11:49 UTC (Fri) by magfr (subscriber, #16052) [Link]

The Biology analogies are interesting.

Trilobites were wildly successful - killing them off required multiple mass extinction events.

As for specialist environments with runaway adaptations, that is usually known as island endemic species like pygmy mammoths, Komodo dragons and lots of other examples.

Unix sockets

Posted Dec 27, 2024 21:34 UTC (Fri) by smurf (subscriber, #17840) [Link] (3 responses)

> Or consider Google Fibers described in another thread. They were beneficial for Google and used on a massive scale there.

You're using the past tense. Are they still used?

Unix sockets

Posted Dec 27, 2024 21:58 UTC (Fri) by khim (subscriber, #9252) [Link] (2 responses)

Last time I have heard about them from reliable sources they were still massively used for, more or less, everything, but people were speaking about them in the past tense because Google was talking about “stopping being a little island” and embracing what everyone else is embracing.

And I'm pretty sure the end result would be the reverse of the outcome in other places: they would still continue to power the majority of Google services for decades to come, while people would start pretending that Google is now “aligned with the rest of the world”, only need to change “few vestigial places” while in reality it would still be 90% Google Fibers and 10% async/await for the foreseeable future.

Unix sockets

Posted Dec 30, 2024 13:37 UTC (Mon) by surajm (subscriber, #135863) [Link] (1 responses)

Its worth noting that Google is quite diverse these days and not everything lives in the monorepo, or runs on a server. Android and chrome codebases are large and don't have anything close to resembling a fiber. Largely because outside of servers, fibers don't scale well in terms of memory usage.

One of the reasons to pursue async await in c++ is that it will make future interop with a c++ successor easier as well. For example, it's likely going to be possible to get rust and c++ code to share async machinery rather than need two language specific runtimes running in parallel.

Lastly it's worth mentioning that the fiber model is not perfect. Microsoft abandoned something similar in Windows and anyone who has used go understands some of the sharp edges it can have.

Unix sockets

Posted Dec 30, 2024 14:11 UTC (Mon) by khim (subscriber, #9252) [Link]

> Microsoft abandoned something similar in Windows

How can it abandon something that it never had? Microsoft never had anything resembling Google Fibers. Even from the description it's completely obvious: fibers are not preemptively scheduled… if a fiber accesses thread local storage (TLS), it is accessing the thread local storage of the thread that is running it… if a fiber calls the ExitThread function, the thread that is running it exits.

Microsoft Fibers do have the issues that spooked my opponents: they do need non-blocking syscalls, they couldn't use synchronous code, etc.

They are closer to async/await machinery then to Google Fibers.

> and anyone who has used go understands some of the sharp edges it can have.

Sure. Because go is yet another machinery that's similar to async/await and different from Google Fibers. Goroutines also need non-blocking syscalls and couldn't use regular thread-pinned Mutexes and other such constructs.

Which is the main selling point of Google Fibers: fibers are threads, patterns that work for threads such as Mutex-based synchronization “just work” with Fibers as well and new constructs introduced with Fibers “just work” when used by a non-Fiber thread. Have they removed that from go/fibers ? From what I understand these were main selling points when Google Fibers were developed – and they were never present in Microsoft's Windows Fibers and/or goroutines.

> For example, it's likely going to be possible to get rust and c++ code to share async machinery rather than need two language specific runtimes running in parallel.

Yeah. “Let's not be an island” mantra that Google started promoting few years ago. We'll see how well would it work. So far I'm skeptical, but then I was skeptical about crazy C++ idea of generating tons of useless code that compiler would then be eliminating – and that one worked pretty decently (in fact without that work of LLVM developers Rust wouldn't be viable).

P.S. My biggest grief with async/await model is that instead of “going back the root cause” and dropping POSIX for something better it tries to add yet more lipstick on the pig. Rust gives the hope, though: it's growing popularity in the embedded space, where POSIX limitations don't exist means that someone may finally build something usable and working out of async/await model. That would actually be better than Google Fibers. And then it can be adopted by Linux… but chances of that happening are very slim. That's a lot of work and most developers don't even know that such alternative may exist, they are too accustomed to what POSIX does.