Not logged in
Log in now
Create an account
Subscribe to LWN
LWN.net Weekly Edition for May 23, 2013
An "enum" for Python 3
An unexpected perf feature
LWN.net Weekly Edition for May 16, 2013
A look at the PyPy 2.0 release
Cool new Free software
Posted Dec 19, 2012 12:20 UTC (Wed) by man_ls (subscriber, #15091)
Our friends in the functional community should appreciate the return of functional programming, with a vengeance: in node, everything is a callback. I remember reading the tortured syntax that Paul Graham had to write to enable some kind of pseudo-multiprocessing in Lisp; in node, it is all there and nicely managed for you.
Posted Dec 19, 2012 14:40 UTC (Wed) by ms (subscriber, #41272)
Callbacks lead to horrible staircases of code which tend to look fugly, and infect all APIs everywhere. Things like promises make for slightly nicer looking code, and give you more choices from an API POV, but are still not brilliant. IMHO, the "fact" is that for nice looking code, which is intuitive and readable, you want to be able to write blocking code.
I just don't understand why people think that callback-based programming is necessary for good performance. Most sane languages with a good RTS and green threads will be able to match the performance of NodeJS for anything involving IO. And almost certainly lead to better looking code. The popularity of NodeJS I find most perplexing. Maybe most other languages do not have good RTS or green threads.
Posted Dec 19, 2012 15:53 UTC (Wed) by man_ls (subscriber, #15091)
Anyway it is possible to write code that hides some of this complexity, avoiding anonymous functions for most of the work. The functional programming model and the object oriented stuff can be made to play together nicely with some effort.
Since a few years ago I have wondered why the kernel doesn't do something similar with an application's event loop: instead of the app having to call and wait for some time (or, god forgive us, poll all the time), the kernel should drop its privileges and give control to the app's process directly. Similar to Java event handlers, I guess, but without the JVM overhead. Async everywhere!
Posted Dec 19, 2012 16:13 UTC (Wed) by nix (subscriber, #2304)
the kernel should drop its privileges and give control to the app's process directly
Posted Dec 19, 2012 16:29 UTC (Wed) by man_ls (subscriber, #15091)
Posted Dec 19, 2012 16:52 UTC (Wed) by nix (subscriber, #2304)
Creating a new process for every single event seems sure to be far more expensive than just a privilege transition out of kernel mode. My system here can manage about five or ten thousand forks per second, but it can manage well over a million context switches into and out of kernel mode per second.
(As for a process being 'based on' a kernel process, the level of your confusion regarding how processes work in Unix is seemingly so great that I can't even imagine what you intend this to mean. Your last line suggests that you expect it to have a whole new address space of its own, which makes the thing a process-creation-level of difficulty rather than a thread-creation-level, and requires among other things a TLB flush. Expensive, expensive...)
Posted Dec 19, 2012 17:27 UTC (Wed) by man_ls (subscriber, #15091)
Posted Dec 19, 2012 19:42 UTC (Wed) by nix (subscriber, #2304)
Posted Dec 19, 2012 19:47 UTC (Wed) by man_ls (subscriber, #15091)
Posted Dec 19, 2012 16:53 UTC (Wed) by mpr22 (subscriber, #60784)
Posted Dec 19, 2012 17:31 UTC (Wed) by man_ls (subscriber, #15091)
Posted Dec 19, 2012 19:48 UTC (Wed) by nix (subscriber, #2304)
The softer version of fork() you refer to is called thread creation. It is faster -- I can generate a few tens of thousands of threads a second on this system -- but still not anywhere near as fast as context switching. You are still throwing performance away for, as far as I can tell, no benefit.
Worse yet -- if you're receiving a stream of events from some input device, they are almost certainly related in some way, and you almost always want to process them in the same process, or at least in related processes. Kicking off a new process for each event seems likely to lead to cursing from developers as they have to marshal the things back together again before they can process them. End result: a slower system, annoyed developers, more complex and buggy code, and... uh... no benefit, as far as I can see. What benefit is all this supposed to bring? Is it just 'more processes -> more isolation -> always necessarily better'? 'cos I used to think like that, and it just isn't so.
Posted Dec 19, 2012 20:46 UTC (Wed) by man_ls (subscriber, #15091)
The thing is that an event should only do what it is supposed to do, and nothing else. If the result of a mouse movement is that a counter is updated, then let the kernel do it, and not bother my own process; when my process needs to read the counter it will do that. If the mouse pointer is moved and I want to paint a cool shadow, then ideally I would let the kernel know and the kernel will paint the cool shadow. Many things work like that now; it would be great if it was the default mechanism to answer user events. Look at it as dtrace on steroids, if you want.
For a long task (such as a menu selection), then by all means create a process and let it deal with the event. Otherwise, store the new value and let me query it if and when I need it. Taken to the extreme I don't even need to have running processes except for background tasks.
Posted Dec 19, 2012 22:50 UTC (Wed) by dlang (✭ supporter ✭, #313)
Taking your mouse example, some graphics cards do support hardware mouse cursors, where the card handles moving the mouse pointer.
however, such support really doesn't help much and is fading from use today because nowdays people want to change the shape of the cursor depending on what you are doing, etc.
you also have to think about how the program should know that it should look for an update to the mouse position. you don't want it spinning in a loop when nothing happens.
you also don't just want the current position of the mouse, you want to know it's path, and how fast it's moving (along with what buttons, if any are pressed during any part of this time)
There are pretty good reasons that systems do things the way they do. People have tried a LOT of things over the years, and what's in place may not be the perfect way to do things, but they've survived lots of competing ideas along the way.
Posted Dec 19, 2012 23:03 UTC (Wed) by man_ls (subscriber, #15091)
Posted Dec 20, 2012 12:53 UTC (Thu) by njs (guest, #40338)
When a user process blocks, it doesn't actually sit there as an entity taking up space. It gets converted into a callback attached to whatever event caused it to block.
Posted Dec 20, 2012 18:37 UTC (Thu) by jch (guest, #51929)
We've done some benchmarks comparing threads with (machine-generated) event-driven code. Unsurprisingly enough, the results indicate that well implemented user-space cooperative ("green") threads are just as fast as event-driven code, but use vastly more memory. Surprisingly enough, Linux' NPTL is only some 50% slower, which I find very impressive.
See Figures 4 and 6 on pages 10 and 12 of http://www.pps.univ-paris-diderot.fr/~jch/research/cpc-20...
Posted Dec 20, 2012 20:21 UTC (Thu) by nix (subscriber, #2304)
Posted Dec 20, 2012 20:57 UTC (Thu) by job (guest, #670)
But I would say that's probably what people like about it. It's not interesting, it works and it's easy to understand. Perhaps that makes for a large ecosystem one day. V8 certainly gives great performance. That's a great combination.
(Personally I think the callback spaghetti makes for hard to read code. Let's just say I wouldn't want to debug someone else's code. And this is from someone who actually likes Perl.)
Copyright © 2013, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds