LWN: Comments on "Inside NGINX: How We Designed for Performance & Scale"

Similar to VoltDB

JB318 — Mon, 15 Jun 2015 13:33:27 +0000

There was a recent article about VoltDB, an in-memory relational database that does very similar things. In particular, the ideas of one process/thread per CPU (not one per user), and of rearchitecting things to avoid waiting on network I/O (either the transmission itself or user "think time").

Celebrate success

bronson — Mon, 15 Jun 2015 06:25:16 +0000

For example, Github Pages: http://githubengineering.com/rearchitecting-github-pages/

"the entire service ran on a single pair of machines"

Celebrate success

Sesse — Sat, 13 Jun 2015 20:30:53 +0000

If you're hosting hundreds of thousands of sites for various customers, and they all get to modify aspects of their own configuration (like making new sites)?

Inside NGINX: How We Designed for Performance & Scale

wahern — Fri, 12 Jun 2015 22:19:55 +0000

The C10K problem was written nearly 15 years ago.

Nothing NGINX does is interesting for people who actually implement these services. We're all using the same patterns, subject to various constraints. The NGINX article touts using features like Linux's SO_REUSEPORT* to address thundering herds with accept(2). But that was added only a couple of years ago, and it was a Google team who added it. And that team wasn't using NGINX, but nonetheless probably using custom software employing the very same patterns. My point being, NGINX isn't leading the pack here--they're following, just like everybody else. These patterns have grown organically from the experience of countless engineers.

The article is just a piece of marketing.

The problem with these patterns is that they're really hard to use. You're stuck using either callbacks or explicit state machines to resume operations between events, or some mixture of the two. Most implementations choose callbacks. In this context callbacks operate as non-local jumps, which means such code is very difficult to reason about and breed bugs. It's why implementations like Go, Warp, and my own Lua cqueues module try to hide this complexity using threads or coroutines. Actually making these patterns easy to build complex, ad hoc applications upon without losing performance is the hard part. NGINX is less than stellar in that regard.

* I say "Linux's" because the semantics are incompatible with the historic BSD option of the same name.

Inside NGINX: How We Designed for Performance & Scale

sodabrew — Fri, 12 Jun 2015 18:46:09 +0000

> Surely everyone has been to a bank or post office?

I think everyone identifies the wrong problems at those places, presented here in sweeping generalizations and personal anecdotes:

Most people have been to establishments where you wait in one line and get dedicated service when it's your turn. Slow service is considered a problem of not enough clerks.

Personal anecdote: I was waiting in line at my local post office to pick up a package. I had a "sorry we missed you" pickup slip. So did half the people in line. One clerk came out and said, "If you have a pickup slip give it to me!" She took 10 slips, went in back, and came back five minutes later with 10 boxes. We all left the post office, satisfied at this quick service. Meanwhile, someone in the middle of the line (who had benefited now that fewer people were ahead of him!) complained that some people with pickup slips had been behind him in line, and he wanted to buy his stamps RIGHT NOW.

Most people complain when they visit an establishment that hands you off between specialists - think hardware store (Clerk says "Go to plumbing in aisle 11 and wait for a plumbing specialist to help you"). Slow service is considered a problem of not having the first clerk cross-trained in plumbing.

Personal anecdote: I went to my local hardware store and asked for a piece of pipe to be cut and threaded. The first clerk I spoke with said sure, he could do that. We went to plumbing, and he struggled for 20 minutes to cut and thread the pipe. He did it incorrectly twice, which would have been a safety hazard for my home. Eventually I demanded that he find someone better trained on the equipment to help me.

Inside NGINX: How We Designed for Performance & Scale

MoSal — Fri, 12 Jun 2015 16:54:23 +0000

H2O is also interesting:

https://h2o.examp1e.net/
http://blog.kazuhooku.com/2014/11/the-internals-h2o-or-ho...

Inside NGINX: How We Designed for Performance & Scale

ncm — Fri, 12 Jun 2015 12:17:17 +0000

Maybe I am too experienced but I find it hard to understand how anyone would have any difficulty with this architecture. Surely everyone has been to a bank or post office?

Inside NGINX: How We Designed for Performance & Scale

ibukanov — Fri, 12 Jun 2015 07:08:58 +0000

Thanks for the link, it was very interesting read indeed.

I am not sure that it is immutable data that helps Warp most. Warp greatly benefited from high performance green threads and the garbage collection in GHC. It allows to use straightforward procedural code style and freely use suitable data structures without the need to worry for memory management.

Also, although the scalability of Warp compared with Nginx regarding throughput was impressive, it would be interesting to get comparison of Warp versus nginx regarding max response time, CPU utilization and memory usage as green thread abstractions and garabage collection typically harm those.

Inside NGINX: How We Designed for Performance & Scale

kleptog — Fri, 12 Jun 2015 06:19:32 +0000

Splitting everything into jobs the way they do and running everything in a few worker threads is a powerful mechanism and exists elsewhere, such as in Python Twisted (though it works with Deferreds (promises) the effect is the same).

What I do find is that it's a mode of programming that many people find very difficult to wrap their heads around. Maybe it's just unfamiliarity and it just needs to be taught in schools more. But some projects where I think it would be a perfect fit I end up not doing it because I fear the people who have to maintain it after me would get completely lost.

Celebrate success

flussence — Thu, 11 Jun 2015 23:12:49 +0000

I can't think of a good example, but it seems like the sort of usage pattern an automated test suite would generate.

Inside NGINX: How We Designed for Performance & Scale

einstein — Thu, 11 Jun 2015 22:47:25 +0000

Very interesting... I might want to dust off that old Haskell book and take another look

Inside NGINX: How We Designed for Performance & Scale

oever — Thu, 11 Jun 2015 16:30:38 +0000

And reading how Warp, a web server written in Haskell, manages to compete with NGINX is an insightful follow-up.
http://www.aosabook.org/en/posa/warp.html

The diagrams explaining the different threading strategies are very insightful.
I find it wonderful to see how having a very safe language with immutable data structures can lead to an elegant but fast design.

Celebrate success

ncm — Thu, 11 Jun 2015 03:46:42 +0000

They probably deserve even their own congratulations. It is an impressive achievement.

I wonder, though, why a site would load a new configuration many times per second.

Inside NGINX: How We Designed for Performance & Scale

miah — Wed, 10 Jun 2015 19:43:18 +0000

I'd also recommend reading their chapter from Architecture of Open Source Applications; http://www.aosabook.org/en/nginx.html