LWN: Comments on "Cracks in the Foundation (PHP Advent)"

Cracks in the Foundation (PHP Advent)

etienne — Tue, 03 Jan 2012 12:41:07 +0000

>> returning flags from functions
> You mean like the errno variable?

Not really, errno is a global variable - I was thinking of something very fast at the asssembly level, for the million of use case.
Something like (doesn't really work in C):

unsigned mybitfield, bitindex;
ifzeroset(bitno = ffs(mybitfield)) // ffs = find first bit set
error("none bit are set");
else
bitindex = bitno;

Translated in ia32 assembler by:
ffs eax,edx
jz call_error
mov edx,bitindex

You can replace ffs by strchr or any basic function which may not be inlined.
Maybe it could also be implemented in GCC by:
register struct flags {
unsigned zero : 1;
unsigned carry : 1;
...
} flags asm ("cc");
but the modification of flags by so many assembly instruction is probably a problem (reordering instructions optimisations).

Why is it a mess?

juliank — Sun, 25 Dec 2011 14:09:19 +0000

> myint <> 0

Use myint != 0. That other form is deprecated in Python 2, and removed in Python 3.

Cracks in the Foundation (PHP Advent)

jezuch — Fri, 23 Dec 2011 10:09:53 +0000

> Sometimes, in C, I would like a function to return a value (like now) *and* the (processor) flags - for instance the "zero" flag would mean no error occured. Very fast and simple...

You mean like the errno variable?

Cracks in the Foundation (PHP Advent)

etienne — Thu, 22 Dec 2011 11:59:11 +0000

> Multiple return values

Sometimes, in C, I would like a function to return a value (like now) *and* the (processor) flags - for instance the "zero" flag would mean no error occured. Very fast and simple...

Cracks in the Foundation (PHP Advent)

justincormack — Thu, 22 Dec 2011 11:20:32 +0000

Multiple return values is the real answer here. Exceptions are probably overkill. Dont suppose php will get them though.

turing complete

raven667 — Thu, 22 Dec 2011 05:32:06 +0000

Where's the fun in that? Go out guns blazing.

turing complete

bronson — Thu, 22 Dec 2011 03:18:09 +0000

If you're going to concede, why not concede gracefully?

This is the most ridiculous commentary...

job — Thu, 22 Dec 2011 00:22:49 +0000

On a large shared web host, a very small percentage of the sites are have traffic at any given time, but it's still a lot in absolute numbers.

The only possible way to cram as many sites as possible on your host is to run them with a common interpreter, which can pose security problems. That's where mod_php succeeded.

(By the way, I got the answer to my specific question below, which is that other restrictions still apply.)

Cracks in the Foundation (PHP Advent)

andreasb — Wed, 21 Dec 2011 13:59:19 +0000

[wildspeculation]Maybe Facebook found that they get off cheaper by investing in a handful of people who know how to write compilers and virtual machines so that they can continue to let dime-a-dozen PHP coders do the bulk of the frontend work.[/wildspeculation]

This is the most ridiculous commentary...

andresfreund — Wed, 21 Dec 2011 12:02:20 +0000

> When the number of idle sites are order of magnitudes larger than the number of active sites at any given moment that solution is equivalent to running PHP as CGI. That would price you out of the market. (And spare me the morals please, as much as I'd like to see mod_php dead it is still an important market which drove PHP's success.)
Right, because idle sites cause that much load that cgi is actually a problem... Its the website that actually use their allotted bandwith/load/whatever that are a problem not the thousand with 3 visitors a day.

This is the most ridiculous commentary...

job — Wed, 21 Dec 2011 08:48:52 +0000

That explains it. I thought basedir restrictions was part of safe_mode.

Why is it a mess?

ekj — Wed, 21 Dec 2011 07:37:39 +0000

Yeah. But even python considers boolean false and integer zero to be equivalent, and all other integers to be true. Indeed the method to override to define your own behaviour for if object is object.__nonzero__ or object.__len__

From the naming alone it's clear that being "true" by convention means "having length" or "not being zero". (why is the method __nonzero__ and not __zero__ (with opposite semantics) by the way, seems an odd kind of superfluous negation. Returning false hear means that the object is *not* *nonzero* i.e. that it's zero.

Cracks in the Foundation (PHP Advent)

nybble41 — Wed, 21 Dec 2011 04:26:23 +0000

> BUT, what do you do with an integer-function that normally returns an integer (negative, positive, or zero) when it needs to return an error?

Return success or failure (or a more specific error code), and store the result in a reference parameter? That's the standard C approach. If the possibility of failure exists, you'll need to check for that first anyway before using the result. For uncommon failure modes, in languages which support it, alternate continuations (including, but not limited to, C++-style exception handling) offer a more efficient solution.

> PHP has adopted the general convention that any function that fails will return false; I think this is actually quite sensible once one knows to expect it.

That's great so long as you don't need to return false in a non-failure situation... The Common Lisp solution to this is rather elegant: return two values, the first being the result or false, and the second (which you can ignore) indicating success or failure. If you know that successful results can't be false you can use the normal return value, but the extended status is there if you need it.

Cracks in the Foundation (PHP Advent)

Richard_J_Neill — Wed, 21 Dec 2011 02:37:02 +0000

Perhaps using strpos() wasn't quite such a good example, because -1 could never be a valid answer, and is therefore potentially OK as an error-flag.
This is in the same spirit as, for example, C's write() .

BUT, what do you do with an integer-function that normally returns an integer (negative, positive, or zero) when it needs to return an error?

There are several ways to do it; of which I think that C's strtol() is the worst possible. Some might suggest returning an object, but that's logically equivalent. PHP has adopted the general convention that any function that fails will return false; I think this is actually quite sensible once one knows to expect it.

As for the casting rules of "0.0" vs 0.0, it's rather a perverse example, which shouldn't happen in real-life.

Why is it a mess?

pboddie — Tue, 20 Dec 2011 23:32:00 +0000

In languages like perl, php and python (which try to "guess" what you meant and "help" you) it's disaster, obviously.

Python doesn't "guess" anything. There's a protocol that all classes support which indicates whether an instance is considered true or false.

Cracks in the Foundation (PHP Advent)

hholzgra — Tue, 20 Dec 2011 23:02:42 +0000

The old ext/mysql extension did indeed not support prepared statements (just as the MySQL client protocol did not support them either at the time when ext/mysql created)

The newer ext/mysqli extension fixed that around 2003 already when prepared statement support became in MySQL, but somehow lots of people prefer to stay with the "classic" extensions API ...

This is the most ridiculous commentary...

hholzgra — Tue, 20 Dec 2011 22:58:01 +0000

Being able to define memory and cpu time limits, and maybe open-basedir restrictions, were (AFAIR) more important to shared hosters than safe-mode (which was never that safe anyway), and all of these still exist.

This is the most ridiculous commentary...

job — Tue, 20 Dec 2011 16:14:18 +0000

When the number of idle sites are order of magnitudes larger than the number of active sites at any given moment that solution is equivalent to running PHP as CGI. That would price you out of the market. (And spare me the morals please, as much as I'd like to see mod_php dead it is still an important market which drove PHP's success.)

PHP won because of mod_php, but mod_php won the low end market because of safe_mode. My question still stands, what should replace it? Maybe SELinux can?

Cracks in the Foundation (PHP Advent)

HelloWorld — Tue, 20 Dec 2011 15:32:56 +0000

> At what point do you get so much NIHitis that you think you should be writing your own compilers and VMs for a crappy web language instead of switching to a non-crappy one?
Well, if it's easier/cheaper to write a better implementation of the language than to switch the language, it's clearly the right thing to do, isn't it?

turing complete

HelloWorld — Tue, 20 Dec 2011 14:22:11 +0000

I had actually considered to learn that language, as it seems to be pretty much the only language with first-class cases. I'm pretty busy with other things right now though.

turing complete

Cyberax — Tue, 20 Dec 2011 14:16:29 +0000

Heh. Check Ur ( http://www.impredicative.com/ur/ ) one day. It works by treating HTML as a state.

Cracks in the Foundation (PHP Advent)

sorpigal — Tue, 20 Dec 2011 12:55:38 +0000

Am I the only one who thinks this is crazy? At what point do you get so much NIHitis that you think you should be writing your own compilers and VMs for a crappy web language instead of switching to a non-crappy one? The first time it was done could be seen as an emergency measure to help along legact code, but at that point shouldn't your lesson be learned and shouldn't you be trying to find the useful replacement for PHP and not just bandaiding it again and again?

This is the most ridiculous commentary...

smurf — Tue, 20 Dec 2011 12:23:24 +0000

Sure it's unreasonable to keep them around. So let them die when they're no longer useful.

Large sites increase the likelihood that somebody, somewhere on your system, is running code with a security hole or two. So it's even more important to shield individual customers from each other.

Your decision, of course. But the web servers I am responsible for will never load mod_interpreter, much less mod_php.

This is funny...

nix — Tue, 20 Dec 2011 11:38:24 +0000

As for last_insert_id et al, well, every abstraction layer has costs. One cost of this one was that you had to use a not-completely-disgusting data model, i.e., that having more than one sequence tracking a single table was not supported. I never felt the lack: compound PKs are all very well, but if you use a sequence (mysql: autoincrement) as a PK it means that nothing else would do. Why on earth would you want to use a compound PK consisting of multiple sequences? That smacks of very bad table design to me. I thought of various more or less ugly ways to fix this but never implemented them because of a total lack of need.

(It was not a very pleasant abstraction layer to write, I'll grant you that. But, still, abstracting over this sort of thing *is* the job of an abstraction layer.)

This is the most ridiculous commentary...

job — Tue, 20 Dec 2011 11:09:06 +0000

For large web apps this is (and has always been) true. But the issue here is altogether different.

A large shared webhost has many thousands of web sites and only a few of them are active at any time. It is unreasonable to keep thousands of idle PHP processes around (and that is without room for concorrency which would be required in production).

That is what the web hotel business is all about, very cheap hosting, and this is what made PHP popular.

turing complete

smurf — Tue, 20 Dec 2011 10:39:44 +0000

Please engage brain.

A PHP without external state is as powerful as a pure Lambda calculus. That's a fact.

PHP has features to talk to the external worls, save state, and whatnot, because it's designed to be (mostly) useful. A pure lambda calculus has not, because it's designed to prove computability theorems and related stuff.

From a mathematical PoV, that's rather superficial. You can write a Web server in COBOL or Intercal or even Brainfuck if you add the required features somehow, and you can write a Prolog theorem prover in them too.

Doesn't mean that doing any of this makes any practical sense whatsoever.

This is the most ridiculous commentary...

smurf — Tue, 20 Dec 2011 10:17:13 +0000

FastCGI and separate PHP processes per user, of course.

It doesn't make any sense at all to bloat your Apache with mod_php. If you have 200 Apache processes, of which 10 are needed for PHP, the (non-trivial, these days) memory space of 190 PHP interpreters sits idle. It can't even be swapped out because the set of Apache processes working on PHP requests constantly changes.

Using FastCGI, you have 10 busy PHP interpreters and 190 "normal" Apache (or lighttp or ) processors, and you get to run each customer's PHP stuff under their own UID instead of having global access to anything that's readable by www_data when (not if) the next security hole is discovered.

turing complete

HelloWorld — Tue, 20 Dec 2011 09:50:10 +0000

> So your main complaint now is that the untyped lambda calculus can't pretend to be a hardware random number generator?
No, it's not. But your response shows yet again that this discussion is utterly pointless. You don't want to get the point, fine with me. Have a nice life.

Yup

niner — Tue, 20 Dec 2011 07:38:04 +0000

"It must only be used with MySQL, really. If you want to use Oracle and Postgresql then it's much better to use sequences and their nextval and currval directly."

Not really. It covers the simple case quite well which is the vast majority in our applications. If I want to use PostgreSQL features, I'd use returning to get the new ID back to the application without an additional roundtrip.

turing complete

tialaramex — Tue, 20 Dec 2011 04:23:25 +0000

So your main complaint now is that the untyped lambda calculus can't pretend to be a hardware random number generator? Are you even paying attention to the discussion? Hardware random number generators aren't a PHP feature. The question wasn't "can a lambda expression correctly simulate the hardware of an entire web server?".

Your objection to state cookies is spurious. The state cookie is opaque outside of the lambda expression, all the hard work happens _inside_ the lambda expression. If the state held between requests is small and client-specific you can sidestep this altogether by asking the user agent to hold on to it for you, in your HTTP response. In fact you may recall that recently Microsoft ASP.NET was found to have introduced a serious security vulnerability by transmitting inadequately encrypted session state in user-accessible HTTP cookies.

It's certainly not fair to pretend that most, or even many web sites depend somehow upon the "state of the whole world". Usually there's a modest database, and perhaps some files stored on disk. A pure functional system could easily wrap all that material into the state cookie. Many sites _appear_ to integrate content from several other systems, but actually that's usually done client side in the user agent these days, the "integration" from the point of view of PHP or our hypothetical lambda expression is just a few lines of unchanging plain text returned in the HTTP response.

There are numerous _practical_ problems which keep web sites from being built out of say, pure Lisp, but theoretically there's no real obstacle.

Cracks in the Foundation (PHP Advent)

andreasb — Tue, 20 Dec 2011 01:26:47 +0000

Specifically, they used to compile PHP to C++ then to native binaries. Now they are deploying a PHP to virtual machine code compiler with a JIT compiling virtual machine.

https://www.facebook.com/note.php?note_id=101504151779289...

turing complete

HelloWorld — Tue, 20 Dec 2011 00:57:25 +0000

Accusing people of "trolling" when they explain why you're wrong is poor form.

I'm not.

The general idea you had was fine, there are indeed applications which necessarily involve side effects and the lambda calculus doesn't have side effects. But answering individual HTTP requests (remember PHP isn't a web server, it doesn't need to care about the details of how those requests are made, where they come from, or how answers are returned) isn't really one of those applications. Thus the lack of side effects isn't a deal breaker, and my explanation deliberately didn't use them.

Yeah, except that it *is* a deal breaker because you need a lot more than just the contents of the HTTP request in order to do anything meaningful (i. e. something that couldn't be done just as well with JavaScript on the client side). You know, things like access the database or communicate with other hosts. OK, you can get the database thing to work by passing the entire contents of the database to your lambda expression and have it return a tuple containing the response and the complete new database contents. Not only may one consider that "cheating" (as you need a more and more elaborate machinery outside of the lambda calculus expression, to the point that you can't actually claim to have done anything meaningful in said expression any longer), but that approach also quickly leads to a dead end where you'd need to pass the state of the whole world to the lambda expression in order to obtain the desired answer. And actually, even that wouldn't suffice. Imagine a web server with a hardware random number generator and a web application that allows you to query the xth number generated from now on (where x is supplied by the client). Due to Heisenberg's uncertainty principle, you can't possibly measure the state of the RNG well enough to predict what the xth generated number will be, so you clearly need some kind of side effect for that kind of application. The reason I didn't write all this stuff earlier is that I just couldn't be bothered to argue with a troll like nybble41. There's simply no value at all in this discussion, and nothing enlightening is going to come out of it (not for me, anyway). The only reason I engaged in it at all is that I don't like being told I'm wrong when I'm not.

Good try...

khim — Tue, 20 Dec 2011 00:35:25 +0000

No, I've never used SQL from python so I never used SQLAlchemy. Looks like fine ORM toolkit, though. As for SQL as any good SQL driver it includes dialects. This is where you can specify sequences in Oracle, disable autoincrement in MySQL, etc.

Even so: I can not see how exactly you can chain multiple INSERTs in one BEGIN/END Oracle statement or execute multiquery in MySQL (like you can do with PHP).

This is funny...

khim — Tue, 20 Dec 2011 00:04:47 +0000

Your belief that RETURNING doesn't work in Oracle is untrue. It does (though you have to RETURNING a .nextval, indeed), and is the recommended approach rather than using .currval, because it saves a roundtrip to the database.

Of course RETURNING works, but... how exactly will it save the roundtrip? When you use nextval/currval idiom you can just batch all your operation in one statement using BEGIN/END - thus using one single roudtrip. How exactly last_insert_id emulation can beat this is mystery to me.

Sure, sometimes you want to do some manipulations in your application before continuing - and in this case it's natural to use last_insert_id with MySQL (but not with Oracle).

Writing an abstraction layer which abstracts over all SQL differences is of course over the top... but abstracting over trivial, always-needed stuff like 'what is the sequence number of the row I just inserted' is, well, trivial.

Actually it's not trivial. In MySQL LAST_INSERT_ID only gives you one number but in Postresql or Oracle you can generate few different numbers. And since one roundtrip can include quite a few elemental operations it's not easy or simple to transform one to another.

I've written such an abstraction layer myself more than once, and that was a little one-man thing, and it was not very much effort.

WOW! Cool.

How exactly have you processed Oracle's insert statements which used two distinct sequences in MySQL?

That PHP couldn't do it says volumes.

About your unreasonable expectations? Sure. About SQL driver? Not much. DBI example which produces suboptimal results does not expire confidence: I'd rather handle such problems at the application level rather then in intense debugging session which will show that that my perfectly tuned code on some other database uses insert over unindexed table "because driver had no other choice". Thnks... but no, thnks.

turing complete

tialaramex — Mon, 19 Dec 2011 23:37:55 +0000

Accusing people of "trolling" when they explain why you're wrong is poor form.

The general idea you had was fine, there are indeed applications which necessarily involve side effects and the lambda calculus doesn't have side effects. But answering individual HTTP requests (remember PHP isn't a web server, it doesn't need to care about the details of how those requests are made, where they come from, or how answers are returned) isn't really one of those applications. Thus the lack of side effects isn't a deal breaker, and my explanation deliberately didn't use them.

All you need are some conventions. The same sort of conventions you'd need if you wanted to write pow() in the lambda calculus, or isalpha().

It's also fine that you didn't understand that, and needed to have it spelled out. The problem arises when you discover, oops, that you were wrong and rather than say "Ah, I see, that's not what I thought was going on" you insist that you're being trolled.

Cracks in the Foundation (PHP Advent)

adamg — Mon, 19 Dec 2011 22:21:49 +0000

A reasonable database layer allows one to switch database backends as needed. It allows me to change the backend driver (a one-line edit), and everything should still work afterwards.

That always makes me wonder, what kind of argument is that anyway? Do you really find yourself so often in situations where in the middle of the project you decide to switch database backend?

If a project developers write SQL queries that are so portable that it as easy to switch to another database backend as doing the mentioned s/my_/pg_/g (or equivalent), my guess is that they are not using much of the initial database features.

.. or maybe there is an easy way (which I don't know) to write stored procedures in a portable way.

Cracks in the Foundation (PHP Advent)

job — Mon, 19 Dec 2011 22:06:18 +0000

... and code injection has never been as fun :)

(see for example this)

This is the most ridiculous commentary...

job — Mon, 19 Dec 2011 21:58:00 +0000

As a side note, I believe what was important about mod_php was safe_mode. It is a rudimentary file i/o sandbox that only allows you to open files with the same UID your script was created with. That enabled shared script hosting, you don't have to use one process per user which was important for the nascent "web hotel" businesses.

Performance was worse than Perl (which was the dominant language at the time, and had decent template frameworks too), security was worse, the language itself was a stripped down Perl 4, but deployment was so easy you just had to install it and you had yourself a shared web hosting environment.

I see now that safe_mode is removed from PHP 5.4 so perhaps we're full circle now. I wonder what shared hosts are supposed to replace it with?

turing complete

HelloWorld — Mon, 19 Dec 2011 20:56:12 +0000

Sorry, this this pointless, you're clearly confused. Find someplace else to troll.

turing complete

nybble41 — Mon, 19 Dec 2011 20:49:52 +0000

> In PHP, the APIs for accessing the HTTP request data and for setting HTTP response headers and sending data are part of the language; in the untyped lambda calculus, they aren't.

That's just a matter of standard libraries, which can exist in the untyped lambda calculus just as easily as in PHP. These APIs are a convenience, nothing more.

> Of course, you can alter the language specification of the untyped lambda calculus to include side effects by specifying that your program will be passed an HTTP request and its output will be sent to the client. But if you alter the language that way, it's not the untyped lambda calculus any longer.

Neither the PHP language nor the lambda calculus define the semantics of the input and output, which depend on the lower software layers and hardware and are outside the scope of language specifications. Applying an additional semantic layer where none existed before is an extension to the specification, not an alteration to the language. There is no such thing as a meaningful program in either the lambda calculus or more common languages like PHP *without* exactly this sort of semantic extension, which is always defined by the implementation, not the abstract language.

Note that PHP programs can exist which have nothing to do with HTTP; witness the php-cli command-line environment. The HTTP CGI I/O semantics are one way to interact with PHP programs, but not the only way.