LWN.net Logo

The birth of the open source enterprise stack

The birth of the open source enterprise stack

Posted Jun 26, 2006 18:32 UTC (Mon) by dmantione (guest, #4640)
Parent article: The birth of the open source enterprise stack

Lets put it blunt:

Now the whole world uses a crap database, which, despite recent
imporvements is still hopelessly behind. The whole world uses a slow,
unelegant language PHP, which is full of pitfalls like SQL and HTML
injection vulnerabilities. Despite improvements over the years, PHP still
sucks.

The result is that there are many, many software packages out there, that
are not as good aso they could be. (phpBB security leaks anyone?)

So, what should we be proud of? An insecure, unreliable web?

Seriously, there were better to much alternatives available at the time,
(PostgreSQL, AOLserver) which became niches, of which one reason was the
LAMP marketing.

So, LAMP a success? It is a matter of taste.


(Log in to post comments)

The birth of the open source enterprise stack

Posted Jun 26, 2006 19:38 UTC (Mon) by dskoll (subscriber, #1630) [Link]

I agree. PHP and MySQL suck. We use them both, unfortunately, because there are some very useful apps that demand them (eg SugarCRM). However, our database of choice is PostgreSQL, and we're moving to Perl for most product development. (Yeah, you can bash Perl, but it's a model of orthogonality compared to PHP.)

The birth of the open source enterprise stack

Posted Jun 30, 2006 7:13 UTC (Fri) by tekNico (subscriber, #22) [Link]

> Yeah, you can bash Perl

Ok, I'll take it: I bash Perl. What next, you'll ask us to dash Ruby, or even (shudder) ash Python? ;-)

The birth of the open source enterprise stack

Posted Jun 26, 2006 20:54 UTC (Mon) by jeroen (subscriber, #12372) [Link]

As Jeff Waugh said in his closing talk at FOSDEM, LAMP means Linux, Apache, Most of our scripting languages start with P, PostgreSQL. :-)

The birth of the open source enterprise stack

Posted Jun 26, 2006 21:53 UTC (Mon) by tjc (subscriber, #137) [Link]

Seriously, there were better to much alternatives available at the time, (PostgreSQL, AOLserver) which became niches, of which one reason was the LAMP marketing.
I don't recall that LAMP had much in the way of marketing (other than good press), but I might not have been paying attention.

One thing that PHP and MySQL do have in common is good documentation, and they are both easy to install, configure, and use, which are things that other projects often overlook or put off, sometime indefinitely.

The birth of the open source enterprise stack

Posted Jun 27, 2006 7:33 UTC (Tue) by nix (subscriber, #2304) [Link]

PostgreSQL's documentation isn't half bad either, of course, unless the several thousand pages of the docs are purely an illusion :)

The birth of the open source enterprise stack

Posted Jun 27, 2006 14:22 UTC (Tue) by tjc (subscriber, #137) [Link]

It's not the amount of documentation that's important, it's the how well it's written, and -- especially important -- how well it's indexed (or searchable -- I'm a big fan of "on one page" manuals).

I can't comment on PostgreSQL's current state of documentation, but five years ago when I was doing a lot of internet programming I found MySQL and PHP both to have decent documentation, which is one of the major reaons I used them. I was able to learn enough PHP in a few days to get started, owing mostly to it's C-like syntax and comprehensive online function index, and MySQL was greatly aided by Paul DuBois' excellent book on the subject.

At the time PostgreSQL was complicated to configure, and Phython didn't look much like C. If I were doing internet programming today (and had more time), I might choose differently.

The birth of the open source enterprise stack

Posted Jun 29, 2006 6:29 UTC (Thu) by nix (subscriber, #2304) [Link]

PostgreSQL still is complex to configure, but only a very small amount needs to be done unless you're running a large site (in which case MySQL would need configuration too, if it coped at all).

The default config values for some things (shared memory size, notably) are laughably small: suitable for testing only. I don't know why they start out *so* small, but it's easy to change them.

The birth of the open source enterprise stack

Posted Jun 27, 2006 19:57 UTC (Tue) by hingo (subscriber, #14792) [Link]

I think you forget the most important reason. MySQL was available for Win32, while PostgreSQL was not (until now). Remember that only a few years ago 99% of students learning to do websites had modem connection to the internet, and 99% of them did not use Linux on their desktop computer.

I honestly think this is the one single reason MySQL beat PGSQL. Those kids simply never got to even try PGSQL. I also think this same issue has played a major role in the popularity of many Linux desktop apps, and how ironic is that!

The birth of the open source enterprise stack

Posted Jun 27, 2006 21:04 UTC (Tue) by tjc (subscriber, #137) [Link]

Remember that only a few years ago 99% of students learning to do websites had modem connection to the internet, and 99% of them did not use Linux on their desktop computer.
I'm not so sure about that. Most universities use UNIX or Linux in the student labs. Maybe some community colleges and trade schools use Windows, and self-taught people learning at home, but I wouldn't think that it would be anywhere near 99%.

The birth of the open source enterprise stack

Posted Jun 30, 2006 7:44 UTC (Fri) by grouch (subscriber, #27289) [Link]

"One thing that PHP and MySQL do have in common is good documentation [...]"

I would agree that PHP has good documentation, but MySQL's is atrocious. The comparison of documentation between MySQL and PostgreSQL alone is sufficient to use and recommend PostgreSQL.

The birth of the open source enterprise stack

Posted Jun 26, 2006 22:33 UTC (Mon) by davidw (subscriber, #947) [Link]

Marketing? Yes, but not quite like you say.

I'm a big Tcl fan (see http://antirez.com/articoli/tclmisunderstood.html for those of you who aren't familiar with the language, or formed your ideas about it in 1996), but I'm going to have to say that rather than a LAMP success, Tcl really fell down on the marketing front a few times:

*) John Ousterhout leaving the helm created lots of confusion. There was no longer a focal point for development, a Larry Wall, a Matz, someone who was the face of Tcl.

*) AOLserver was late to the open source game and didn't "play nicely" with what was already *the* web server. When I wrote mod_dtcl (subsequently Apache Rivet) it certainly wasnt the first server side Tcl system, but I think it was the first open source one, and this was in 1998, when PHP was already starting to be used...

As far as Mysql is concerned, I guess it's just a case of worse is better - "oh, I don't care about data integrity, I just need *fast*" - followed by "well, now we need to add payments, and mysql hasn't really given us trouble so far, and I guess we can handle transactions in PHP...".

The birth of the open source enterprise stack

Posted Jun 26, 2006 23:08 UTC (Mon) by khim (subscriber, #9252) [Link]

Don't know about PHP but for MySQL... they are mostly right. I've seen huge systems based on MySQL which are handling billions of $$ every year - and they do work. IMO it's mostly matter of programming culture and less matter of pure SQL-capability of the system used. While MYSQL is not perfect (and neither is PostgreSQL) it does scale, you can make it work - may be not as easily as PostgreSQL, but it can be done: I've seen it done.

PHP... I'm yet to see big, complex, reliable system with heavy use of PHP. Sometimes PHP is used "on the side", but when system becomes complex and PHP is in the middle of it... it's inevitable disaster...

P.S. I've seen huge systems written in Python, Java (of course) and even Ruby (hmm, it works) or Perl (horrors), but PHP... nope. Don't really know why... But may be it's the attitude ? PHP is the only language where "security update" can break half of the scripts on your site and when the question will be raised in the mailing list the answer will be "oh, these scripts where never written to the specifications, so it's Ok to break them - you just need to fix them".

The birth of the open source enterprise stack

Posted Jun 27, 2006 7:44 UTC (Tue) by nix (subscriber, #2304) [Link]

In any case, it's amazing how often huge systems can get away without using transactions, as long as their load is comparatively low with respect to system speed and the things they connect to are reliable enough that they don't need to roll back often.

One of my least finest hours was a day in 1999 when I broke the transaction management in a (large!) financial application by typoing in a hook and making all rollbacks throughout that application into commits instead. Nobody noticed for *six months*, until a major stockmarket feed went down. (Then all our customers tried to roll back at once...)

(And as for isolability, we all know how good certain major databases are at *that*. The PostgreSQL manual makes the point that perfect isolation isn't possible without giving the database what amounts to a theorem prover *and* complete knowledge of your app's control flow: but most databases, including major expensive ones with clowns with E in their surnames as CEOs of their controlling company, don't even try: you have to *ask* for half-decent isolation, and when you do your transaction goes read-only! PostgreSQL never had *that* problem, but it doesn't seem to stop people using said major database...)

The birth of the open source enterprise stack

Posted Jun 30, 2006 6:43 UTC (Fri) by oak (guest, #2786) [Link]

> One of my least finest hours was a day in 1999 when I broke the
> transaction management in a (large!) financial application by typoing
> in a hook and making all rollbacks throughout that application into
> commits instead. Nobody noticed for *six months*, until a major
> stockmarket feed went down.

This seems an important functionality which should have had a test case
(preferably written before the functionality was fully implemented)
which tests both conditions where rollbacks should happen and where not.

How easy it's to write test-cases for LAMP (and particularly PHP)
software? Is there some test-suite that could be used? I would think
the end user UI tests (needing to use "HTTP API") to be particularly
onerous, but at least LAMP would be good for storing and visualizing
the test results. ;-)

The birth of the open source enterprise stack

Posted Jun 30, 2006 17:06 UTC (Fri) by nix (subscriber, #2304) [Link]

Yes, it should have had a testcase. Alas the environment in question had such an insane build system that I hadn't at the time written a testsuite framework for it. (I wrote one immediately after this, ahem, incident: I only need my face rubbing in the bleeding obvious *once* ;) )

There are several test frameworks for webbish stuff: Cactus (part of Apache Jakarta) is half of what's needed. I don't have much experience with any of them (due to violent allergies to HTML and webbish stuff in general).

The birth of the open source enterprise stack

Posted Jul 5, 2006 19:48 UTC (Wed) by pimlott (guest, #1535) [Link]

it's amazing how often huge systems can get away without using transactions
It is amazing, but it doesn't make it right! When something finally does go wrong, it's extremely hard to debug and fix. Yes, I know, sometimes the cost of cleaning up the occasional mess is lower than the cost of doing it right. But given the ubiquity of DBMSs with transactions, it's hard to see why you'd risk it.

The PostgreSQL manual makes the point that perfect isolation isn't possible without giving the database what amounts to a theorem prover *and* complete knowledge of your app's control flow
I'm curious what you're talking about, because I can't find anything like that in the PostgreSQL manual. I think perfect isolation is straightforward using read and write logs, and that PostgreSQL does exactly that. What am I missing?

that perfect isolation isn't possible without giving the database what amounts to a theorem prover *and* complete knowledge of your app's control flow: but most databases, including major expensive ones with clowns with E in their surnames as CEOs of their controlling company, don't even try: you have to *ask* for half-decent isolation, and when you do your transaction goes read-only!
I agree that they don't even try. It's a travesty that the isolation level required for reliability in most applications--that the SQL standard says must be the default--is not the default in any DBMS I know. And when you do turn it on, many systems perform badly; PostgreSQL, with multi-version concurrency control, does quite well. But what do you mean about transactions going read-only?

The birth of the open source enterprise stack

Posted Jul 7, 2006 0:38 UTC (Fri) by nix (subscriber, #2304) [Link]

The PostgreSQL manual makes the point that perfect isolation isn't possible without giving the database what amounts to a theorem prover *and* complete knowledge of your app's control flow
I'm curious what you're talking about, because I can't find anything like that in the PostgreSQL manual. I think perfect isolation is straightforward using read and write logs, and that PostgreSQL does exactly that. What am I missing?
See section 12.2.2.1 of the PostgreSQL manual, and the discussion of predicate locking. (Note, when it says `the details of every query', it means the details. In the presence of random-access cursors I suspect this reduces to solving the halting problem.)
But what do you mean about transactions going read-only?
If you turn on serializable isolation in some transaction in said expensive proprietary RDBMS, you can no longer carry out INSERTs or UPDATEs in that transaction. It makes it really very useful :/

The birth of the open source enterprise stack

Posted Jul 7, 2006 1:24 UTC (Fri) by pimlott (guest, #1535) [Link]

See section 12.2.2.1 of the PostgreSQL manual, and the discussion of predicate locking.
Thank you, I am enlightened. I guess this could be solved by logging at the table level: when the where clause is non-trivial, mark the entire table read, and mark the table written on insert. Then, when you commit, you would see (in one of the transactions) that the table has been modified since you queried it.
If you turn on serializable isolation in some transaction in said expensive proprietary RDBMS, you can no longer carry out INSERTs or UPDATEs in that transaction.
Hmm... I've used serializable in (if memory serves) Informix, Oracle, and Sybase, reading and writing. Have I not gone expensive enough?

The birth of the open source enterprise stack

Posted Jul 8, 2006 22:58 UTC (Sat) by nix (subscriber, #2304) [Link]

My experience of Oracle is that at least in some releases, it had (and was documented as having!) the nasty read-only semantics I described :(

The birth of the open source enterprise stack

Posted Jun 27, 2006 15:48 UTC (Tue) by dmantione (guest, #4640) [Link]

The fact that MySQL powers big $$ systems does mean it should power those
systems. I also still need to meet the first person that concludes
PostgreSQL is too slow for his application. In fact, I have yet to see
the first installation where MySQL was consiously chosen after comparison
to other databases.

The birth of the open source enterprise stack

Posted Jun 29, 2006 6:01 UTC (Thu) by jwb (subscriber, #15467) [Link]

Well, I'd be happy to introduce you to some of these applications. For an example, I have written a system which caches the information about the last trade of hundreds of thousands of stocks. During active trading, this information comes at the rate of thousands of updates per second. The amount of information to be stored is basically fixed, as there are a fixed number of stocks to be traded. In MySQL, the database size stays nice and constant, I get atomic updates, and I also get a transaction rate up to 10k/sec on hardware costing less than $2000.

If I were to use PostgreSQL for this load, I would end up with a huge, ever-growing file, because updates are handled like inserts. I would have an always-running VACUUM process, but the file would probably grow slightly at peak times anyway. I would likely have unbounded index growth, as VACUUM does not reclaim index space (only REINDEX does this, and it requires exclusive access in some versions). In all likelihood, I would need a nightly maintenance period, which I can't afford because there are stock markets all over the planet.

So there's an example of a load where you'd better use MySQL.

The birth of the open source enterprise stack

Posted Jun 29, 2006 16:08 UTC (Thu) by dskoll (subscriber, #1630) [Link]

If I were to use PostgreSQL for this load, I would end up with a huge, ever-growing file, because updates are handled like inserts.

Correct, sort-of. VACUUM would reclaim the space.

I would likely have unbounded index growth, as VACUUM does not reclaim index space

This is no longer true. Since 8.0 (and possibly since 7.4), VACUUM reclaims index space.

The birth of the open source enterprise stack

Posted Jun 30, 2006 17:07 UTC (Fri) by nix (subscriber, #2304) [Link]

... and therefore so does the autovacuum daemon in 8.1+.

The birth of the open source enterprise stack

Posted Jul 4, 2006 7:57 UTC (Tue) by nlucas (subscriber, #33793) [Link]

You might want to check SQLite for this kind of applications.
No server setup as it's only a library using a single file as a database, but *very* fast (but ACID by default, so you need to know how to make it fast).

It's not good for multiple simultaneous writers, because it locks the entire database on a write, but most applications only write once and read many.

The birth of the open source enterprise stack

Posted Jun 29, 2006 13:41 UTC (Thu) by jkouyoumjian (guest, #34201) [Link]

Wow. Thank you for that wisdom.

Microsoft, Sun, Oracle, SAP and the rest are slow, insecure and suck too, especially if you configure them wrong and don't understand what they were designed to do.

The difference is that I would rather not pay for my sucky software.

It is possible to write good, secure and fast applications with any of these tools. They don't have to suck. Unfortunately, things don't “just work”. You have to understand how to use them.

Copyright © 2008, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds