You're running a bank and want to debit $10 from one account and credit $10 to another account. You want it all to happen or none to happen. (Atomicity)
You don't want your "end-of-month" summary report to be off by $10 when that money just happened to be "in transit" when the report was run. Nobody should see the in-between "bad" states. (Consistency, Isolation)
Even more important, you don't want a server failure (crash, power off) to *ever* leave things in that intermediate state permanently. (Durability)
ACID databases can't do much in parallel because it must always think about the strict ordering of transactions.
On the other hand, if you're running a web forum, maybe you're willing to live with the possibility of loosing a few messages (on server failure) or allowing new posts in a deleted forum (for a few seconds) in exchange for scaling 100x better.
(I predict that non-relational/non-ACID will become the dominant form of databases -- because very few things actually need all properties of ACID.)