The seven deadly sins of software deployment

Posted Aug 8, 2013 22:38 UTC (Thu) by jameslivingston (guest, #57330)
In reply to: The seven deadly sins of software deployment by Cyberax
Parent article: The seven deadly sins of software deployment

Unfortunately there are quite a lot of problems which only appear with more powerful systems.

If production has more CPU cores, then it's much more likely to run into issues that only show up under highly-concurrent load.

If you're using Java or similar applications, GC tuning is highly dependant on the system. Doubling the memory in production could make certain problems disappear, or cause many more. The throughput collectors pause times increase super-linearly, and CMS has many settings that depend on the exact used-memory to available-memory ratio.

Latency of various components will be different, affecting all sorts of timing.

Some of the worst things I've seen:
* Production being split across two datacentres, where staging was only in one (unfortunately quite common)
* Running Solaris on SPARC in production but Linux on x86 in Staging because it's cheaper.
* Running production on physical machines, but staging on virtual machines
* "Staging" being a desktop machine under someone's desk :(

The seven deadly sins of software deployment

Posted Aug 11, 2013 1:24 UTC (Sun) by pr1268 (guest, #24648) [Link] (1 responses)

Just when I thought I'd been cured of deployment nightmares, your last bullet just set me back a ways. ;-)

"Deployment" servers don't belong under someone's desk, because the overnight cleaning crew might unplug it when it makes funny noises. Spoken from personal experience¹.

¹ We were deploying remotely from home at 11 PM, and while the cleaning crew was emptying wastebaskets, right when they got to our sysadmin's cubicle, the computer's disk drives started churning loudly (in an otherwise very quiet office), and perhaps the janitor thought it was possessed or something, so she unplugged it. Needless to say our release had to be rescheduled. Fortunately the sysadmin later convinced the guy holding the purse strings to cough up the $$$ for a real deployment server.

The seven deadly sins of software deployment

Posted Aug 11, 2013 20:40 UTC (Sun) by jberkus (guest, #55561) [Link]

Hah! Don't be surprised if I re-use that anecdote in a future talk ;-)