| |||||||||||||
Scale Fail (part 2) Hardware often works ...Scale Fail (part 2) Hardware often works ...Posted May 20, 2011 14:58 UTC (Fri) by mrjk (subscriber, #48482)Parent article: Scale Fail (part 2)
I used to buy hardware at the level (or above) that is kind of derided in this article, and I'll defend it. Just throwing in a bunch of hardware works a tremendous amount of the time for several reasons.
First it can be capitalized with no arguments so is moved off the books for expenses. You can do this with software and consulting too, but not without wrangling. This isn't a technical but it is a real organizational issue.
Second it is mostly immune to staff changes. If you actually apply thought and put in a smart caching system and perfect redundancy, when you leave and your wonderful documentation leaves with you or some background knowledge is lost, I as your thickheaded successor will have gaps in my knowledge of the system. They will bite back at some point. That is how single points of failure creep in over time. With just a cursory look at some of the most obvious issues many many applications will live in a big chunk of hardware, if the hardware is 90% unused, so what? It will likely be less overall cost, because the unthinking apps just roll along without all the manpower needed to maintain them.
Third it actually makes the IT department more resilient. They are always putting in some new server or moving applications to a new one. This to me, is really more important to disaster recovery than staged tests. The Systems folk and the DBA's know directly what is going on and where stuff is. This also gives slack for growth and the ability to ride the technology wave forward.
The fact that systems might be more standard (except in areas directly worked on) and not smart saves a bunch of time and when you do get the boss with the bright idea, it isn't going to crush what you have.
The thing is, the cost of computing power has crashed and killed companies (hi Sun ...) over the last 20 years. This goes back decades. I remember looking at our major database server footprint around 2005 and realizing it was 20 times bigger and ten times cheaper than it was in 1994 ... When that is true and consultants are no cheaper ... I think the "dumb" throw hardware idea made and still makes a lot of sense.
Not that bad settings shouldn't be corrected and decent modeling of scale and flow shouldn't be done!
(Log in to post comments)
Scale Fail (part 2) Hardware often works ... Posted May 20, 2011 15:43 UTC (Fri) by jberkus (subscriber, #55561) [Link]
Mrjk,
First, you make a lot of good points.
The problems with the "just throw hardware at it" solution are two-fold:
1. The cost balance is often extremely disproportionate. That is, it's frequently the case that $20,000 worth of smarter software will save you $200,000 worth of additional hardware, rack space, cooling and sysadmin time.
2. In the fairly common cases where dumb software consumes geometrically increasing quantities of hardware for a linearly increasing workload, the "more hardware" solution is a very temporary measure.
Obviously there are cases where the tradeoff of "let's just buy more hardware" is completely viable, and I've implemented a few. But that only works if you're making an informed tradeoff, where you actually calculate the costs and capabilities of each path.
Scale Fail (part 2) Hardware often works ... Posted May 22, 2011 3:56 UTC (Sun) by willy (subscriber, #9762) [Link]
One extreme example (in the opposite direction :-)
http://thedailywtf.com/Articles/That-Wouldve-Been-an-Opti...
Scale Fail (part 2) Hardware often works ... Posted May 26, 2011 1:30 UTC (Thu) by adavid (subscriber, #42044) [Link]
Twice in my career as a sysadmin I have pushed back on applications that wanted to spend more money on hardware to rush a project that had major memory problems. One project saved one hundred thousand dollars and the other saved nearly a million dollars by tracking and fixing memory leaks.
What it means to capitalize something Posted May 20, 2011 18:51 UTC (Fri) by jhhaller (subscriber, #56103) [Link]
Many people don't understand what it means to have a capital asset. Yes, capitalizing something does take much of the expense off the books. But it affects cash flow, as you still have to pay for the equipment when you get it. On top of that, a portion of the capitalization comes back as expense every fiscal period, as the purchase still has to come off revenue eventually. This shows up on budgets as depreciation.
Think of buying something on capital as like buying a car. There are payments every month, and your outstanding loan amount counts against your available credit. You still have to pay off the car, but not all at once. Capitalization is slightly different in that there is no one loaning the money up front, but to an organizational budget, it looks more like the loan, while corporate worries about where the money comes from, and if they can meet the payroll next month.
With capitalization, it's quite easy to get oneself into a bind, where last year you bought a huge amount of equipment, this year you fixed the software bottleneck which makes the extra equipment no longer necessary. But, your organization will be paying the depreciation on that now useless equipment until its fully depreciated or you sell it. The first organization I worked at had a seven year depreciation schedule, and everything was worthless by year four. We eventually started buying everything on expense accounts, as it allowed us to adjust to different staff and work levels. But, we were stuck with that depreciation for quite some time.
What it means to capitalize something Posted May 30, 2011 0:43 UTC (Mon) by giraffedata (subscriber, #1954) [Link] With capitalization, it's quite easy to get oneself into a bind, where last year you bought a huge amount of equipment, this year you fixed the software bottleneck which makes the extra equipment no longer necessary. But, your organization will be paying the depreciation on that now useless equipment until its fully depreciated or you sell it. And that's the whole reason buying stuff with capital dollars is often better than buying stuff for which you have to use expense dollars. You certainly won't hold onto that equipment you're not using anymore and keep paying for the depreciation. You'll sell it and get back some of what you originally spent. But if you had originally spent money on consulting instead of hardware (and the consulting wasn't capitalizable), you're stuck. The money is gone forever. That's why Management is more willing to authorize hardware purchase than consulting. Obviously, you can do the accounting incorrectly and make it look like some decision is better when it's not -- for example, depreciating equipment over 7 years when you know it will be worthless in 4.
What it means to capitalize something Posted May 30, 2011 10:41 UTC (Mon) by nix (subscriber, #2304) [Link] That's why Management is more willing to authorize hardware purchase than consulting.Not anywhere I've ever worked (though perhaps this is because ostentatiously pointless expenditure is perhaps *the* way to retain empires in the City).
Scale Fail (part 2) Hardware often works ... Posted May 29, 2011 14:53 UTC (Sun) by hein.zelle (guest, #33324) [Link]
Some good points in there, although coming from a company which has long suffered from your proposed approach, there's some important cons to consider too:
> Third it actually makes the IT department more resilient. They are
In our case, exactly the opposite. We had one (old/ancient) machine per web service, where we should have instead been using one virtual machine on a properly maintained piece of hardware. IT didn't know a single thing about these machines, because they'd been running so long that noone dared to touch them. Hardware failure equaled disaster, almost by definition.
Once we switched to a cluster for computations and virtual machines for web services, IT has finally become able to deal with all these machines and services.
> This also gives slack for growth and the ability to ride the
That also worked exactly the wrong way around, as we were ending up with ancient machines. It's still running - why replace it?
Our management used to think buying hardware was more cost-efficient. If you add up the required hardware support contracts, installation and configuration costs though, it turned out rather badly. We now clone machines for expansion regularly, either in a VM or as part of a cluster. Expanding has become a matter of hours instead of weeks.
I suppose you could argue that now we've finally caught up with some state of the art systems (cluster / virtualization), hardware expansion will actually work well again. I've yet to see how that will work out in the future. The first proponents of installing cheap, single machines for a single service are already coming up again.
|
|||||||||||||