Fedora harnesses the power of idle computers with Nightlife

Bryan Che, a member of the product management team at Red Hat, recently introduced Fedora Nightlife, a project he hopes will motivate people to donate their computer's downtime to processing data for scientific research and other socially beneficial work. The heavy lifting will be done by the University of Wisconsin-Madison's Condor workload management system which will be responsible for the scheduling and logistics of donated computer power and, in the end, Che hopes to build a network of more than a million nodes of Fedora systems to help process data for everything from Web-indexing projects to medical research.

"[W]e have begun talking with the guys over at Wikia about helping them index the Web for their open source search engine," says Che. "It would be great if we could help with tasks for the Fedora infrastructure team at some point with things like automated builds or tests. There is a lot of scientific research that requires lots of computing power, and there are lots of students who could use access to a grid for research. I'd love to have all sorts of projects like these participate."

Che says that the scope and type of projects that join will largely be dictated by the community, and he's hoping to draw on its collective expertise to "shape Nightlife into a useful community service." His end goal, however, isn't just to make computer resources available but to also develop a basis for larger infrastructure projects. Che notes, "For example, much of the high performance computing (HPC) jobs these days are done on Linux — and particularly Fedora or Red Hat. This puts us in a prime position to be able to shape and build out an entire open source stack for research computing on grids. Today, many people depend upon proprietary (and often costly) libraries for their scientific research or even enterprise computing. Nightlife will provide us a great forum to engage these users to see what are their needs and provide them with a fully open source solution that they can use for their valuable research."

Naturally, security is of primary importance when individual computers are clustered together or outside data is inserted into a system for processing. Che says the Nightlife team takes security very seriously and has a number of measures in place to protect users' computers and ensure the application code is safe as well.

"[W]e will require that projects that want to leverage Nightlife must distribute their packages and source code through Fedora," explains Che. "This will allow us to inspect what the applications are doing and make sure there isn't anything malicious. On the execution side, one of the capabilities that we've added to Condor recently is integration with our libvirt virtualization technology. This will enable people to execute Nightlife jobs entirely within a virtual machine bubble that is shielded from their physical computers.

"We are also looking at taking advantage of SELinux technology, which we've developed with the NSA, as a mechanism for tightly locking down jobs so that they can only perform tasks for which they are explicitly granted permission."

Che is quick to point out that although Fedora has committed plenty of resources to Nightlife, it is not Fedora-specific — indeed it's not even Linux-specific. Since Condor supports executing processes on many different platforms, Mac OS, Windows, Unix, and Linux distributions of any flavor are capable of donating resources. Not all features will be available on non-Linux platforms, however, if they lack certain underlying technologies. For instance, Windows lacks a built-in hypervisor for running virtual environments and doesn't support SELinux for lock-downs.

"I would welcome anyone to donate spare capacity to Nightlife [and] I'd hope that people from all sorts of platforms join us," encourages Che. "[T]here isn't any reason why other communities couldn't participate with us and even start adding some of these capabilities to a Nightlife client for their platforms. From a development standpoint, the upstream code lives in the Condor project at the University of Wisconsin. So, anyone can contribute at that project as well without having any involvement with Fedora."

When the project was announced last week, some community members were puzzled as to why Fedora chose to use Condor instead of BOINC, a similar project developed by University of California-Berkeley. Che points out that, though the two efforts have a lot in common, they each have an entirely different focus. He says BOINC's mission is "very much focused on enabling desktops/laptops to provide computing capacity as part of a larger grid [while] Condor is more general-purpose; it can take idle capacity and utilize it well, but it is primarily a good resource scheduler for dedicated grids."

While some people's comparisons of Condor and BOINC focus on the technology behind the projects, others see similarities between the Condor and Nightlife projects themselves. In actuality, they are really quite different. "Condor's client can use a BOINC client to process data as backfill (when there are no other jobs to run)," notes Che. "So, there is no need to view these projects as competitive. Indeed, one possibility is to use Nightlife to increase the number of machines participating in BOINC." Of course, a low barrier to entry is also important for widespread adoption of Nightlife. Since many enterprises and researchers already run Condor for their dedicated grids, Che says it was a logical choice for the project.

Dr. Keith Laidig can easily see the intrinsic value of Nightlife and how it will benefit the scientific community at large. He runs the computing infrastructure for the computational biophysics group in the Department of Biochemistry at University of Washington, and regularly relies on outside computing power to crunch data for researchers. Under the direction of Professor David Baker, about four years ago the group created Robetta, an automated prediction server that farms out work to other systems via Condor which has proven "quite successful at keeping the wait times [for research results] down to the range of 'months'."

Laidig recently told the Nightlife community, "If we had access to more computing power, even that available from modest periods of inactivity, we could put that power to work to address many pressing issues in bio-medical research such as HIV/AIDS vaccine design, improvement of existing drugs and/or design new drugs, and creation of new methods to harness biology to address issues such as carbon sequestration."

As Laidig explained to LWN, reducing the wait times for results to even a matter of weeks is not out of the question. "Given sufficient computing power, the processing time would drop even further. In principle, the processing could take a day or less — depending on computing power, queue depth, etc."

Laidig says it's hard to estimate just how much donated computer access his lab would need in order to see an appreciable rise in research turn-around time, but he estimates they currently use around 300 - 400 processors running around the clock to maintain the current work flow. "Should we gain, say, 1,500 machines that could work for 8 hours... we'd be matching that — taking into account overhead. Now, I'd like to increase that by a factor of ten or more."

Though he would be happy to see Nightlife flourish, Laidig notes there are some things to consider before committing your computer's resources to the project. "Not to throw a wet blanket on things, but [there are] issues that folks should keep in mind. Their gear would be using electricity and generating heat. There are also network bandwidth considerations as well — some data-sets necessary to undertake distributed work can be sizable (100 MBs) which can soak up resources. There's the local disk space usage, too.

"Folks should be made aware of the 'costs' of contributing. Then, should their desire to contribute outweigh the costs, they should join up!"

Some community members have indeed expressed concerns about the energy consumption associated with idling computers and suggest that the ecological harm of running the CPUs and fans of an unattended machine outweighs the benefit of charity in the name of science. In response to an animated discussion about Nightlife at Slashdot, one enterprising commenter tested how much energy his idle computer uses and discovered it was upwards of $70 per year. Che responded to the criticism by acknowledging that although cycle harvesting can be viewed as a "waste of energy," it can, in fact, save energy in the long run. In addition to the notion that energy to process data will eventually be used at some point or another anyway, Nightlife also distributes energy consumption over a wide geographical area, thereby reducing the overall energy burden on a single data center or location.

Future plans for Nightlife include making it a first-boot option for Fedora so when a user does a fresh install, they are prompted to donate computer power to the project. Of course, before Che can attain his million-node goal, there are several smaller goals to accomplish along the way. "At the earliest, we wouldn't be able to start reaching numbers at this level until after Fedora 10 — and that's probably pushing it."

One way to run a project like this and not be wasting power would be to run it during the
winter.  If your dwelling uses any electric heat, then running computer systems is better than
running a heater.

All power use returns as heat in the end, so there is no better way to get that heat than to
do computing with it, instead of wasting that potential by just driving the power through a
resistive heating coil!

Still a few caveats...

I used to think this way but then I spent more time pondering it. If you do have electric
heating it will usually be optimised to heat spaces where people live and work. Heating any
part of the house when it's empty (ignoring frost protection), or the closet with a household
server in it, or a spare bedroom, or even a living room when everyone's asleep upstairs, is
still mostly useless because no people benefit.

It's true in principle that you can find situations where the power isn't "wasted" by turning
it into heat, but they're rarer than might first appear. Maybe $100 spent running a PC
translates to $10 of "free" heating for some people, and even more for a few, but that's still
a poor reason to leave all that hardware running when energy prices are rising.

I don't know about you, but in the UK we have a wide choice of heating: we 
can heat the whole house on a thermostat or we can heat none of it at all.

So the `heating unoccupied rooms' thing is irrelevant at best, because the 
standard central heating heats *all* rooms, including all the unoccupied 
ones.

whilst systems this dumb are indeed very common, things are steadily improving as people get
programmable room stats, TRVs, boiler managers, and even fancy linux-controlled home-autoation
systems. As eneergy prices rise the benefits of having a more flexible control system
increase.

I agree with your fundamental point that exactly how usefully your computers do or do not
contribute to house heating depends on the control system installed. 

Central heating in towns and cities in the UK is pretty universally 
natural gas-based. Outlying regions might use oil-based heating, storage 
heaters, or stranger systems, and places with broken or very old central 
heating or bad insulation might choose to stick electrical heaters in some 
rooms. Pure house-wide electricity-based systems are unheard of (by me at 
least): even heating your water with electricity is an emergency fallback 
for when the gas or boiler goes out. (It's also pretty much a historical 
curiosity: in thirty years I've never seen a built-in electrical immersion 
heater used, but they're still widely fitted).

So fine, I live in the UK too. If you have a whole house on a single thermostat that usually
means you're using gas central heating. In which case heating your house by leaving a PC
turned on is already horribly inefficient because of the price difference between gas and
electricity. You also live in a temperate country, where the house doesn't need to be heated
for most of the year (and indeed may be uncomfortably hot for a month or two each summer).

Any radiators installed since the popularity of home central heating in the UK really began
will have bypass valves so that you can disable radiators in unused parts of the house (e.g. a
spare bedroom) or add a cheap local thermostat (included in newer installations) which
bypasses when that room is above a certain temperature. Most installers will skip rooms that
are rarely occupied and can be heated by conduction or convection from elsewhere, such as
closets.

Any installation that's less than 30 years old will have a timer as well as the thermostat and
manual control, and newer ones will have a seven day variable timer. Typically this means you
only heat the house for a few hours every day, usually when you wake up (it's not nice to wake
in a cold house, and the timer may also control production of stored hot water for the
bathroom) and for a while in the afternoon or evening. There's no need to heat the house while
you're asleep, you will be comfortable at a lower temperature and the bedding insulates you
anyway.

The idea that all heat energy released by inefficient use is "free heating" in some way just
doesn't work out in reality. Unless you've got an electric element or fan heater next to the
PC that you leave switched on all the time, chances are that turning the PC off is a
significant net saving (I tend to work to £1 per watt as a rule of thumb).

Sorry, that should be £1 per watt _per annum_ of course, ie over a year's usage switching off
something that wastes 100 watts saves you about £100. If it wastes 100 watts for 6 hours per
day, that's £25. It's a rule of thumb, so your specific tariffs may be rather different
depending on the mixture of fixed versus variable costs, but probably not by an order of
magnitude.

The last time I checked (a year or two ago), Condor wasn't open source. I'm surprised RH would
push it, if that's still the case.

Looks like as of 6.9.5 its released under the Apache license.

http://www.cs.wisc.edu/condor/license.html

Who do you think convinced them to open-source it?

I understand your implication that it was Red Hat. Is that in fact the truth, or are you just
suggesting it's likely? My recollection is that they said they intended to open the source at
some point, but the build was extremely messy.

http://www.redhat.com/mrg/grid/condor/

Excellent, thanks for taking the time to point that out!

Folks, I suggest that people start turning off their computers during night. I know it is cool
to have a system running for 523 days without reboot, but just think how much energy you can
save in those days as well.