Backing up data stored on computers is one of the most important jobs of a systems administrator. A regular backup routine can save large amounts of heartache and frustration when a disk drive or system fails. Disk failure should be treated as something that is guaranteed to happen in the life (and death) of every disk drive. And, disk failures always seem to happen at the worst possible moment. Typical failures happen on Friday afternoon before a vacation is about to start or when the boss comes into your office demanding that critical report that lives on the machine with the smoke curling out of the power supply.
Over the years, your author has lived through many backup technologies. In the early days of home computing and CP/M systems, floppy to floppy transfer was the only method to save data. Floppies were unreliable and multiple copies were important. When hard drives became normal hardware on DOS-based microprocessor systems, backups were performed on piles of floppy disks or short-lived tape technologies. It was a bad day when floppy disk 29 of a batch of 30 encountered a read error during the restoration of a disk.
Mainframe systems in the early 1980s required copying the contents of washing machine sized disk drives to piles of 9 track open reel tapes. As drives were added, the piles of tapes became larger. Large storage areas were required for storing racks of tapes. The 1990s brought larger disk drives and the capabilities of the mainframes and PCs were converging. Single filesystems could be copied to DC100 serpentine tape cartridges, if one had a lot patience. Helical scan SCSI tape drives such as Exabyte 8mm and DDS 4mm were able to store the contents of multiple filesystems on one tape. For a brief while, tape capacity surpassed filesystem size. Robotic tape library machines could be programmed to automate the backup process and allow large numbers of filesystems to be copied to stacks of tapes.
Disk capacities continued to expand rapidly. AIT tapes were good for larger backups, but the media was pricey. RAID arrays became a good way to increase storage capacity and improve reliability, but downtime could be long in the event of a controller failure. RAID backups are still critically important. Disk drive prices continued to fall. At some point after the year 2000, the price/performance of disks versus tapes made it more economical to buy another disk drive to copy data to. For the moment, it appears that the disk/tape competition is over and disks won.
With a removable drive sled or a USB drive, a hard drive can now be treated as a high speed random access data cartridge. With multiple online machines, it is possible to use one machine as a backup repository for another's data. Today, it is possible to buy a 300GB disk drive for just over $100. Larger drives can be had for a slightly higher cost per byte.
On a Linux platform, two of the oldest and most common backup utilities are dump and tar. Both work with tape and disk-based archives. Dump has the advantage of being able to dig through the contents of an archive and pick individual files to restore before reading the entire media. Unfortunately, the dump archive format has gone through a lot of changes. This means, for example, that a dump archive that was created on a Red Hat 7.3 system may be unreadable on a Ubuntu 7.04 system. Reading old tar files is more likely to result in success across systems of different vintages. Your author decided to standardize on tar-based backups.
Now for some current real-world examples for performing disk-to-disk backups: Here's how to use dump to copy the local / filesystem to a compressed and datestamped file on the same machine's /backup filesystem:
cd /backup /sbin/dump 0ufa - / | bzip2 > ./localslash`date +%Y%m%d`.bz2Here's how to use tar to do the same type of local to local backup:
cd /backup /bin/tar cf - / | bzip2' > ./localslash`date +%Y%m%d`.tar.bz2Here's how dump is used to backup the / filesystem on a machine called remote to the local machine's /backup partition:
cd /backup ssh remote '/sbin/dump 0ufa - / | bzip2' > ./remoteslash`date +%Y%m%d`.bz2Here's how to use tar to do the remote to local backup:
cd /backup ssh remote '/bin/tar cf - / | bzip2' > ./remoteslash`date +%Y%m%d`.tar.bz2The above commands should be run from the root account and the remote backups can work without passwords if ssh is setup correctly. Ssh and the ssh server should be installed and configured on the machines. The dump and/or tar manual pages should be consulted for more information on the various command options. Restoration of the filesystems involves using bunzip2 to uncompress the archive, then restore (for dump) or tar (for tar) to split out the contents to a local disk. Restoration across the network is possible with the use of ssh.
A good backup scheme should be devised. Your author has a dedicated machine with a large disk drive and an old DDS3 tape drive that is used to backup all of the rest of his machines to. Variations on the above examples are used in several machine-specific scripts to backup one machine at a time. The backups are performed several times a month. Backups can be copied from the backup machine's disk to tape for offsite storage. The entire backup set is occasionally copied to another machine's large disk for redundancy. Datasets can simply be copied with cp to removable media. A 100GB+ audio archive is managed differently than standard filesystems, the rsync command is used to clone the data from one machine to another.
In the early 1990s, your author couldn't imagine ever getting close to filling up a 9GB disk drive. Then came audio archives, digital cameras with movie modes and other large data sources. Several hard drive failures and machine meltdowns have occurred, but no data has been lost. With a little planning, your data can be kept safe.
Database SoftwareThanks to all the testing, feedback and bug reports the community has performed with the first and second betas, we now have our third beta of 8.3. We hope that this will be our last beta before release candidate so please download and continue testing to ensure that any issues you raised have have been resolved. As always, our community is the first line of defense to help us find any "corner cases" of possible issues."
Embedded Systemsannounced. "These are bugfix-only releases. 1.8.2 contains fixes for inetd, lash, tar, tr, and build system. 1.7.4 contains a fix for inetd."
InteroperabilityThis is a bug fix release of the Samba 3.0.27 code base and is the version that servers should be run for for all current Samba 3.0 bug fixes."
Package Managementannounced. "The open source package manager for Google applications, GooPackage, officially announces its initial release (0.1). GooPackage is a apt-like package manager that offers the ability to install Google web programs onto a desktop using Mozilla Prism."
Securityannounced. "OSsonar monitors systems. It continually audits by managing a collection of auditors called sensors. If a sensor becomes concerned, it can launch countermeasure to handle the concern. Helpful with SOX compliance, remote monitoring - CYOA for Sys Admins. OSsonar 2.4 has been released. New remote monitoring sensors added for Linux and Unix servers."
Web Site Developmentannounced. "2step is a development framework written in Java for developing complete web applications (including web services). As a servlet framework features access control, logging, I18N, database access, deploy, XML/XSL, etc, and uses Jetty, FOP and jCharts. A new version of the 2step plug-in for Eclipse (v2.0.0) has been released. It has been testes with latest Eclipse release (Europa, v3.3), and also v3.2, but it might work with any 3.x Eclipse version. The main new feature of the plug-in is a shiny new content assistant for the 2step language." announced. See the changelog file for more information.
Data Visualizationannounced. "Graphviewer is a generic data visualization tool written in FLTK 1.1 that reads input from a command line data source on the standard input and renders that into a scrolling graph, similarly to perfmeter-like applications." Graphviewer is based on FLTK.
Electronicsannounced. "This snapshot provides (over the old one) ... 1. Bug fix .. "build" exits properly now. This bug was introduced in the 2007-11-02 snapshot as a result of fixing another bug. 2. Plugins should work on MS-Windows, compiled with mingw." Thanks to feedback, comparing with the old version, the new document corrected tens (may be more!) mistakes, and a new documentation was added for the blocks that were not documented in the first published documents. The most important enhancement in the new version is the adding of an alphabetical index to all gnuradio functions and blocks." Icarus Verilog, an electronic simulation language compiler, has been announced. According to the changelog: "This is a bug fix update of the 0.8 stable version of Icarus Verilog. The v0.8 series tries to remain as stable as possible while still fixing bugs that are safe to fix." Kicad, a schematic capture and printed circuit CAD application, is out. See the release notes for details.
Gamesannounced by the WorldForge game project. "Atlas-C++ is the standard implementation of the WorldForge Atlas protocol. This release is primarilly aimed at developers and users who want to build the WorldForge system for themselves. Major changes in this version: * The codec and negotiation parsers are now more efficient. * A small addition has been made to the objects interface for testability."
Music ApplicationsI'm pleased to announce the release of FluidSynth 1.0.8. Dubbed "Its about funky time!", since it was almost 2 years ago since the last release."
Office ApplicationsHylaFAX, a FAX modem interface application, has been announced. "These releases are maintenance releases, and do not contain any new features or functionality, but only contain bugfixes".
Office Suitescovers the latest alpha release of KOffice 2. "Immediately after the release of KDE 4.0 RC1, the KDE office suite KOffice today announced the release of version 2.0 Alpha 5. As with the previous alpha versions of 2.0, this is a technology preview more than a version for users to test out. Nonetheless there are some exciting new features and developments here."
Web Browserstakes a look at the latest security holes to be patched with the release of Firefox 188.8.131.52. LWN is tracking vendor updates here.
Languages and Tools
CWe are in Stage 3. When we reach 100 open regressions, we will go to regression-only mode. When we approach the 4.3.0 release, we will create a branch, and open Stage 1 for 4.4.0."
Perlannounced a new Perl 6 on Parrot Roadmap. "On Friday I wrote up a brief ROADMAP for Perl 6 on Parrot development, and rather than repeat it all as a journal post I'll just link to it. It really does describe where we've been, and where we're going in the next couple of months as far as Perl 6 development."
Bug Trackersannounced. The software is a: "Highly configurable bug tracking, project management, and help desk solution. Project focus is on configurability, usability, and clean code. We're putting the finishing touches on the 2.6.3 release and we'll be rolling it into production some time tonight."
Miscellaneousannounced. "xMarkup is a text transformation utility for processing of a set of ASCII files. All Win-32 and POSIX/UNIX platforms. The transformations performed by utility can be extremely complicated comparing to ordinal search & replace procedures. Actually the utility uses a procedural language, with help of which any algorithms of text transformations can be implemented."
Page editor: Forrest Cook
Next page: Linux in the news>>
Copyright © 2007, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds