User: Password:
|
|
Subscribe / Log in / New account

Development

How The Backup Process Has Changed

By Forrest Cook
November 28, 2007

Backing up data stored on computers is one of the most important jobs of a systems administrator. A regular backup routine can save large amounts of heartache and frustration when a disk drive or system fails. Disk failure should be treated as something that is guaranteed to happen in the life (and death) of every disk drive. And, disk failures always seem to happen at the worst possible moment. Typical failures happen on Friday afternoon before a vacation is about to start or when the boss comes into your office demanding that critical report that lives on the machine with the smoke curling out of the power supply.

Over the years, your author has lived through many backup technologies. In the early days of home computing and CP/M systems, floppy to floppy transfer was the only method to save data. Floppies were unreliable and multiple copies were important. When hard drives became normal hardware on DOS-based microprocessor systems, backups were performed on piles of floppy disks or short-lived tape technologies. It was a bad day when floppy disk 29 of a batch of 30 encountered a read error during the restoration of a disk.

Mainframe systems in the early 1980s required copying the contents of washing machine sized disk drives to piles of 9 track open reel tapes. As drives were added, the piles of tapes became larger. Large storage areas were required for storing racks of tapes. The 1990s brought larger disk drives and the capabilities of the mainframes and PCs were converging. Single filesystems could be copied to DC100 serpentine tape cartridges, if one had a lot patience. Helical scan SCSI tape drives such as Exabyte 8mm and DDS 4mm were able to store the contents of multiple filesystems on one tape. For a brief while, tape capacity surpassed filesystem size. Robotic tape library machines could be programmed to automate the backup process and allow large numbers of filesystems to be copied to stacks of tapes.

Disk capacities continued to expand rapidly. AIT tapes were good for larger backups, but the media was pricey. RAID arrays became a good way to increase storage capacity and improve reliability, but downtime could be long in the event of a controller failure. RAID backups are still critically important. Disk drive prices continued to fall. At some point after the year 2000, the price/performance of disks versus tapes made it more economical to buy another disk drive to copy data to. For the moment, it appears that the disk/tape competition is over and disks won.

With a removable drive sled or a USB drive, a hard drive can now be treated as a high speed random access data cartridge. With multiple online machines, it is possible to use one machine as a backup repository for another's data. Today, it is possible to buy a 300GB disk drive for just over $100. Larger drives can be had for a slightly higher cost per byte.

On a Linux platform, two of the oldest and most common backup utilities are dump and tar. Both work with tape and disk-based archives. Dump has the advantage of being able to dig through the contents of an archive and pick individual files to restore before reading the entire media. Unfortunately, the dump archive format has gone through a lot of changes. This means, for example, that a dump archive that was created on a Red Hat 7.3 system may be unreadable on a Ubuntu 7.04 system. Reading old tar files is more likely to result in success across systems of different vintages. Your author decided to standardize on tar-based backups.

Now for some current real-world examples for performing disk-to-disk backups: Here's how to use dump to copy the local / filesystem to a compressed and datestamped file on the same machine's /backup filesystem:

cd /backup
/sbin/dump 0ufa - / | bzip2 > ./localslash`date +%Y%m%d`.bz2
Here's how to use tar to do the same type of local to local backup:
cd /backup
/bin/tar cf - / | bzip2' > ./localslash`date +%Y%m%d`.tar.bz2
Here's how dump is used to backup the / filesystem on a machine called remote to the local machine's /backup partition:
cd /backup
ssh remote '/sbin/dump 0ufa - / | bzip2' > ./remoteslash`date +%Y%m%d`.bz2
Here's how to use tar to do the remote to local backup:
cd /backup
ssh remote '/bin/tar cf - / | bzip2' > ./remoteslash`date +%Y%m%d`.tar.bz2
The above commands should be run from the root account and the remote backups can work without passwords if ssh is setup correctly. Ssh and the ssh server should be installed and configured on the machines. The dump and/or tar manual pages should be consulted for more information on the various command options. Restoration of the filesystems involves using bunzip2 to uncompress the archive, then restore (for dump) or tar (for tar) to split out the contents to a local disk. Restoration across the network is possible with the use of ssh.

A good backup scheme should be devised. Your author has a dedicated machine with a large disk drive and an old DDS3 tape drive that is used to backup all of the rest of his machines to. Variations on the above examples are used in several machine-specific scripts to backup one machine at a time. The backups are performed several times a month. Backups can be copied from the backup machine's disk to tape for offsite storage. The entire backup set is occasionally copied to another machine's large disk for redundancy. Datasets can simply be copied with cp to removable media. A 100GB+ audio archive is managed differently than standard filesystems, the rsync command is used to clone the data from one machine to another.

In the early 1990s, your author couldn't imagine ever getting close to filling up a 9GB disk drive. Then came audio archives, digital cameras with movie modes and other large data sources. Several hard drive failures and machine meltdowns have occurred, but no data has been lost. With a little planning, your data can be kept safe.

Comments (60 posted)

System Applications

Database Software

PostgreSQL 8.3 Beta3 released

Version 8.3 Beta3 of the PostgreSQL DBMS has been announced. "Thanks to all the testing, feedback and bug reports the community has performed with the first and second betas, we now have our third beta of 8.3. We hope that this will be our last beta before release candidate so please download and continue testing to ensure that any issues you raised have have been resolved. As always, our community is the first line of defense to help us find any "corner cases" of possible issues."

Full Story (comments: none)

Postgres Weekly News

The November 25, 2007 edition of the Postgres Weekly News is online with the latest PostgreSQL DBMS articles and resources.

Full Story (comments: none)

Embedded Systems

BusyBox 1.7.4 and 1.8.2 released

Stable versions 1.8.2 and 1.7.4 of BusyBox, a collection of command line utilities for embedded systems, have been announced. "These are bugfix-only releases. 1.8.2 contains fixes for inetd, lash, tar, tr, and build system. 1.7.4 contains a fix for inetd."

Comments (none posted)

Interoperability

Samba 3.0.27a available

Version 3.0.27a of Samba has been announced. "This is a bug fix release of the Samba 3.0.27 code base and is the version that servers should be run for for all current Samba 3.0 bug fixes."

Full Story (comments: none)

Package Management

GooPackage: Initial Release (SourceForge)

The first version of GooPackage has been announced. "The open source package manager for Google applications, GooPackage, officially announces its initial release (0.1). GooPackage is a apt-like package manager that offers the ability to install Google web programs onto a desktop using Mozilla Prism."

Comments (none posted)

Security

OSsonar 2.4 released (SourceForge)

Version 2.4 of OSsonar has been announced. "OSsonar monitors systems. It continually audits by managing a collection of auditors called sensors. If a sensor becomes concerned, it can launch countermeasure to handle the concern. Helpful with SOX compliance, remote monitoring - CYOA for Sys Admins. OSsonar 2.4 has been released. New remote monitoring sensors added for Linux and Unix servers."

Comments (none posted)

Web Site Development

2step plug-in for Eclipse 3.x v2.0.0 released (SourceForge)

Version 2.0.0 of 2step plug-in for Eclipse 3.x has been announced. "2step is a development framework written in Java for developing complete web applications (including web services). As a servlet framework features access control, logging, I18N, database access, deploy, XML/XSL, etc, and uses Jetty, FOP and jCharts. A new version of the 2step plug-in for Eclipse (v2.0.0) has been released. It has been testes with latest Eclipse release (Europa, v3.3), and also v3.2, but it might work with any 3.x Eclipse version. The main new feature of the plug-in is a shiny new content assistant for the 2step language."

Comments (none posted)

mnoGoSearch 3.3.6 released

Version 3.3.6 of mnoGoSearch, a web site search engine, has been announced. See the changelog file for more information.

Comments (none posted)

Quixote 2.5 announced

Version 2.5 of the Quixote web development platform is out with a bug fix and support for Python 2.5.

Full Story (comments: none)

Desktop Applications

Data Visualization

Graphviewer 1.1.0 released

Version 1.1.0 of Graphviewer, a data visualizer tool, has been announced. "Graphviewer is a generic data visualization tool written in FLTK 1.1 that reads input from a command line data source on the standard input and renders that into a scrolling graph, similarly to perfmeter-like applications." Graphviewer is based on FLTK.

Comments (none posted)

Desktop Environments

GNOME Software Announcements

The following new GNOME software has been announced this week: You can find more new GNOME software releases at gnomefiles.org.

Comments (none posted)

Test Latest Builds with KDE4Daily (KDE.News)

KDE.News has a call for testing on the daily builds of KDE4. "We are now on the home stretch of the road to KDE 4.0, but KDE still needs extensive user testing to make sure everything arrives in the best possible shape for the release."

Comments (none posted)

KDE Commit-Digest (KDE.News)

The November 18, 2007 edition of the KDE Commit-Digest has been announced. The content summary says: "A Calculator and Show Desktop Plasmoid, units conversion and contacts "runners", enhanced composite-based effects, a "dashboard" view and applet hover handles in Plasma. Updated artwork for "about" pages (like the one present in Konqueror upon application startup). Support for quick user switching in Kickoff. Continued development progress in KDevelop 4. Work begins on resurrecting KEduca for the KDE 4.1 timeframe..."

Comments (none posted)

KDE Commit-Digest (KDE.News)

The November 25, 2007 edition of the KDE Commit-Digest has been announced. The content summary says: "A Trash applet, various general improvements, and support for sharing configuration layouts in Plasma. "Undo close tab" feature in Konqueror. Development continues towards Amarok 2.0, with services becoming plugins and support for the Amapche music server. Continued progress in KDevelop and KEduca. More work on album display and improved thumbnails (with RAW format support) in Digikam. A BitTorrent plugin for KGet, based on the recently created libktorrent..."

Comments (none posted)

KDE Software Announcements

The following new KDE software has been announced this week: You can find more new KDE software releases at kde-apps.org.

Comments (none posted)

Electronics

gnucap development snapshot 2007-11-21

Development snapshot 2007-11-21 of gnucap, a circuit analysis package, has been announced. "This snapshot provides (over the old one) ... 1. Bug fix .. "build" exits properly now. This bug was introduced in the 2007-11-02 snapshot as a result of fixing another bug. 2. Plugins should work on MS-Windows, compiled with mingw."

Comments (none posted)

Updated Unofficial Gnuradio Documentation (Simple User Manual)

New documentation is available for the Gnuradio software controlled radio development platform. "Thanks to feedback, comparing with the old version, the new document corrected tens (may be more!) mistakes, and a new documentation was added for the blocks that were not documented in the first published documents. The most important enhancement in the new version is the adding of an alphabetical index to all gnuradio functions and blocks."

Full Story (comments: 1)

Icarus Verilog 0.8.6 released

Version 0.8.6 of Icarus Verilog, an electronic simulation language compiler, has been announced. According to the changelog: "This is a bug fix update of the 0.8 stable version of Icarus Verilog. The v0.8 series tries to remain as stable as possible while still fixing bugs that are safe to fix."

Comments (none posted)

Kicad 2007-11-19-RC announced

Release 2007-11-19-RC of Kicad, a schematic capture and printed circuit CAD application, is out. See the release notes for details.

Comments (none posted)

Games

Atlas-C++ 0.6.1 released

Version 0.6.1 of Atlas-C++ has been announced by the WorldForge game project. "Atlas-C++ is the standard implementation of the WorldForge Atlas protocol. This release is primarilly aimed at developers and users who want to build the WorldForge system for themselves. Major changes in this version: * The codec and negotiation parsers are now more efficient. * A small addition has been made to the objects interface for testability."

Comments (none posted)

Music Applications

FluidSynth 1.0.8 released

Version 1.0.8 of FluidSynth is out with numerous new features and bug fixes. "I'm pleased to announce the release of FluidSynth 1.0.8. Dubbed "Its about funky time!", since it was almost 2 years ago since the last release."

Full Story (comments: none)

Office Applications

HylaFAX 4.4.3/4.3.7 releases

Versions 4.4.3 and 4.3.7 of HylaFAX, a FAX modem interface application, has been announced. "These releases are maintenance releases, and do not contain any new features or functionality, but only contain bugfixes".

Comments (none posted)

Office Suites

KOffice 2 Alpha 5 Released (KDE.News)

KDE.News covers the latest alpha release of KOffice 2. "Immediately after the release of KDE 4.0 RC1, the KDE office suite KOffice today announced the release of version 2.0 Alpha 5. As with the previous alpha versions of 2.0, this is a technology preview more than a version for users to test out. Nonetheless there are some exciting new features and developments here."

Comments (none posted)

Web Browsers

Mozilla Firefox 2.0.0.10 Released (MozillaZine)

MozillaZine takes a look at the latest security holes to be patched with the release of Firefox 2.0.0.10. LWN is tracking vendor updates here.

Comments (1 posted)

Languages and Tools

C

GCC 4.3.0 Status Report

The November 27, 2007 edition of the GCC 4.3.0 Status Report has been published. "We are in Stage 3. When we reach 100 open regressions, we will go to regression-only mode. When we approach the 4.3.0 release, we will create a branch, and open Stage 1 for 4.4.0."

Full Story (comments: none)

Caml

Caml Weekly News

The November 27, 2007 edition of the Caml Weekly News is out with new articles about the Caml language.

Full Story (comments: none)

JSP

Myna Javascript Application Server 1.0 alpha 6 released (SourceForge)

Version 1.0 alpha 6 of Myna has been announced, it features an API change among other things. "Myna is a Java web development platform based on Mozilla's Rhino JavaScript engine. It allows developers to access the full power of Java with the development speed of JavaScript."

Comments (none posted)

Perl

Perl 6 on Parrot Roadmap (use Perl)

brian_d_foy has announced a new Perl 6 on Parrot Roadmap. "On Friday I wrote up a brief ROADMAP for Perl 6 on Parrot development, and rather than repeat it all as a journal post I'll just link to it. It really does describe where we've been, and where we're going in the next couple of months as far as Perl 6 development."

Comments (1 posted)

Python

Python-URL! - weekly Python news and links

The November 28, 2007 edition of the Python-URL! is online with a new collection of Python article links.

Full Story (comments: none)

Tcl/Tk

Tcl-URL! - weekly Tcl news and links

The November 23, 2007 edition of the Tcl-URL! is online with new Tcl/Tk articles and resources.

Full Story (comments: none)

Bug Trackers

zenTrack 2.6.3 released (SourceForge)

Version 2.6.3 of zenTrack has been announced. The software is a: "Highly configurable bug tracking, project management, and help desk solution. Project focus is on configurability, usability, and clean code. We're putting the finishing touches on the 2.6.3 release and we'll be rolling it into production some time tonight."

Comments (none posted)

Miscellaneous

xMarkup 2.1.0 released (SourceForge)

Version 2.1.0 of xMarkup, a Text Transformation Utility, has been announced. "xMarkup is a text transformation utility for processing of a set of ASCII files. All Win-32 and POSIX/UNIX platforms. The transformations performed by utility can be extremely complicated comparing to ordinal search & replace procedures. Actually the utility uses a procedural language, with help of which any algorithms of text transformations can be implemented."

Comments (none posted)

Page editor: Forrest Cook
Next page: Linux in the news>>


Copyright © 2007, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds