Obnam 1.0 released [LWN.net]

Obnam 1.0 released

Posted Jun 2, 2012 6:10 UTC (Sat) by pr1268 (guest, #24648) [Link] (2 responses)

Silly me, glancing too quickly at the article heading and thinking it read "Obama 1.0 released".

Nevertheless, this program looks interesting; I might have to try it out. Thanks, Lars.

Obnam 1.0 released

Posted Jun 2, 2012 9:01 UTC (Sat) by alankila (guest, #47141) [Link] (1 responses)

I have an actual btrfs rsync+snapshot -based backup solution. Still, Lars's scheme is better because it handles those pesky file renames and reuse of data blocks between files... We still do not have dedup for btrfs, unfortunately. It's not a huge deal, but I could see running it once a week or something.

Browsable backups

Posted Jun 4, 2012 5:32 UTC (Mon) by gmatht (subscriber, #58961) [Link]

I like backups where you can browse the files in the backup easily. It gives me the warm fuzzies to be able to see the files, and quickly check that a couple of important files are recoverable and up-to-date. Additionally, I like my backups to be able to double as archives.

Obnam 1.0 released

Posted Jun 2, 2012 11:36 UTC (Sat) by boog (subscriber, #30882) [Link] (1 responses)

Correct me if I'm wrong, but obnam seems to be unusual in that it supports a push model with compression (deduplication), so one doesn't need to give privileged access to the backed up machine (and each backed up machine can have restricted rights on the backup server). For this reason alone I intend to try it. And of course it's well integrated into debian.

Obnam 1.0 released

Posted Jun 3, 2012 23:32 UTC (Sun) by mattdm (subscriber, #18) [Link]

rdiff-backup supports that same model, although with no intrinsic deduplication or compression.

Yet another backup tool :-)

Posted Jun 2, 2012 14:37 UTC (Sat) by rvfh (guest, #31018) [Link]

This guy started his backup project at the same time I started mine. Looks nice indeed! Maybe I too should make an official announcement sometimes...

For those interested: http://sourceforge.net/projects/hbackup/ which depends on http://sourceforge.net/projects/htoolbox/ and other generic libs (see the README file.)

Obnam 1.0 released

Posted Jun 2, 2012 17:09 UTC (Sat) by theophrastus (guest, #80847) [Link] (2 responses)

Looks like an interesting project. I couldn't figure out how easy it would be to set it up with a NAS/NSLU(2)/NAS200 (http://en.wikipedia.org/wiki/NAS200) if it permits pushing then I guess it would be no problem..?

Obnam 1.0 released

Posted Jun 3, 2012 12:33 UTC (Sun) by liw (subscriber, #6379) [Link] (1 responses)

Pushing is definitely the preferred way to use Obnam at this time.

Obnam 1.0 released

Posted Jun 3, 2012 16:34 UTC (Sun) by bronson (subscriber, #4806) [Link]

Off-topic and meaningful only for base 10 sentimentalists, congrats on posting article 500000. :)

Obnam 1.0 released

Posted Jun 7, 2012 9:21 UTC (Thu) by callegar (guest, #16148) [Link] (4 responses)

Cannot execute the example in the man. Consistently getting:

Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/cliapp/app.py", line 172, in _run
self.process_args(args)
File "/usr/lib/python2.7/dist-packages/obnamlib/app.py", line 127, in process_args
cliapp.Application.process_args(self, args)
File "/usr/lib/python2.7/dist-packages/cliapp/app.py", line 407, in process_args
method(args[1:])
File "/usr/lib/python2.7/dist-packages/obnamlib/plugins/backup_plugin.py", line 115, in backup
self.add_client(client_name)
File "/usr/lib/python2.7/dist-packages/obnamlib/plugins/backup_plugin.py", line 140, in add_client
if client_name not in self.repo.list_clients():
File "/usr/lib/python2.7/dist-packages/obnamlib/repo.py", line 201, in list_clients
listed = set(self.clientlist.list_clients())
File "/usr/lib/python2.7/dist-packages/obnamlib/clientlist.py", line 78, in list_clients
if self.init_forest() and self.forest.trees:
File "/usr/lib/python2.7/dist-packages/obnamlib/repo_tree.py", line 63, in init_forest
vfs=self.fs)
File "/usr/lib/python2.7/dist-packages/larch/forest.py", line 167, in open_forest
assert allow_writes is not None
AssertionError

Don't know if I'm doing something wrong... but surely some exception catching and reformulation of exceptional conditions in user friendly words would be good for an 1.0 release. The fact that there is no apparent agreement on when the 'beta' or 'alpha' indications should be removed may cause some harm to free software (see KDE 4.0). BTW, how it is that the log says that 1.0 is in fact obnam version 0.24.1?

Obnam 1.0 released

Posted Jun 7, 2012 11:16 UTC (Thu) by pkern (subscriber, #32883) [Link]

Assertions are explicitly for unexpected states. You cannot sanely convert that to any user-friendly meaning because then it wouldn't have been an assertion but a regular check for errors.

On the other hand I wonder why one would default initialize the first parameter only to check that it isn't the default in the next line.

Obnam 1.0 released

Posted Jun 7, 2012 11:55 UTC (Thu) by liw (subscriber, #6379) [Link] (2 responses)

The Python stack trace is awful for the user, but it's there as a development aid. If a user sees it, something is badly wrong in Obnam.

Are you sure you have the right version of Obnam? How do you check that? Obnam is supposed to report the right version number in the log file or with "obnam --version", and if it isn't 1.0 then something's wrong.

Do you have the right version of the Larch Python library installed? That stack trace looks like there's an old version installed. Are you running on Debian? Ubuntu? Somewhere else? The Ubuntu PPA is in the process of being updated to fix a few problems with wrong versions. If you're on either Ubuntu or Debian, what is the output of "dpkg -l obnam python-larch"?

(See also my response at http://liw.fi/obnam/bugs/Obnam_cannot_backup_with_failed_... )

Obnam 1.0 released

Posted Jun 7, 2012 22:50 UTC (Thu) by callegar (guest, #16148) [Link] (1 responses)

My big fault. All my apologies. Went to the ubuntu ppa, saw the 1.0 release, added the PPA, added obnam without noticing that due to build problems the 1.0 release wasn't actually available, probably mixed an older obnam with a too modern larch.

Obnam 1.0 released

Posted Jun 8, 2012 6:56 UTC (Fri) by liw (subscriber, #6379) [Link]

No worries: it's really my fault for generating such a large pile of packages to be kept in sync. I hope Obnam is more to your satisfaction now.

Obnam 1.0 released

Posted Jun 12, 2012 23:19 UTC (Tue) by Baylink (guest, #755) [Link] (6 responses)

B!tch, please.

If it copies to another hard drive, it's not a backup.

It if copies to spinning optical media, it's only barely a backup.

As someone who's been a professional system administrator for over 2 decades (and can still read cartridge tape backups* made that long ago, please believe me when I say: if it's not on tape, it's not a backup.

If you don't have at least 2 (preferably 3) copies and one of them isn't outside the building (and preferably the metropolitan area; pity the people who kept their 1WTC backups in 2WTC, or even 7WTC), it's not a backup.

If you don't verify it after write, test-restore it, and check it about every 3 years, it's not a backup.

"Backup" is a word the semantics of which people depend on; just like with version numbers. There are really good reasons not to repurpose that word to mean less secure things, because even if you tell people, they won't get it, or remember it.

(* I have DC-600s from 1996 that I test-read last year, after a retention. No appreciable hits.)

Obnam 1.0 released

Posted Jun 12, 2012 23:38 UTC (Tue) by dlang (guest, #313) [Link] (5 responses)

I disagree with your first two statements.

there's nothing inherently wrong with using hard drives or optical media for backups.

the key is having copies offsite and testing them so you know you can use them

Obnam 1.0 released

Posted Jun 13, 2012 21:52 UTC (Wed) by Baylink (guest, #755) [Link] (4 responses)

Well, you were more polite than I was. :-}

But, no, my research and my experience both concur: there are in fact /inherent/ problems for using spinning magnetic or optical storage media rather than tape, for archival storage in the category which most people expect when they say 'backup' - and they're order of magnitude differences.

If you have a backup on disk or CDr from 1992 that you can still read reliably, I'd be happy to hear about it.

Obnam 1.0 released

Posted Jun 13, 2012 22:02 UTC (Wed) by dlang (guest, #313) [Link] (2 responses)

given the changes in tape technology, how many tape drives do you still have around that can read tapes written in 1992 reliably?

if you are doing long-term archiving, no single copy is a valid long-term storage. In every case you need to have multiple copies, with error correction, and test them frequently enough so that when one copy fails to read you can still get the data from one of the other copies.

you should be migrating your data from one generation of backup to another on a frequent basis if you are doing long-term archiving.

However, most people are not talking about long-term archiving when they talk about backups, they are talking about disaster recovery, getting the systems running again with the data that is online and available just before the disaster. For that you need multiple copies, and geographic distribution of the copies, but you don't need to read media that's been in storage for 20 years.

Obnam 1.0 released

Posted Jun 15, 2012 3:08 UTC (Fri) by Baylink (guest, #755) [Link]

Everything written on a DC-600 class cartridge is readable on the 4GB model I have around; I didn't treat the DC-3000 class tapes as long-term archival, nor the 8mm's; the last DAT will read everything older; the SuperDLT 220 reads all the way back to DLT I; an LTO-5 reads all the way back to at *least* LTO-1, and possibly the SuperDLT formats.

In short: yes: tape drive hardware design engineers are nearly fetishistic about backwards compatibility, drive longevity, and interchange.

And you're correct in saying that it's generally not a *requirement* to read data that old -- but the fact that you can usually contributes to making it easier to read data that's much newer.

One other reason, BTW, that that was possible?

Non-proprietary *backup software*; those tapes were written with a "super-Tar" program, Microlite's BackupEdge, I think, so that even if that
software was either no longer available or incompatible, I wouldn't care: *tar* could read the tapes.

You're certainly correct, though, in noting that for "true" archival storage, it's a job in itself, migrating to newer formats and layouts. This would be a vote in Obnam's favor; at least you could keep the package around in source, and as long as you could make it build, you could retrieve stuff.

I have in fact had to retrieve 5 year old backups, of accounting year-ends when a merger was being contemplated. That was when my client turned out to be really happy about all that money they'd had to "waste" on backup tape (nightly full, 5 Friday's, 3 EOM, 4 EOQ, pre-close and post-close).

Obnam 1.0 released

Posted Jun 19, 2012 12:42 UTC (Tue) by ekj (guest, #1524) [Link]

That's the only sane answer, furthermore even if old media remained readable and compatible forever, it would *still* be the only sane answer because of the rapidly expanding size of storage-media.

I could keep a dozen 20MB hdds around, or I could store their entire content as a tarball on a single current disc, and have them take up 240MB, or 0.012% of the capacity of one disc.

Then I could add in a half-dozen 500MB hdds from a few years later, a few 10GB discs from a few years later, and 2 or 3 100GB-discs from a few years ago. This adds up to dozens of discs.

Or I could do the sane thing and store all of these as files on a current disc. In sum total they take up about 10% of the storage-capacity of a single disc. And they're instantly accessible if I actually want to use any of the old files.

For security, I have a second copy on a disc in my basement, and a tertiary copy in an online account on a different continent that costs less for a year than the electricity for the old discs would cost.

Why try desperately to keep old storage-media alive when it's not the media but the *data* that has value, and that data is by nessecity trivial in size today ?

Barring exceptional circumstances, *nobody* have digital data from 1993 that aren't trivial in size.

Obnam 1.0 released

Posted Jun 14, 2012 1:21 UTC (Thu) by bronson (subscriber, #4806) [Link]

> If you have a backup on disk or CDr from 1992 that you can still read reliably, I'd be happy to hear about it.

Yes, I have a 1996-era backup on disk. It's an 80MB tarfile sitting on my 1TB stata hard disk. Of course, it's not the SAME disk. So what? Next time I upgrade disk drives, I'll just copy it over with everything else. Works great.

(Not much of value in there, just some Mac System 7 utilities I wrote and an old mail spool. Utterly worthless. I'd delete it if 80 MB weren't completely inconsequential these days...)

CDr, you're absolutely right. Not a chance. Those start dropping bits after 6 months.