Tarsnap advisory provides a few lessons
An interesting and brutally honest security
advisory for the Tarsnap "secure online
backup service
" was released on January 18. It certainly shows a
refreshing amount of candor that other projects and companies would do well
to emulate. But there are some other lessons to be learned from the
vulnerability including the value of source code availability and the
perils of refactoring.
Tarsnap is a company founded by Colin Percival that provides encrypted online storage for backups. The client code is available, but it is not free software. The code can only be used, unmodified, to talk to the Tarsnap service. The server code is evidently completely unavailable, but Percival is interested in hearing from folks with ideas for improvement to the client—or those who have found a security hole.
Percival was contacted by Taylor R. Campbell on January 14 with just such a bug. It turns out that a refactoring of the code for the 1.0.22 release, which was made in June 2009, introduced a bug that potentially would allow anyone with access to the data to decrypt it. The data is stored in the Amazon S3 "cloud", which limits the access to a small group, but that doesn't really fit well with the security model espoused by Tarsnap. In the advisory, Percival makes that clear:
In doing the refactoring, Percival removed an auto-increment of a nonce value used in the Advanced Encryption Standard (AES) Counter (CTR) mode for encrypting blocks of data. The impact of that is that someone can decrypt the data without having the key.
There are two ways that the decryption could be done when the nonce value is reused, either by comparing two ciphertexts or by using known plaintext. The former attack is considered by Percival to be unusable on the Tarsnap data because of the compression done to the data blocks before they are encrypted. On the other hand, known plaintext attacks are quite plausible if there is some known data in the blocks. As Percival points out, full backups are likely to have any number of files with known contents, namely the files that are installed by the operating system—binaries, configuration files, and so on.
The bug was found by Campbell by "reading the Tarsnap source code
purely out of curiosity
", which certainly shows the advantage of
making that source available. One wonders if the server code might also
benefit from curious hackers. Percival is creating a bug bounty program
(and seemingly retroactively paying one out to Campbell) to hopefully
ferret out any other problems in the client sooner.
Refactoring is meant to be strictly a clean-up operation that does not
change the semantics of the code in question. When doing refactoring, it
is helpful if there are a set of regression tests that can detect when
refactoring has gone awry. In the comments on the advisory, Percival said
that Tarsnap does not have a test suite of that sort, and pointed out that
it is difficult to create one for cryptographic software, but "I
should probably find some way of automatically testing and/or assert()ing
for nonce-reuse bugs though
".
The lack of regression tests is unfortunate, but Tarsnap is hardly alone in that. There are countless projects that refactor their code without such a test suite. This particular incident should serve as something of a reminder to projects, especially those that are implementing security features, that refactoring can and does introduce bugs. A test suite is great, but even just some regression testing of the areas that have been refactored may find bugs like this one.
Percival is to be congratulated for quickly turning around a fix for the
problem, as well as for being so forthright with the gory details of the
bug and its impact. It is far too often that we see companies trying to
sweep the details of their security holes under the rug—free software
projects sometimes do as well. Bugs happen, security or otherwise, and
there is value in seeing what they are and how they came about. We can
learn from incidents like this.
| Index entries for this article | |
|---|---|
| Security | Encryption |
Posted Jan 20, 2011 7:02 UTC (Thu)
by ion (guest, #58683)
[Link]
Posted Jan 20, 2011 9:29 UTC (Thu)
by mjthayer (guest, #39183)
[Link] (1 responses)
At least theoretically, a regression test that catches (some) refactoring errors seems feasible to me. You decide on a set of test workloads to run and make sure that the result (in this case the encrypted output) is identical before and after. After any non-refactoring change though you will need to update your expected test results.
Posted Jan 20, 2011 12:27 UTC (Thu)
by epa (subscriber, #39769)
[Link]
A parallel approach would be to write a dummy cryptography library which essentially spews out the inputs unchanged - so the encrypt() function, rather than returning encrypted data, gives a string saying 'key = xxx, plaintext = yyy, parameters = zzz'. The dummy random number generator will just return 1, 2, 3 etc. You can then inspect the output by hand, or by an automated tool, to check for logic errors such as the same key being used twice when it should not be. This test would only be as good as the person writing the automated checker, but it might provide another chance to catch certain bugs.
Posted Jan 20, 2011 17:12 UTC (Thu)
by kruemelmo (guest, #8279)
[Link]
Posted Jan 21, 2011 7:22 UTC (Fri)
by zooko (guest, #2589)
[Link] (2 responses)
The author of the software and owner of the company, Colin Percival, posted to twitter that he has had a sharp increase in new sign-ups since this news went out!
http://twitter.com/#!/cperciva/status/28233291609935872
I think this is funny! But also it means something -- I'm not sure what.
I must say that as a contributor to Tahoe-LAFS -- an open source project that is vaguely competitive (but also, as is often the case, sort of not competitive) -- I'm a bit jealous. When we accidentally expose our users to security failures we don't get a flood of new users! Maybe our security advisories are not as well-written:
http://tahoe-lafs.org/hacktahoelafs/
:-)
Also our security failures have never been this bad yet. I have committed one or two defects this bad to our source code over the last few years, but in each case our automated tests or my brilliant coding partners caught it before we published it in a new release.
Hm. http://www.tarsnap.com/ says he charges $0.30/GB-month. That's roughly three times what Amazon Web Services is charging him. This really makes me wonder if I could go into business doing this. I could use Tahoe-LAFS, charge the same rates that tarsnap charges, and my marketing pitch could be "charges the same as tarsnap, uses the same backend, has similar or better security properties, and is all Free/Open Source Software". Yeah, that would be fun! And then tarsnap and Tahoe-LAFS would be actually competitive.
Oh well -- I have too many other things going on right now. But somebody should consider this! Those look like healthy margins to me.
By the way, I have been following Colin Percival's work and writings on tarsnap since long before this incident and I respect the way he does security engineering and the way he deals forthrightly with customers. And I greatly appreciate that his entire business model is focussed on the use case that nobody, not even tarsnap itself, gets access to the user's files. It is too bad about this defect that violated that intention, but the fact that he is selling that and (apparently) prospering from it is great!
Posted Jan 21, 2011 19:14 UTC (Fri)
by dlang (guest, #313)
[Link]
and yes, I do think it would be good to have someone use your project to provide a similar service to tarsnap, due to the low barrier of entry, they may not make very much money on it, but competition is good.
I think the biggest reason that tarsnap got so many new sign-ups is the 'any publicity is good publicity' effect, most people had not heard of the option, in investigating what the fuss is about they see it's something that they could use.
Posted Jan 21, 2011 21:16 UTC (Fri)
by jeremiah (subscriber, #1221)
[Link]
Posted Jan 21, 2011 12:37 UTC (Fri)
by nettings (subscriber, #429)
[Link] (4 responses)
I wonder if the alleged increase in business after this exemplary disclosure will ring a bell with beancounters at other security-relavant enterprises :-D
Hey Jon, where was your prediction that 2011 would see the first security incident related PR triumph for open source? ;)
Posted Jan 21, 2011 17:17 UTC (Fri)
by patrick_g (subscriber, #44470)
[Link] (3 responses)
Posted Jan 21, 2011 21:24 UTC (Fri)
by Darkmere (subscriber, #53695)
[Link] (2 responses)
Posted Jan 22, 2011 11:47 UTC (Sat)
by jpnp (guest, #63341)
[Link] (1 responses)
Posted Jan 22, 2011 12:47 UTC (Sat)
by Darkmere (subscriber, #53695)
[Link]
Id also point out that having subtle side-effects within function call parameters might be something to be avoided. Had Tarsnap advisory provides a few lessons
encr_aes->nonce++ been a separate expression it might have been less likely to be forgotten when refactoring.
Tarsnap advisory provides a few lessons
Test suites for cryptography
Tarsnap advisory provides a few lessons
Tarsnap advisory provides a few lessons
@cperciva Colin Percival
Tarsnap signups since I announced the CTR nonce missing-increment bug are 500% above normal. Crazy.
Tarsnap advisory provides a few lessons
Tarsnap advisory provides a few lessons
Tarsnap advisory provides a few lessons
Tarsnap advisory provides a few lessons
Tarsnap advisory provides a few lessons
Tarsnap advisory provides a few lessons
Tarsnap advisory provides a few lessons