Weekly Edition
Return to the Security page
| |||||||||||||
Tarsnap advisory provides a few lessonsAn interesting and brutally honest security advisory for the Tarsnap "secure online backup service" was released on January 18. It certainly shows a refreshing amount of candor that other projects and companies would do well to emulate. But there are some other lessons to be learned from the vulnerability including the value of source code availability and the perils of refactoring. Tarsnap is a company founded by Colin Percival that provides encrypted online storage for backups. The client code is available, but it is not free software. The code can only be used, unmodified, to talk to the Tarsnap service. The server code is evidently completely unavailable, but Percival is interested in hearing from folks with ideas for improvement to the client—or those who have found a security hole. Percival was contacted by Taylor R. Campbell on January 14 with just such a bug. It turns out that a refactoring of the code for the 1.0.22 release, which was made in June 2009, introduced a bug that potentially would allow anyone with access to the data to decrypt it. The data is stored in the Amazon S3 "cloud", which limits the access to a small group, but that doesn't really fit well with the security model espoused by Tarsnap. In the advisory, Percival makes that clear:
I will not attempt to decrypt and read your data. Amazon claims that it
does not inspect Amazon Web Services users' data. And the US government is
theoretically bound by a constitution which prohibits unreasonable
searches. This is all, however, entirely irrelevant: The entire point of
Tarsnap's security is to remove the need for such guarantees. You shouldn't
need to trust me; you shouldn't need to trust Amazon; and you most
certainly shouldn't need to trust the US government.
In doing the refactoring, Percival removed an auto-increment of a nonce value used in the Advanced Encryption Standard (AES) Counter (CTR) mode for encrypting blocks of data. The impact of that is that someone can decrypt the data without having the key. There are two ways that the decryption could be done when the nonce value is reused, either by comparing two ciphertexts or by using known plaintext. The former attack is considered by Percival to be unusable on the Tarsnap data because of the compression done to the data blocks before they are encrypted. On the other hand, known plaintext attacks are quite plausible if there is some known data in the blocks. As Percival points out, full backups are likely to have any number of files with known contents, namely the files that are installed by the operating system—binaries, configuration files, and so on. The bug was found by Campbell by "reading the Tarsnap source code purely out of curiosity", which certainly shows the advantage of making that source available. One wonders if the server code might also benefit from curious hackers. Percival is creating a bug bounty program (and seemingly retroactively paying one out to Campbell) to hopefully ferret out any other problems in the client sooner. Refactoring is meant to be strictly a clean-up operation that does not change the semantics of the code in question. When doing refactoring, it is helpful if there are a set of regression tests that can detect when refactoring has gone awry. In the comments on the advisory, Percival said that Tarsnap does not have a test suite of that sort, and pointed out that it is difficult to create one for cryptographic software, but "I should probably find some way of automatically testing and/or assert()ing for nonce-reuse bugs though". The lack of regression tests is unfortunate, but Tarsnap is hardly alone in that. There are countless projects that refactor their code without such a test suite. This particular incident should serve as something of a reminder to projects, especially those that are implementing security features, that refactoring can and does introduce bugs. A test suite is great, but even just some regression testing of the areas that have been refactored may find bugs like this one. Percival is to be congratulated for quickly turning around a fix for the problem, as well as for being so forthright with the gory details of the bug and its impact. It is far too often that we see companies trying to sweep the details of their security holes under the rug—free software projects sometimes do as well. Bugs happen, security or otherwise, and there is value in seeing what they are and how they came about. We can learn from incidents like this. (Log in to post comments)
Tarsnap advisory provides a few lessons Posted Jan 20, 2011 7:02 UTC (Thu) by ion (guest, #58683) [Link] Id also point out that having subtle side-effects within function call parameters might be something to be avoided. Had encr_aes->nonce++ been a separate expression it might have been less likely to be forgotten when refactoring.
Tarsnap advisory provides a few lessons Posted Jan 20, 2011 9:29 UTC (Thu) by michaeljt (subscriber, #39183) [Link]
> In the comments on the advisory, Percival said that Tarsnap does not have a test suite of that sort, and pointed out that it is difficult to create one for cryptographic software, but "I should probably find some way of automatically testing and/or assert()ing for nonce-reuse bugs though".
At least theoretically, a regression test that catches (some) refactoring errors seems feasible to me. You decide on a set of test workloads to run and make sure that the result (in this case the encrypted output) is identical before and after. After any non-refactoring change though you will need to update your expected test results.
Test suites for cryptography Posted Jan 20, 2011 12:27 UTC (Thu) by epa (subscriber, #39769) [Link]
You're right, a simple before-and-after test would have caught it.
A parallel approach would be to write a dummy cryptography library which essentially spews out the inputs unchanged - so the encrypt() function, rather than returning encrypted data, gives a string saying 'key = xxx, plaintext = yyy, parameters = zzz'. The dummy random number generator will just return 1, 2, 3 etc. You can then inspect the output by hand, or by an automated tool, to check for logic errors such as the same key being used twice when it should not be. This test would only be as good as the person writing the automated checker, but it might provide another chance to catch certain bugs.
Tarsnap advisory provides a few lessons Posted Jan 20, 2011 17:12 UTC (Thu) by kruemelmo (subscriber, #8279) [Link]
the good example to deal with a security-related bug made us consider using this service.
Tarsnap advisory provides a few lessons Posted Jan 21, 2011 7:22 UTC (Fri) by zooko (subscriber, #2589) [Link] The author of the software and owner of the company, Colin Percival, posted to twitter that he has had a sharp increase in new sign-ups since this news went out! http://twitter.com/#!/cperciva/status/28233291609935872 @cperciva Colin Percival Tarsnap signups since I announced the CTR nonce missing-increment bug are 500% above normal. Crazy. I think this is funny! But also it means something -- I'm not sure what. I must say that as a contributor to Tahoe-LAFS -- an open source project that is vaguely competitive (but also, as is often the case, sort of not competitive) -- I'm a bit jealous. When we accidentally expose our users to security failures we don't get a flood of new users! Maybe our security advisories are not as well-written: http://tahoe-lafs.org/hacktahoelafs/ :-) Also our security failures have never been this bad yet. I have committed one or two defects this bad to our source code over the last few years, but in each case our automated tests or my brilliant coding partners caught it before we published it in a new release. Hm. http://www.tarsnap.com/ says he charges $0.30/GB-month. That's roughly three times what Amazon Web Services is charging him. This really makes me wonder if I could go into business doing this. I could use Tahoe-LAFS, charge the same rates that tarsnap charges, and my marketing pitch could be "charges the same as tarsnap, uses the same backend, has similar or better security properties, and is all Free/Open Source Software". Yeah, that would be fun! And then tarsnap and Tahoe-LAFS would be actually competitive. Oh well -- I have too many other things going on right now. But somebody should consider this! Those look like healthy margins to me. By the way, I have been following Colin Percival's work and writings on tarsnap since long before this incident and I respect the way he does security engineering and the way he deals forthrightly with customers. And I greatly appreciate that his entire business model is focussed on the use case that nobody, not even tarsnap itself, gets access to the user's files. It is too bad about this defect that violated that intention, but the fact that he is selling that and (apparently) prospering from it is great!
Tarsnap advisory provides a few lessons Posted Jan 21, 2011 19:14 UTC (Fri) by dlang (✭ supporter ✭, #313) [Link]
just a suggestion, but when posting a link to your project, a link to the main page is better than a link to a hackfest contest.
and yes, I do think it would be good to have someone use your project to provide a similar service to tarsnap, due to the low barrier of entry, they may not make very much money on it, but competition is good.
I think the biggest reason that tarsnap got so many new sign-ups is the 'any publicity is good publicity' effect, most people had not heard of the option, in investigating what the fuss is about they see it's something that they could use.
Tarsnap advisory provides a few lessons Posted Jan 21, 2011 21:16 UTC (Fri) by jeremiah (subscriber, #1221) [Link]
We do this for our clients, but it's an in house service. And the margins are Crazy! Definitely worth your or someone else's time. Or Tarsnap's as it were. It's a fun easy business model that doesn't require much effort if you roll it into software and products you already sell.
Tarsnap advisory provides a few lessons Posted Jan 21, 2011 12:37 UTC (Fri) by nettings (subscriber, #429) [Link]
Chapeau!
I wonder if the alleged increase in business after this exemplary disclosure will ring a bell with beancounters at other security-relavant enterprises :-D
Hey Jon, where was your prediction that 2011 would see the first security incident related PR triumph for open source? ;)
Tarsnap advisory provides a few lessons Posted Jan 21, 2011 17:17 UTC (Fri) by patrick_g (subscriber, #44470) [Link]
Tarsnap is not free software.
Tarsnap advisory provides a few lessons Posted Jan 21, 2011 21:24 UTC (Fri) by Darkmere (subscriber, #53695) [Link]
No, but it is (partially) open source. Not Free Software, but the Source is there.
Tarsnap advisory provides a few lessons Posted Jan 22, 2011 11:47 UTC (Sat) by jpnp (subscriber, #63341) [Link]
Please. Visible source is not Open source. It's certainly much better to have access to the source code of security components than not, but Open Source Software has a more explicit meaning.
Tarsnap advisory provides a few lessons Posted Jan 22, 2011 12:47 UTC (Sat) by Darkmere (subscriber, #53695) [Link]
I'll concede to that, I just took the time to re-read the Open Source definition. Sorry for the error.
|
|||||||||||||