OpenPGP in Rust: the Sequoia project
In 2018, three former GnuPG developers began work on Sequoia, a new implementation of OpenPGP in Rust. OpenPGP is an open standard for data encryption, often used for secure email; GnuPG is an implementation of that standard. The GPLv2-licensed Sequoia is heading toward version 1.0, with a handful of issues remaining to be addressed. The project's founders believe that there is much to be desired in GnuPG, which is the de facto standard implementation of OpenPGP today. They hope to fix this with a reimplementation of the specification using a language with features that will help protect users from common types of memory bugs.
While GnuPG is the most popular OpenPGP implementation — especially for Linux — there are others, including OpenKeychain, OpenPGP.js, and RNP. OpenPGP has been criticized for years (such as this blog post from 2014, and another from 2019); the Sequoia project is working to build modern OpenPGP tooling that addresses many of those complaints. Sequoia has already been adopted by several other projects, including keys.openpgp.org, OpenPGP CA, koverto, Pijul, and KIPA.
Sequoia was started by Neal H. Walfield, Justus Winter, and Kai Michaelis; each worked on GnuPG for about two years. In a 2018 presentation [YouTube] (slides [PDF]) Walfield discussed their motivations for the new project. In his opinion, GnuPG is "hard to modify" — mostly due to its organic growth over the decades. Walfield pointed out the tight coupling between components in GnuPG and the lack of unit testing as specific problem areas. As an example, he noted that the GnuPG command-line tool and the corresponding application libraries do not have the same abilities; there are things that can only be done using the command-line tool.
Community is a big part of the Sequoia project, as Walfield explained in his presentation. The project is financially backed by the p≡p (pep) and Wau Holland foundations, with "all development done in the open". Before code was even written, the project founders met with prominent members of the OpenPGP community as well as end users to discuss the project's plans and make sure their approach was sound. The project's current status, Git repository, and issue tracker are all available. The repository logs indicate approximately 30 contributors to the project and, since announcing the pending version 1.0 release in April 2020, three releases have been made. The latest release, version 0.19.0, was made in August 2020 — its most notable improvement is the inclusion of Windows Cryptography API: Next Generation (CNG) as a backend, replacing Nettle which has issues in non-POSIX environments.
Because it is written in Rust, Sequoia benefits from all of the memory-safety advantages that the language provides. The repository shows significant efforts are being made to write unit tests to prevent regressions and improve quality. Unlike GnuPG, where the command-line gpg tool is more powerful than the library, Sequoia is an OpenPGP library first; all of its functionality is available via the exposed APIs. The project plans to provide two "levels" of API, a low-level unopinionated implementation of the OpenPGP specification and a high-level API with sensible defaults to make common tasks like signing and signature verification easier for users. Walfield's presentation was clear that, while the project endeavored to be unopinionated on the low-level API implementation, it does avoid the outdated parts of the specification such as the use of MD5 hashes.
Sequoia is targeted at "modern platforms" including Linux, Windows, macOS, Android, and iOS. When possible, this includes using existing cryptographic tooling; it is a design goal of the project to tightly integrate with platform-specific cryptographic services. For example, Sequoia plans to make use of the Secure Enclave coprocessor on iOS devices when available. It also provides a foreign function interface (FFI) for integrating the project with programs written in other languages. Currently, Sequoia offers C and Python FFI bindings. Readers should note that programs using the bindings lack memory safety, so the project's rules must be followed in order to use them correctly.
Walfield responded to my question on the Sequoia development mailing list with information about the current state of the project and the upcoming version 1.0 release:
First, for our upcoming 1.0 release (anytime now (tm), although the API and feature set have been stable for months; we are just documenting everything, and carefully reviewing the API & code), we are only releasing the low-level API, i.e., the sequoia-openpgp crate, and its dependencies.
This means readers shouldn't expect Sequoia to be an end-user replacement for
tools like GnuPG just yet; the first major release will be focused on a
library for developers. For the project to be ready for end users, it will
need the equivalent of the gpg command found in GnuPG. For Sequoia,
this is the sq command-line tool, which is not included for version 1.0. Walfield described sq as "missing quite a bit of
functionality,
" explaining "we want to reserve the right to
change its interface
" before it is released to the public.
The sq tool isn't the only thing missing from the version 1.0
release; key-storage services are still in progress. According to Walfield,
it is one of the project's "top priorities
" after the version 1.0 release.
Key storage is one of the areas where Sequoia's plans are unique compared to other OpenPGP implementations. For added security, the project plans to use process separation between the services handling public and private keys; Cap'n Proto will be used for interprocess communication. Walfield notes in his presentation that process separation is not always possible, such as in iOS environments. When it is unavailable, Sequoia plans to use a shared SQLite database to communicate between the two services in a process Walfield described as "colocation".
Conceptually, Sequoia takes an identity-based approach to its public
keyrings, where the keyring is designed to be "more like a per-domain
address book than a PGP keyring.
" Keys are designed to be stored and
accessed using a user-assigned Petname, with the ability to
associate arbitrary structured data that will be useful in the implementation
of trust models. Walfield also argues that this approach is more in line with
how users actually think about keys: associated with names, rather than a
collection of abstract IDs. Further, all keys are assigned a "realm" in
Sequoia indicating the intended purpose of the key; presently realms include
"contacts" and "software update key" designations. When completed, Sequoia's
keyring service will automatically update public keys from remote servers
(similar to parcimonie), ensuring
that changes like new sub-keys and revocations are discovered in a timely
fashion. The API documentation indicates
this can be done using anonymized services like Tor in addition to more common
TLS-encryption methods.
The private keyring service is slated to offer an optional one-password solution for unlocking local keys. The library will provide access to the private keyring using what Walfield describes as a "Smartcard-like" API. Currently Sequoia does not support Smartcards, but based on this ticket, the community would like to see it happen in the future. Sequoia's private keyring service will be written with forward secrecy in mind, by using the OpenPGP specifications supporting the distinction between data "at rest" (encrypted storage) and data "in motion" (encrypted transmission), which is not found in many other implementations. It is a sound security practice to frequently rotate "in motion" keys, but that needs to be done in a way that still allows archived encrypted data to be decrypted. A presentation (slides [PDF]) by Winter compares the project's forward secrecy features to other OpenPGP implementations.
Wrapping up
Those who are interested in getting involved in the project should be aware
that it requires contributors to assign their copyright
to the p≡p foundation. The project further states that code may then end up
being published under multiple licenses, but "all software is also
published under a GNU License as Free Software, without exception.
"
In all, it is good to see a project focused on making OpenPGP a more easily
accessible technology, and it looks like the project has made steady progress
since it started almost three years ago. The project's documentation also provides a reasonable
starting point to using the library in applications. That said, Sequoia has a
ways to go before it will be a trusted cryptographic tool; for starters, the
code still needs to be audited. The project's status page states it has
"has not been audited yet, but as soon as we release the core Sequoia
crate, it will be audited by a third party.
" Readers may be interested
in checking out the project's contributors page, which provides
details on their mailing list and IRC channel. As Walfield said, there isn't
a timetable for when Sequoia version 1.0 will land, but it does sound like it
will be soon. End users of OpenPGP will still have to wait a bit longer,
however, before Sequoia becomes a viable alternative.
Posted Sep 11, 2020 19:14 UTC (Fri)
by Cyberax (✭ supporter ✭, #52523)
[Link] (24 responses)
Posted Sep 11, 2020 19:32 UTC (Fri)
by jazzy (subscriber, #132608)
[Link] (16 responses)
Posted Sep 11, 2020 20:43 UTC (Fri)
by IanKelling (subscriber, #89418)
[Link] (14 responses)
I think there are a few GPL apps on iOS (GNU Jami is the one I know of) that have been there for years, so Apple just doesn't seem to care or enforce their TOS.
So, the real question is, why did they pick GPLv2 when GPLv3 is better? They want their program embedded in tivoized devices without any legal recourse? They should upgrade their license.
Posted Sep 11, 2020 20:44 UTC (Fri)
by IanKelling (subscriber, #89418)
[Link]
Posted Sep 11, 2020 21:31 UTC (Fri)
by Cyberax (✭ supporter ✭, #52523)
[Link] (10 responses)
Posted Sep 12, 2020 13:29 UTC (Sat)
by Deleted user 129183 (guest, #129183)
[Link] (9 responses)
Please stop spreading FUD. GPL 3 improves upon GPL 2 by fixing the tivoisation loophole, providing patent grants, increasing compatibility with licences like Apache 2 and some other things which I’ve forgotten.
Posted Sep 12, 2020 15:04 UTC (Sat)
by Wol (subscriber, #4433)
[Link] (7 responses)
Yes v3 has a bunch of bug-fixes for v2, and if they'd limited to that it would be a real improvement.
(And I've had people say that - for things that the FSF said were bug fixes - they thought v2 was a feature and didn't WANT the fix!)
So I think a LOT people wouldn't say v3 was an improvement.
Cheers,
Posted Sep 12, 2020 19:47 UTC (Sat)
by martin.langhoff (subscriber, #61417)
[Link] (4 responses)
GPLv3 is _different_.
Some bug fixes that would be nice to fold into a "GPLv2.1".
And some major changes of social contract, which would be better suited in a different license.
Posted Sep 13, 2020 23:24 UTC (Sun)
by cyphar (subscriber, #110703)
[Link] (3 responses)
I do want to point out (since I assume your reference to "change in the social contract" is in relation to the tivoisation clause), that while folks like to wax lyrical about GPLv3's tivoisation clause, many seem to forget that GPLv2 actually had a similar (in spirit) requirement: > For an executable work, complete source code means all the source code for all modules it contains, plus any associated interface definition files, plus the scripts used to control compilation and installation of the executable. [emphasis added] I don't think it's a stretch to say that if firmware keys were widely deployed in 1991 that GPLv2 would've had a more substantial clause dedicated to making sure that the "scripts used to control compilation and installation of the executable" would've included any firmware signing keys necessary to make it possible to actually use the software you've modified. And yes, the modern interpretation of "scripts used to control compilation and installation of the executable" does include instructions on how to install the software on to whatever hardware you've been sold. My point is, maybe you disagree with the tivoisation clause -- but GPLv2 clearly had a similar spirit in this area (it's just that the FSF didn't predict that firmware signing was going to be a widespread method for stopping people from being able to run modified software). So it's less a "change in the social contract" and more "updating the legal wording to match the original intent in a world where firmware signing keys exist"
Posted Sep 14, 2020 21:09 UTC (Mon)
by martin.langhoff (subscriber, #61417)
[Link] (2 responses)
No, that does not hold. You get the tools so can build install the executable... somewhere.
The _gist_ of GPLv2 is share and share-alike. It's about the source code.
GPLv3 improved on v2 on many aspects, but also brought it a new front: control of the hardware. Applied to software not tightly tied to hw -- a web app -- it doesn't matter. Applied to kernels, device drivers, etc it's a massive problem. As a result, folks who work closely to hardware don't want to use it.
To be clear. I don't intend to re-hash the GPLv3 controversies here. Just to point out -- GPLv3 is different, in a meaningful way. Pick GPLv2 or GPLv3, but know they are different beasts.
Posted Sep 15, 2020 5:06 UTC (Tue)
by cyphar (subscriber, #110703)
[Link] (1 responses)
> No, that does not hold. You get the tools so can build install the executable... somewhere. Yes, GPLv2 and GPLv3 are obviously legally speaking quite different on this point. My point is that since the discussion was about "social contracts", it should be noted that the GPLv2 does have a spiritually similar requirement. If you don't see the similarity in spirit between "scripts that control the compilation and installation of the executable" and "any methods, procedures, authorization keys, or other information required to install and execute modified versions of a covered work in that User Product from a modified version of its Corresponding Source", I don't really know what else to say. My point was simply that the new requirements didn't come out of nowhere.
Posted Sep 15, 2020 14:23 UTC (Tue)
by martin.langhoff (subscriber, #61417)
[Link]
Good? Bad? That was the social contract under GPLv2.
GPLv3 introduces rules about the User Product. We're trying to leverage _software_ to put rules on _hardware_. That is a new frontier, and a new social contract.
No significant/popular low-level software projects, where this matters, have adopted GPLv3. So on that front, it did not find traction.
And here's the funny thing -- I personally dislike Tivoization. If the current GPLv3 was called T(ivo)GPL, similar to what happened with AGPL, and we had a GPLv2.1, life would be much better.
Posted Sep 13, 2020 8:15 UTC (Sun)
by cyphar (subscriber, #110703)
[Link]
If you want compatibility with Apache-2.0 (which GPLv2 doesn't have), you need to have patent grants.
Posted Sep 24, 2020 3:21 UTC (Thu)
by donbarry (guest, #10485)
[Link]
Posted Sep 23, 2020 23:04 UTC (Wed)
by flussence (guest, #85566)
[Link]
…like remediating the sudden death aspect of GPLv2. The old version certainly is better if you're Patrick McHardy.
Posted Sep 11, 2020 21:33 UTC (Fri)
by neal (subscriber, #7439)
[Link] (1 responses)
Posted Sep 27, 2020 17:14 UTC (Sun)
by oldtomas (guest, #72579)
[Link]
I assume it was private communicaton. If not, I'd appreciate a link to that, to learn motivation and background.
Posted Sep 13, 2020 13:56 UTC (Sun)
by link2xt (guest, #141333)
[Link]
It is licensed under MIT/Apache 2.0, like most of the code in Rust.
Posted Sep 11, 2020 19:48 UTC (Fri)
by geofft (subscriber, #59789)
[Link] (6 responses)
Posted Sep 11, 2020 19:54 UTC (Fri)
by Cyberax (✭ supporter ✭, #52523)
[Link] (5 responses)
This is not really sustainable. For example, if Apple is forced to split the Apple Store into a different company and rename it to "Pear Store", the disclaimer will become void. They'll have to get agreement for a new exception from every copyright holder who has ever contributed to the core code.
I actually won't mind a lawyer-proofed GPL exception to allow distribution through app stores, but that won't allow abuse ("my application is a store, so I can distribute GPL-ed app through it!").
Posted Sep 12, 2020 15:40 UTC (Sat)
by rvolgers (guest, #63218)
[Link] (1 responses)
Posted Sep 12, 2020 18:57 UTC (Sat)
by Wol (subscriber, #4433)
[Link]
That's the problem with any extension to the GPL - you can't take advantage of other GPL code to include in your own project.
Cheers,
Posted Sep 12, 2020 18:51 UTC (Sat)
by Wol (subscriber, #4433)
[Link] (2 responses)
Just say "it is acceptable to distribute binaries on a binary-only site, provided the download points the user to the corresponding source on a 3rd-party source distribution site (3rd party as in "not yours, not the app-store"). Note that the GPLv3 permits separate distribution of binary and source without triggering the "binary-only means you must offer source for 3 years" clause. (That was a bug in v2, which at least some people would love to have seen make its way into v3). It's a bug because it means the actions of the recipient change the liability of the provider.
Cheers,
Posted Sep 13, 2020 12:53 UTC (Sun)
by ballombe (subscriber, #9523)
[Link] (1 responses)
Posted Sep 23, 2020 23:54 UTC (Wed)
by Wol (subscriber, #4433)
[Link]
Cheers,
Posted Sep 12, 2020 10:04 UTC (Sat)
by grawity (subscriber, #80596)
[Link] (1 responses)
Is that significantly different from GnuPG 2.x using gpg-agent for all private key operations?
Posted Sep 12, 2020 16:25 UTC (Sat)
by neal (subscriber, #7439)
[Link]
Sequoia's key store firstly uses petnames to address keys. You can think of a petname like an entry in a personal address book: it's controlled by you, not the person who created the key. In this way, address book entries are directly associated with keys. Currently, this is done indirectly: you select a recipient from your address book, your MUA pulls out some information and queries GnuPG's key store for a key with a matching identifier, and uses the returned key. That works okay, as long as you carefully curate your keyring, and don't have multiple keys with the same identifier. But, once that happens, the logic becomes rather unpredictable for most users.
Posted Sep 12, 2020 16:55 UTC (Sat)
by vadim (subscriber, #35271)
[Link] (11 responses)
This sounds exactly like the sort of thing I needed. Some time back I tried doing a PGP related project, and found GPG's structure very inconvenient. One of the things I want is to go through the archive of a keyserver and parse the data. The need to invoke GPG, feed it a key, and parse the result was slow and painful. What I really want is a library where I can take an ASCII encoded key, feed it to some function and get all the data back -- who it belongs to, who signed it, when it expires, etc, while ensuring the key is fully valid and all the signatures are correct and so on. All of that should ideally happen without needing to launch a million processes and needing to work around GPG's desire for things like a home directory and a keyring.
If this project is usable for such a task, I'm very interested, and Rust sounds like a bonus because there's obvious security concerns in parsing a huge key archive that may well have been attacked in many ways over the years.
Posted Sep 12, 2020 20:17 UTC (Sat)
by dbnichol (subscriber, #39622)
[Link] (6 responses)
I saw sequoia a while ago and am excited to see where it goes.
Posted Sep 12, 2020 22:19 UTC (Sat)
by vadim (subscriber, #35271)
[Link]
I think PGP was greatly handicapped by the backwardness of GPG's design. Had it been better it'd be far more pleasant to integrate into other software, would be better supported, more performant, and there'd be a much larger ecosystem of tools built around it. Pity, really.
The bad performance is no joke, by the way. GPG is noticeably slow even on modern hardware. This could probably be avoided with a library that caches data internally, but when you run gpg once per message it requires paying a quite steep startup cost that's very noticeable.
Still, I think this is a fixable problem, because nothing really superseded it as far as I know, so there's still hope that Sequoia and other projects fix this situation.
Posted Sep 13, 2020 4:43 UTC (Sun)
by alison (subscriber, #63752)
[Link] (4 responses)
GnuPG is far from being the only common Linux tool which needs a library in addition to a CLI tool. Two that bit me personally have been mtd-utils and sgdisk. I needed libraries for my own code, so I basically creating what was, in effect, a libmtd and libgpt in a project's build system. I was hoping to have employer support to do a cleaner job of creating libraries and posting patches upstream, but they never agreed. As a result, I finally had to rip all of the code out in order to avoid license violations.
It's great the Sequoia is taking a more sensible approach: make a great library with good test coverage, and then a reference implementation binary that other can try out or even use.
Posted Sep 13, 2020 5:35 UTC (Sun)
by josh (subscriber, #17465)
[Link]
Posted Sep 17, 2020 7:48 UTC (Thu)
by mezcalero (subscriber, #45103)
[Link] (2 responses)
Posted Sep 17, 2020 14:36 UTC (Thu)
by alison (subscriber, #63752)
[Link] (1 responses)
Posted Sep 22, 2020 6:30 UTC (Tue)
by jwilk (subscriber, #63328)
[Link]
Posted Sep 13, 2020 7:34 UTC (Sun)
by neal (subscriber, #7439)
[Link] (1 responses)
Posted Sep 13, 2020 17:16 UTC (Sun)
by vadim (subscriber, #35271)
[Link]
I'm a complete newbie in Rust, but been thinking of trying to make something in it for some time.
Posted Sep 14, 2020 10:21 UTC (Mon)
by abo (subscriber, #77288)
[Link] (1 responses)
Posted Sep 17, 2020 20:05 UTC (Thu)
by Comet (subscriber, #11646)
[Link]
Posted Sep 13, 2020 14:42 UTC (Sun)
by kushal (subscriber, #50806)
[Link]
Posted Oct 5, 2020 6:52 UTC (Mon)
by cpitrat (subscriber, #116459)
[Link]
Does that mean I may not be able to decipher something I stored on an cd-rom 15 years ago and didn't re-encrypt since? It sounds like being retro-compatible is important at least for decoding.
OpenPGP in Rust: the Sequoia project
OpenPGP in Rust: the Sequoia project
OpenPGP in Rust: the Sequoia project
OpenPGP in Rust: the Sequoia project
OpenPGP in Rust: the Sequoia project
GPLv3 is NOT better...
OpenPGP in Rust: the Sequoia project
OpenPGP in Rust: the Sequoia project
Wol
OpenPGP in Rust: the Sequoia project
OpenPGP in Rust: the Sequoia project
OpenPGP in Rust: the Sequoia project
> that GPLv2 actually had a similar (in spirit) requirement
OpenPGP in Rust: the Sequoia project
OpenPGP in Rust: the Sequoia project
OpenPGP in Rust: the Sequoia project
OpenPGP in Rust: the Sequoia project
OpenPGP in Rust: the Sequoia project
OpenPGP in Rust: the Sequoia project
OpenPGP in Rust: the Sequoia project
OpenPGP in Rust: the Sequoia project
The copyright holders can disclaim any restrictions of the GPL that conflict with the App Store terms. For instance, Blink Shell is GPL'd and has been available on iOS for a while, relying on a waiver from the mosh developers (since mosh is also GPL'd).
OpenPGP in Rust: the Sequoia project
OpenPGP in Rust: the Sequoia project
OpenPGP in Rust: the Sequoia project
OpenPGP in Rust: the Sequoia project
Wol
OpenPGP in Rust: the Sequoia project
Wol
OpenPGP in Rust: the Sequoia project
OpenPGP in Rust: the Sequoia project
Wol
OpenPGP in Rust: the Sequoia project
OpenPGP in Rust: the Sequoia project
OpenPGP in Rust: the Sequoia project
OpenPGP in Rust: the Sequoia project
OpenPGP in Rust: the Sequoia project
OpenPGP in Rust: the Sequoia project
OpenPGP in Rust: the Sequoia project
OpenPGP in Rust: the Sequoia project
OpenPGP in Rust: the Sequoia project
As far as I can see, libfdisk does support EFI partition labels:
OpenPGP in Rust: the Sequoia project
extern int fdisk_partition_set_name(struct fdisk_partition *pa, const char *name);
extern const char *fdisk_partition_get_name(struct fdisk_partition *pa);
Here is a small program that iterates over each certificate ("OpenPGP Key") in an SKS dump, and prints a bit of information about the certificate. You can look at the source to "sq dump" to see how to extract other information.
OpenPGP in Rust: the Sequoia project
OpenPGP in Rust: the Sequoia project
OpenPGP in Rust: the Sequoia project
OpenPGP in Rust: the Sequoia project
OpenPGP in Rust: the Sequoia project
OpenPGP in Rust: the Sequoia project