Re: sq and keystores

Hi John,

On Thu, 10 Sep 2020 09:38:30 +0200,
John Coggeshall wrote:
> I have a question

Thanks for reaching out!

> One thing I noticed was that, while I was able to import my PGP keys,
> etc.. I couldn't seem to be able to USE those PGP keys in the store to
> encrypt or decrypt messages. The only way I could get it to work was
> to specify the keyfile using things like --secret-key-file.
> 
> Is this intended behavior, or am I just missing how this is supposed
> to work?

There are a few things going on here.

First, for our upcoming 1.0 release (anytime now (tm), although the
API and feature set have been stable for months; we are just
documenting everything, and carefully reviewing the API & code), we
are only releasing the low-level API, i.e., the sequoia-openpgp crate
[1], and its dependencies.  The high-level library [2], the services
(the sequoia key store [3] & network component [4]), and the
command-line tool sq [5] are functionally incomplete, and their
interfaces are still experimentally.


  [1] https://docs.sequoia-pgp.org/sequoia_openpgp/
  [2] https://docs.sequoia-pgp.org/sequoia_core/index.html
  [3] https://docs.sequoia-pgp.org/sequoia_store/index.html
  [4] https://docs.sequoia-pgp.org/sequoia_net/index.html
  [5] https://docs.sequoia-pgp.org/sq/index.html

(FWIW, some experience suggests that our low-level API is actually
pretty high level.  For instance, we replaced the gpg backend in the
pep engine with a Sequoia backend.  This replaced 3.1 kLOC [6] with
3.3 kLOC [7].  That sounds like a small increase, but just under 1
kLOC are for implementing a key store.  So, in this case, implementing
equivalent functionality using our low-level API required less code
(~900 fewer LOC) than using gpgme's high-level API.)

  [6] https://pep.foundation/dev/repos/pEpEngine/file/b484f5c19...
  [7] https://pep.foundation/dev/repos/pEpEngine/file/b484f5c19...

One of the main bits of missing functionality in the key store is
support for handling secret key material.  So, as you correctly
observed, it is currently only possible to use secret key material
when it is stored in a file using the --secret-key-file option to sq.
We actually have some code to manage secret keys, but it is at such an
early stage, that we haven't checked it in yet.  Finishing up the key
store is one of our top priorities after the 1.0 release.

Modifying sq to take advantage of that functionality should be
straightforward.  However, we aren't releasing sq either, because we
want to reserve the right to change its interface: sq is missing quite
a bit of functionality.  And, as we expose more of the low-level
functionality, experience has taught us that we'll discover
deficiencies in our current API.  We want to be able to fix those
issues.  Also, we don't yet have a machine readable interface (we plan
to implement a REST-like JSON-based API so users won't have to
implement complex state machines and parse custom formats), and we
don't want people to screenscrape sq.


That said, encrypting to keys stored in the key store (and verifying
messages signed by keys in the key store) should work.  In this case,
you need to address them by their label.  For instance, to create a
mapping and then encrypt a message to that key, this should work:

  $ sq mapping import neal neal.asc
  $ echo foo | sq encrypt -r neal
  -----BEGIN PGP MESSAGE-----

  wcBMA8K4GQVsZSWY...

But, this won't work:

  $ sq encrypt -r neal@walfield.org
  Error: No such key found

  Caused by:
      Key not found

even though my key has that email address:

  $ sq inspect neal.asc
  neal.asc: OpenPGP Certificate.

      Fingerprint: 8F17 7771 18A3 3DDA 9BA4  8E62 AACB 3243 6300 52D9
      ...
           UserID: Neal H. Walfield <neal@walfield.org>

because the email address is not authenticated.

Right now, the key store only implements a petname interface [8] for
addressing certificates ("PGP keys").  In the future, we'll add
support for the WoT.

  [8] https://en.wikipedia.org/wiki/Petname

One reason why you can only use authenticated identifiers is that we
view the key store as a cache of all keys, not a curated keyring.
That means, just because you have a key with an email address locally,
there isn't any more reason to use it than if you found it on an SKS
key server.  We have ideas for how to simplify authentication.  In
particular, we take an address-book view of key management where
identifiers are central rather than a key-centric view like gpg does,
which is confusing when there are multiple keys with the same
identifier.


Since you are writing about Sequoia, you might also want to check out
our affiliated projects:

  - Hagrid, the verifying key server that powers
    https://keys.openpgp.org

    https://gitlab.com/hagrid-keyserver/hagrid

  - OpenPGP CA, a tool for simplifying key management and
    authentication in groups and organizations

    https://gitlab.com/openpgp-ca/openpgp-ca

  - Koverto, a tool for adding a encrypion / signing gateway in front
    of a service, like a wiki

    https://gitlab.com/koverto/koverto

  - The OpenPGP Interoperability Test Suite
    A tool for exploring edge cases in many OpenPGP implementations
    https://tests.sequoia-pgp.org/

  - Sequoia Dump
    A web frontend to 'sq packet dump', a tool for exploring OpenPGP
    messages

    https://dump.sequoia-pgp.org/


If you have any more questions, feel free to follow up here or ask on
our irc channel (Freenode, #sequoia).

:) Neal