Encrypted file transfer with Magic Wormhole

At PyCon 2016 in Portland, Oregon, Brian Warner gave a presentation about his most recent project: Magic Wormhole. It is a way to securely transfer files over an encrypted channel. As might be guessed from the venue of his talk, Magic Wormhole is written in Python.

Warner has worked on Python projects for a long time. He created Buildbot and was one of the developers of the Tahoe-LAFS distributed data store. He has also worked on various Mozilla projects, such as Firefox Sync and Mozilla Persona.

Magic Wormhole is a file-transfer program, though it can also be used to transfer simple strings or directories. Over the years, people have been using SSH, email, or shared directories for transferring files, but Magic Wormhole is easier to use, he said, especially for unrelated computers.

A simple "pip install magic-wormhole" (in a Python virtual environment, he suggested) is used to install the package. The sender then simply does a "wormhole send filename" which prepares to send the file and provides a code (such as "7-guitarist-revenge") that is used by the receiver. That code is transmitted to the receiver via some other mechanism (e.g. over the phone); the receiver runs "wormhole receive", types in the code, and the file is transferred.

In his mind's eye, Warner sees the code as a "magic string" that two wizards on distant hilltops speak to establish a channel between them. The code is a way to "make two computers meet each other", once the humans using them have met through some means. That could be in person, on IRC, or via some other means.

File transfer may seem like a solved problem—FTP has been around since 1980, for example. But he likes to characterize the solutions by both their safety from eavesdroppers and the amount of data that a sender or receiver needs in order to do the transfer. For example, an email attachment can be sent, but requires the sender to know roughly 30 characters for the destination email address and the contents of the email can potentially be seen by intermediaries.

His slides [Speaker Deck] list other options, along with their cost and eavesdropping potential. FTP or HTTP require roughly 60 characters for the receiver, as does using a service like Dropbox. Those transfers can also be intercepted by intermediaries or endpoints. There is a potential transcription problem, though, for long URLs and the like (even email addresses sometimes) if the transfer is done by phone or in-person.

If the two people are in the same room, a USB drive or similar could be used, but "ewww cooties"—people may be reluctant to put those devices into their computers. SSH provides a secure mechanism that thwarts eavesdroppers, but requires that a 740-character public key be exchanged. Magic Wormhole, on the other hand, is also immune to interception, but uses a roughly 20-character string that is designed to be easy to transcribe and dictate.

Behind the scenes, Magic Wormhole has a few different phases that it steps through to handle the transfer. It starts with a "rendezvous message" exchange using a central rendezvous server. That establishes the channel that will be used for the transfer, which is identified by a channel ID that is the number at the beginning of the code. Then it uses password-authenticated key exchange (PAKE) to agree on keys. After that, IP addresses are exchanged between the endpoints, an encrypted transit connection is made, and the data is transferred. Once that happens, the channel is closed, so only one transfer to one other endpoint is done; other transfers require new channels and codes.

There is a rendezvous server that he runs, which doesn't participate in the transfer, but just facilitates finding the other endpoint and establishing a key by passing messages between the endpoints using a given channel ID. It is a WebSocket relay server and the URL of it is baked into the client, which allows the IP address and port to be omitted from the code. Those wanting to run their own server can do so and either change the client or use command-line options to override those values.

There are multiple PAKE protocols in use, Warner said. The best known is the Secure Remote Password (SRP) protocol, but its security proof is not that good. SPAKE2 [PDF] is what is being used by the community today, he said, so it is what Magic Wormhole uses as well. The idea is that the same password results in the same session key and it can turn a short string into a larger key. It is similar to Diffie-Hellman key exchange, but adds a blinding factor.

He has written a pure-Python implementation of SPAKE2 using the Ed25519 elliptic curve that is quite fast. The idea behind PAKE is that a weak secret coupled with interaction (message exchange) can produce a strong secret (the session key). A passive eavesdropper gets zero information, while an active man in the middle gets one guess at the weak secret. If the guess fails, both endpoints know that it occurred.

The Magic Wormhole codes are the input to the PAKE. They are built from a list of 256 words that are chosen to have a distinct sound. Each code is single-use and forward-secure, he said. The default is to use two words (prepended with a channel ID number) for the codes, which gives 64K possibilities. Each code is only active for a particular transfer; once that completes, the channel is destroyed and the code is no longer useful. If the code is entered incorrectly (or guessed wrong), wormhole will exit with an error message. A user would have to rerun the program 655 times to give an attacker a 1% chance of guessing the code. So, "an unwillingness to continue in the face of multiple errors" would be a good thing for users.

The local IP address is discovered using ifconfig, then a listening connection is set up. Both sides exchange their address and port number and, once they have received that information, try to connect with the other side to trade encrypted handshakes. The first successful connection is the one that gets used.

That all works fine when both systems are using public IP addresses or are on the same local network, but doesn't work if both are behind network address translation (NAT). He also runs a "transit relay" to shuttle data between the endpoints if a direct connection cannot be established. He is working on NAT hole-punching to avoid the transit relay.

The data is encrypted using libsodium with the crypto_secretbox interface, which uses the Salsa20 stream cipher and the Poly1305 message-authentication code (MAC). The data is also hashed using SHA256 and the final ACK confirms the hash value.

Magic Wormhole has a library API that can be used to programmatically exchange data using the system. In the future, he would like to add a GUI application and a browser extension to make it easier to use. He would also like to add to the transports being used, potentially adding support for WebRTC, Tor Onion Services, and others. He is hoping to add SPAKE2 to libsodium and to port Magic Wormhole to other languages, such as Go, Rust, and JavaScript.

There are uses for Magic Wormhole beyond simple file transfer, Warner said. It can be used anywhere there is a need to deliver a credential or to get a key from one application to another. For example, provisioning clients with keys is now generally done by typing a password into the server's web application, then typing the same password into the client. With Magic Wormhole, that could be done by getting a code from the server and typing that into the client; once the key exchange has been done, the code is no longer useful.

In addition, messaging applications could use it to exchange keys directly between participants. Rather than having Alice upload her public key to a server and requiring Bob to get it from there, Alice could just provide Bob with a Wormhole code and he can retrieve it directly from her system.

But the "real reason" behind building the tool is to get more developers to add PAKE to their toolbox. Various cryptographic tools have slowly become part of the security arsenal (symmetric encryption, signing, hashes, and so on); he would like to see PAKE get there too. In order for that to happen, there need to be good examples and use cases, as well as helpful libraries. File transfer is just the "foot in the door"; it is "very handy" but not that exciting. PAKE, on the other hand, is something that could be quite useful in a variety of situations.

Warner took questions from the audience that mostly focused on the limitations of Magic Wormhole. The protocol does have a low barrier to entry, which means that it also has a "low barrier to abuse", he said, so denial of service is definitely a problem area. Using up all of the channel IDs on the rendezvous server would be one way to do so.

A YouTube video of the talk is available for those interested in more details.

[ I would like to thank LWN subscribers for supporting my travel to Portland for PyCon. ]