Sharing stories on Scuttlebutt

Not many people live on sailboats. Things may be better these days, but back in 2014 sailboat dwellers had to contend with lag-prone, intermittent, low-bandwidth internet connections. Dominic Tarr decided to fix the problem of keeping up with his friends by developing a delay-tolerant, fully distributed social-media protocol called Scuttlebutt. Nearly twelve years later, the protocol has gained a number of users who have their own, non-sailboat-related reasons to prefer a censorship-resistant, offline-first social-media system.

In Scuttlebutt, each person has an append-only log of information where each entry is signed with their private key; this log contains both data (such as social-media posts) and metadata (such as information on who is following whom). Since each entry in the log is signed, it doesn't matter by what route those entries reach interested recipients. When two computers running software that supports Scuttlebutt connect to each other, they exchange a list of which feeds they are interested in, and then share any entries that one has that the other doesn't. This is a straightforward example of a gossip protocol, and it provides a simple foundation for higher-level social-media applications.

Since each person's logs are separate, there is no need for a global consensus algorithm — it is normal and expected for each participant in the network to have a different view of what is going on. In fact, there is no central registry of protocol participants. Instead, people add their friends' feeds directly, and then organically discover other people by seeing who their friends interact with. By default, many Scuttlebutt clients will replicate the feeds of anyone within two degrees of separation: direct friends, and friends of friends. If a friend interacts with the post of someone more distant than that (replying to it, or just reacting to it with an emoji), most clients will display the unknown post as a button that adds the unknown post's feed to the set of feeds to follow.

Connections between computers can be made locally (typically announced over local WiFi via mDNS), or via relay servers on the internet called "rooms". This ensures that two people can keep in touch as long as they have some regular contact — meeting up in person, being online at the same time, or just having a mutual friend who meets those criteria. There is also a related project, tinySSB, that operates over LoRa radio with much smaller packet sizes and longer range. Either way, it's difficult to say how many people are using Scuttlebutt on a regular basis since the connections are formed opportunistically and the replication distance of feeds is limited. There is no such thing as a message that will propagate to the network as a whole.

The underlying network communication behind Scuttlebutt is relatively simple; it's an agreed set of binary formats for establishing connections, exchanging information on what feeds each participant is interested in, and then exchanging signed log entries. Since the underlying protocol is so simple, there are a number of implementations for users to choose from. Several of these also agree on the storage format for data at rest, allowing users to seamlessly switch between them. In addition to the expected social-media applications, there are also programs that use Scuttlebutt as a transport layer for Git, for sharing JavaScript packages, for sharing recipes, and for playing chess.

Below is a screenshot of one of the more popular clients, Manyverse, showing the friends-of-friends feed. The feed can be filtered and searched in various ways, but defaults to showing all posts the client is aware of in reverse chronological order. The other tabs on the left show private (encrypted) messages, interactions with one's own posts, and what other Scuttlebutt clients are currently connected (directly or via a relay), respectively.

Since getting connected to Scuttlebutt requires starting with a friend's feed, it can be difficult to join in. The historical solution to this has been "pubs" — automated servers that do nothing but automatically follow your feed if you ask them to. There are a handful of public ones available. Following a pub's feed lets people new to the network see what is going on generally, and lets their own posts be visible to other people. In that sense, a pub can serve much the same role as a local Mastodon server by allowing people to connect to a specific existing community with a general theme. But, due to the nature of the internet, that also puts the burden of moderation on a pub's operator, to some extent. More generally, pubs run somewhat counter to the idea of a decentralized network to begin with, by creating a point of failure for people trying to join the network.

Over time, the Scuttlebutt community has recommended moving away from pubs and toward rooms, which just act as a traffic relay and not a part of the social graph. Rooms do allow people to see who else is online simultaneously (identified by public key, since without replicating their feed there is no metadata such as username available). So, one approach to joining the network is connecting to a chosen public room, looking at the posts of the random people who happen to be online, and deciding to follow some subset of them. Getting one's own posts out there still requires using a pub, contacting someone out-of-band, or being randomly chanced upon in a room. Another approach is to meet people in person and exchange Scuttlebutt feeds. Because users are free to curate their own lists of interesting people and ignore everyone else, spam is not too much of a problem.

The internet being what it is, however, the system does need some safeguards. In a centralized system, there can be a specific team responsible for moderation. In a distributed setting, that becomes a lot harder. For one thing, not all users of the protocol will agree on what content needs moderation. For another, the lack of a global consensus algorithm means that even if there were some group that agreed on moderation, there is no way that they could apply that decision to the whole network — by design.

The system that the Scuttlebutt ecosystem has settled on is based on blocking specific people and propagating those blocks. Anyone can simply decline to replicate a specific feed that they don't want to share, and by recording that decision in their own feed in a machine-readable way, it can propagate out to their friends who trust their judgment. That way a particular friend group only needs to block a troublesome source of spam or abuse once, instead of once per person. This isn't perfect — it essentially requires everyone to do a little moderation work for their friends — but it's the best solution available at the moment. The Scuttlebutt community has done some research into alternate approaches, such as distributed reputation systems, but none are in wide use.

If the experience of using Scuttlebutt were to be summarized in one word, that word would be "slow". Not in the sense that the applications are unresponsive, but in the sense that getting set up as part of the community takes a certain amount of time and attention. Following a new friend means synchronizing their feed and any new friends-of-friends, which in turn requires fetching their entire history of posts. Media, such as photos and videos, is stored and shared out-of-band, so this isn't as much data as it could be, but it still takes minutes to hours, depending on volume and internet connection quality.

When new posts are made, they also don't propagate instantly. It's entirely possible, and reasonably common, for someone to read their friends' posts, write lengthy replies, share a picture of some isolated location, all offline. When they reconnect, it results in a sudden flurry of activity. That style of interaction tends to encourage asynchronous, long-form exchanges — although of course there are still plenty of people who post short, microblogging-style messages as well.

There are many federated social-media platforms; fully distributed systems are much rarer because of the real challenges introduced by not having a global consensus or feed. On the other hand, given the toxicity of modern social media, maybe having software that focuses on individual relationships and eschews the idea of requiring people to stay constantly connected to participate in online discussions is a good thing. Interested readers should refer to Scuttlebutt's documentation on getting started.