StatusNet, Identi.ca, and transitioning to pump.io

Evan Prodromou surprised a number of free software microbloggers in December 2012 when he announced that he would be closing down Status.Net, the "Twitter like" software service he launched in 2008, in favor of his new project, pump.io. But Status.Net's flagship site, Identi.ca has grown into a popular social-networking hub for the free and open source software community, and a number of Identi.ca users took the announcement to mean that Identi.ca would disappear, much to the community's detriment. Prodromou has reassured users Identi.ca will live on, though it will move from StatusNet (the software package, as distinguished from Status.Net, the company) over to pump.io. Since then, pump.io has rolled out to some test sites, but it is still in heavy development, and remains something of an unknown quantity to users.

Prodromou has some markedly different goals in mind for pump.io. The underlying protocol is different, but more importantly, StatusNet never quite reached its original goal of becoming a decentralized, multi-site platform—instead, the debut site Identi.ca was quickly branded as an open source "Twitter replacement." That misconception hampered StatusNet's adoption as a federated solution, putting the bulk of the emphasis on Identi.ca as the sole destination, with relatively few independent StatusNet sites. The pump.io rollout is progressing more slowly than StatusNet's, but that strategy is designed to avoid some of the problems encountered by StatusNet and Identi.ca.

The December announcement started off by saying that Status.Net would stop registering new hosted sites (e.g., foo.status.net) and was discontinuing its "premium" commercial services. The software itself would remain available, and site maintainers would be able to download the full contents of their databases. Evidently, the announcement concerned a number of Identi.ca users, though, because Prodromou posted a follow-up in January, reassuring users that the Identi.ca site would remain operational.

But there were changes afoot. The January post indicated that Identi.ca would be migrated over to run on pump.io (which necessarily would involve some changes in the feature set, given that it was not the same platform), and that all accounts which had been active in the past year would be moved, but that at some point no new registrations would be accepted.

Indeed Identi.ca stopped accepting new user registrations on March 26. The shutdown of new registrations was timed so that new users could be redirected to one of several free, public pump.io sites instead. Visiting http://pump.io/tryit.html redirects the browser to a randomly-selected pump.io site, currently chosen from a pool of ten. Users can set up an account on one of the public servers, but getting used to pump.io may be a learning experience, seeing as it presents a distinctly different experience than the Twitter-like StatusNet.

What is pump.io anyway?

At its core, StatusNet was designed as an implementation of the OStatus microblogging standard. An OStatus server produces an Atom feed of status-update messages, which are pushed to subscribers using PubSubHubbub. Replies to status updates are sent using the Salmon protocol, while the other features of Twitter-like microblogging, such as follower/following relationships and "favoriting" posts, are implemented as Activity Streams.

The system is straightforward enough, but with a little contemplation it becomes obvious that the 140-character limit inherited from Twitter is a completely artificial constraint. StatusNet did evolve to support longer messages, but ultimately there is no reason why the same software could not deliver pictures à la Pinterest or Instagram, too, or handle other types of Activity Stream.

And that is essentially what pump.io is; a general-purpose Activity Streams engine. It diverges from OStatus in a few other respects, of course, such as sending activity messages as JSON rather than as Atom, and by defining a simple REST inbox API instead of using PubSubHubbub and Salmon to push messages to other servers. Pump.io also uses a new database abstraction layer called Databank, which has drivers for a variety of NoSQL databases, but supports real relational databases, too. StatusNet, in contrast, was bound closely to MySQL. But, in the end, the important thing is the feature set; a pump.io instance can generate a microblogging feed, an image stream, or essentially any other type of feed. Activity Streams defines actions (which are called "verbs") that handle common social networking interaction; pump.io merely sends and receives them.

The code is available at Github; the wiki explains that the server currently understands a subset of Activity Streams verbs that describe common social networking actions: follow, stop-following, like, unlike, post, update, and so on. However, pump.io will process any properly-formatted Activity Streams message, which means that application authors can write interoperable software simply by sending compliant JSON objects. There is an example of this as well; a Facebook-like farming game called Open Farm Game. The game produces messages with its own set of verbs (for planting, watering, and harvesting crops); the pump.io test sites will consume and display these messages in the user's feed with no additional configuration.

The pump.io documentation outlines the other primitives understood by the server—such as the predefined objects (messages, images, users, collections, etc.) on which the verbs can act, and the API endpoints (such as the per-user inbox and outbox). Currently, the demo servers allow users to send status updates, post images, like or favorite posts, and reply to updates. Users on the demo servers can follow one another, although at the moment the UI to do so is decidedly unintuitive (one must visit the other user's page and click on the "Log in" link; only then does a "Follow" button become visible). But Prodromou said in an email that more is still to come.

For those users and developers who genuinely prefer StatusNet, the good news is that the software will indeed live on. There are currently two actively-developed forks, GNU social and Free & Social. Prodromou said there was a strong possibility the two would merge, although there will be a public announcement with all of the details when and if that happens.

Where to now?

Pump.io itself (and its web interface) are the focus of development, but they are not the whole story. Prodromou is keen to avoid the situation encountered at the StatusNet launch, where the vast majority of new users joined the first demo site (Identi.ca), and it became its own social network, which ended up consuming a significant portion of StatusNet's company resources. Directing new registrations to a randomly-selected pump.io service is one tactic to mitigate the risk; another is intentionally limiting what pump.io itself will do.

For instance, while StatusNet could be linked to Twitter or other services via server-side plugins, pump.io will rely on third-party applications for bridging to other services. Prodromou cited TwitterFeed and IFTTT as examples. "My hope is that hackers find pump.io fun to develop for," he said, "and that they can 'scratch an itch' with cool bridges and other apps." The narrow scope of pump.io also means that a pump.io service only serves up per-user content; that is to say, each user has an activity stream outbox and an inbox consisting of the activities the user follows, but there is no site-wide "public" stream—no tag feeds, no "popular notices."

That may frustrate Identi.ca users at the beginning, Prodromou says, but he reiterates that the goal is to make such second-tier services easy for others to develop and deploy, by focusing on the core pump.io API. For example, the pump.io sites forward all messages marked as "public" to the ofirehose.com site; any developer could subscribe to this "fire hose" feed and do something interesting with it. Ultimately, Prodromou said, he hopes to de-emphasize the importance of "sites" as entities, in favor of users. Users do not care much about SMTP servers, he said; they care about the emails sent and received, not about enumerating all of the accounts on the server.

That is true in the SMTP world (one might argue that the only people who care to enumerate the user accounts on a server probably have nefarious goals in mind), but it does present some practical problems in social networking. Finding other users and searching (both on message content and on metadata) have yet to be solved in pump.io. Prodromou said he is working on "find your friend" sites for popular services (like Facebook and Twitter) where users already have accounts, but that search will be trickier.

Identi.ca and other things in the future

Eventually, the plan is for Identi.ca to become just one more pump.io service among many; the decentralization will mean it is no harder to follow users on another pump.io server or to carry on a conversation across several servers than it is to interact with others on a monolithic site like Twitter. But getting to that future will place a heavier burden on the client applications, be they mobile, web-based, or desktop.

Prodromou has not set out a firm timeline for the process; he is working on the pump.io web application (which itself should be mobile-friendly HTML5) and simple apps for iOS and Android. In the medium term, the number of public pump.io sites is slated to ramp up from ten to 15 or 20. But at some point Prodromou will start directing new registrations to a free Platform-as-a-Service (PaaS) provider that offers pump.io as a one-click-install instead (AppFog and OpenShift were both mentioned, but only as hypothetical examples).

Where pump.io goes from there is hard to predict. Prodromou is focused on building a product developers will like; he deliberately chose the permissive Apache 2.0 license over the AGPL because the Node.js and JavaScript development communities prefer it, he said. Applications, aggregation, and PaaS delivery are in other people's hands, but that is evidently what he wants. As he explained it, running Status.Net took considerable resources (both human and server) to manage hosted instances and public services like Identi.ca, which slowed down development of the software itself. "I want to get out of the business of operating social networking sites and into the business of writing social networking software."

At some point in the next few months, Identi.ca will switch over from delivering OStatus with StatusNet to running pump.io. That will be a real watershed moment; as any social-networking theorist will tell you, the value of a particular site is measured by the community that uses it, not the software underneath. Identi.ca has grown into a valued social-networking hub for the free software community; hopefully that user community survives the changeover, even if it takes a while to find its bearings again on the new software platform.