Stem 1.3 makes hidden services easier to deploy

The Tor project recently released a new version of Stem, its Python library for monitoring and controlling a Tor connection. The release includes speed improvements and some new utilities, but the key feature is simplified tools for starting and running Tor hidden services.

Stem has been in development since late 2011. It replaced a older library called PyTorCtl that was designed for Tor's "bandwidth authorities"—a set of trusted Tor directory authorities that actively measure the bandwidth of the network in order to spread the load out as evenly as possible. The older library worked, but lacked the flexibility needed to be useful for many other Python projects. The 1.0 release of Stem arrived in March 2013, and there were 1.1 and 1.2 updates subsequently. 1.3 was announced on December 22, 2014.

Stem is available through the Python Package Index (PyPI) and there are packages available for many Linux distributions as well. Python 2.6 or newer is required, and the library works with Python 3.

The changelog for the new release highlights a handful of new additions. There is a considerable speed-up (on the order of 40%) when parsing the status documents sent out by Tor directory authorities and mirrors. Critically, the update also refreshes the list of Tor directory authorities itself—since Tor is expected to be most useful in environments where the network itself cannot be trusted, it may not be possible to reliably retrieve such a list over the Internet.

There are also new methods for querying the current state of the Tor connection. get_effective_rate() returns the maximum rate supported by the connected Tor relay, while connection_time() returns the Unix timestamp when the current connection was opened (if it is still open) or closed (if the current connection is closed). get_accounting_stats() returns a number of statistics (such as the number of bytes written and read) about the current connection in order to enable connection accounting.

Hidden services

But the big addition is a suite of methods for working with hidden services. For the unfamiliar, a Tor hidden service is a server program whose connection to the network is bound to a Tor node, and not to any port on a public IP address. The most common examples are running a web server or SSH server that is only connected to Tor (and, thus, is reachable only through the .onion pseudo-domain). Historically, setting up a hidden service was not an arduous process, but it did require editing several text configuration files and restarting the tor daemon.

The new method available through Stem is simpler. Stem's core module is the Tor Controller, which provides a high-level API for opening, configuring, and monitoring a Tor connection. Stem 1.3 adds four new methods that allow developers to work with hidden services directly through the Controller interface. They are:

• create_hidden_service(), which sets up a hidden service on the running Tor connection
• remove_hidden_service(), which shuts down a hidden service
• get_hidden_service_conf(), which return information about the currently running services
• set_hidden_service_conf(), which creates and sets up multiple services at once, based on a configuration file

Creating a new hidden service requires calling the create_hidden_service() method with at least two parameters: the directory path where the hidden service's files are located (for example, the web server configuration and data) and a virtual port number on which the service will be accessible (that is, the "port" on the Tor connection endpoint, analogous to a normal TCP port). One can optionally add two other parameters: the IP address of the host running the service (which, by default, is assumed to be 127.0.0.1) and the port number of the service (which, by default, is assumed to be the same port number as the virtual port parameter provided). If Tor succeeds in setting up the hidden service, it hands back the .onion address of the new service in return.

A hidden service can be shut down by calling

    remove_hidden_service(ServicePath,ServicePort)

create_hidden_service() returns a dictionary containing the configuration of the newly created service. When get_hidden_service_conf() is called, it will return a dictionary including the state of all active hidden services. Conversely, calling set_hidden_service_conf() with a dictionary will launch all of the services detailed in the dictionary, so it is fairly easy to save and restore the state of a configuration even when managing multiple services.

In earlier Stem releases, the library was usually promoted as a way to automate the connection set-up and tear-down process—much like Tor's now-discontinued GUI tool Vidalia would do. While it is certainly nice to be able to monitor the status of a Tor connection, this new hidden-service functionality may have more wide-reaching impact.

In the past few years, Tor has been a wild success story in many places around the globe, but that success has almost always been on the "read" site of the network pipe—in other words, Tor allowing a user to access the Internet with privacy and anonymity. Hidden services are meant to be the flip side of that coin: allowing users even in hostile environments to publish web sites and provide network services of their own. Stem 1.3's advancements in this area may not make running hidden services trivial, but they do at least provide a means for other software projects to make hidden services more available to Tor users.