Roundup: managing issues for 20 years

The Roundup Issue Tracker is a flexible tool for managing issues via the web or email. However, Roundup is useful for more than web-based bug tracking or help-desk ticketing; it can be used as a simple wiki or to manage tasks with the Getting Things Done (GTD) methodology. The 20th-anniversary edition of Roundup, version 2.1.0, was released in July; it is a maintenance release, but there have been a number of larger improvements in the last year or so. Here we introduce Roundup's features along with the recent developments that have helped make Roundup even more useful for tracking issues to their resolution.

Background

Roundup manages one or more "trackers", which is where the issue handling happens. Roundup and a tracker provide web, email, and command-line interfaces for interacting with issues. Each tracker managed by Roundup has its own users, schema, interfaces, and behaviors. The Roundup core code enforces few limitations on a tracker; one could support email notifications while another disables notifications. Trackers exist for managing software bugs and/or feature requests, sales leads, projects, conference-paper submission and review, interaction with a customer, and more.

A tracker "template" is an un-deployed tracker; it includes the various pieces that make up the tracker, including database schemas, configuration settings, HTML template files, library routines, initial data, enhancements to the Roundup core, and more. The roundup-admin command-line tool is used to deploy and activate templates. Roundup provides three different templates for software development projects. They add properties for tracking software version, dependencies between issues, and more. These templates address use cases similar to those of Bugzilla, Request Tracker, or Jira.

The objects in a tracker's schema store specific information. For example, the required user object stores properties (email addresses, login name, real name, password, etc.) about the people who interact with the tracker. An issue object stores a title, list of messages, list of attachments, and the "nosy" list of users who should be notified about issue updates. It also stores information about the creation and last change to an issue.

Roundup's schema is specified in Python. The hyperdatabase (hyperdb) layer implements the database handling, which automatically deals with schema changes by modifying the underlying database. It also enforces role-based access control using its permission model, which provides fine-grained control down to the property level.

Creating a tracker based on the classic template, for example, adds more issue properties. These include the user responsible for resolving the issue, the current priority selected from a list of priorities, the current status from a list of statuses (e.g. unread, chatting, need-eg, resolved), and a list of keywords/tags (selected from a list) to categorize the issue. All of these predefined lists are editable from the web interface.

A tracker processes HTML template files to generate the web interface; it should be noted that there are two distinct uses of the word "template" for Roundup, tracker templates for creating trackers versus the HTML variety mentioned here. The HTML generated from the templates is customizable to mirror changes to the schema; the operations available in the web interface can be enhanced, as well. For example, an action can be added to export a query result as a comma-separated value (CSV) file, which provides information for further analysis in a spreadsheet or database.

Tracker behaviors are not hard-coded in the core. Bits of Python called auditors and reactors add behaviors. Auditors run before changes are committed to the database to validate changes like user login names, manage the notification list, prevent tracker spam, or implement a status workflow. Reactors run after changes are committed to the database. They send notifications, update RSS feeds, and can trigger calls to webhooks on other systems. More examples are available on the wiki and in the source repository.

Roundup is easy to deploy. CGI, WSGI, Zope, or standalone server are all supported deployment methods. For low-traffic sites, using an embedded database simplifies deployment. For larger sites, Roundup can use single sign-on when placed behind a web server like Apache or NGINX and can also integrate with LDAP or Active Directory. Because trackers are easy to deploy, organizations can run more than one of them to support different sets of users or to track different types of issues.

Roundup users

Many different projects and organizations have used Roundup for more than a decade. DARCS version control has been using it for 16 years as a bug and feature tracker. For 15 years, Cendio AB has run its customer support help desk using Roundup. Intevation GmbH has used Roundup for 19 years; the company runs multiple trackers for software development (some with scrum-like support), customer relationship management, internal help desk, and customer support (via email only or email and web access).

In addition, the Department of Laboratory Medicine and Pathology at the University of Washington Medical Center has run the tool for 13 years to handle problem reporting, tracking computing resources, customer support, and recording chemistry instrument quality assurance data. The Python Software Foundation (PSF) used Roundup for Python bug and feature tracking starting in 2006. But the PSF is now transitioning to GitHub Issues since Python's version control migrated to GitHub in 2017. Lastly, Roundup itself uses it for bug and feature tracking.

As mentioned, people interact with Roundup using the web interface, but REST and XML-RPC interfaces are also available. Roundup accepts issue updates via email, which can be PGP signed; Roundup is able to verify the signatures and can also sign email that it sends. The email interface allows changing an issue's properties in addition to adding a message or attachments. The roundup-admin command-line shell and Python language support allow users to write scripts for local interaction and automation.

Origins

Ka-Ping Yee designed Roundup. His design won the "Track" category of the Software Carpentry competition in 2000. In 2001 Richard Jones was looking for a new project and chose Roundup.

Roundup was originally written in Python 2, but currently supports Python 2.7.2+ or Python 3.4+ using only the Python standard library. Extra functionality (improved full-text search, markdown support, email encryption, better HTTP compression, etc.) is enabled by installing additional modules.

Originally, Roundup used the dbm key-value store for data storage. Relational database support was added later. Today it runs with Berkeley DB, SQLite, PostgreSQL, or MySQL/MariaDB.

Code from the Zope templating attribute language (TAL) processes templates to generate HTML. Today Roundup HTML templates support Zope TAL, Chameleon (an alternate TAL implementation), and Jinja2.

Bundled HTML templates use basic HTML and CSS; the goal is to allow customization by administrators who are not developers. The customizing document provides many examples for administrators to customize the web interface. JavaScript provides progressive enhancement and isn't required for core functionality; this provides a low barrier for users, who can even use a text-based browser, to interact with Roundup.

Recent development

Around 2011, Jones stepped back as the primary developer; Ralf Schlatterbeck and Anatoly Techtonik stepped in to update and release Roundup. Four years ago, I became the release engineer and a Roundup developer.

Two improvements significantly modernized Roundup. The 2.0.0 release in July 2020 added Python 3 support, which was driven by Joseph Myers with help from Christof Meerwald. The changes were made using the method suggested by Peter A. Donis and Eric S. Raymond, which allowed the code base to support Python 2 (for backward compatibility) and Python 3.

The second major improvement in that release added a REST API to complement Roundup's existing XML-RPC interface. Chau Nguyen, who was mentored by Ezio Melotti, developed it as a Google Summer of Code (GSoC) project for the PSF in 2015. Schlatterbeck merged the code into the core in 2020 for the 2.0.0 release. Features to better support JavaScript web frameworks like React, Angular, and Vue were also added. These include entity tag (ETag) support, field embedding, pagination, and Post One Exactly. Extending the REST interface allows custom support for front-end applications; for example, that can be used to render markdown for the front end.

Adding JSON Web Token (JWT) support using PyJWT provides a mechanism to integrate third-party applications with a tracker. A JWT authenticates the user as well as limits their permissions. For example, it could limit a third-party time tracking application to only creating time log entries on issues. The user interface has also seen improvements. Meerwald and Cédric Krier added support for formatting issue updates using Markdown to the jinja2 template. SimpleMDE provides a WYSIWYG-like editing environment for updates. The server then converts the Markdown to HTML for display. Although it has a few rough edges, the Markdown support works well.

The current development version has a patch that makes attaching files easier. Roundup has always supported file attachments using the standard HTML file input, but the patch would allow files to be attached using drag and drop. In addition, images (e.g. screenshots in the clipboard) can be pasted into the issue. While this work was not complete in time for the 2.1.0 release, it is running on the Roundup tracker to get feedback. Hopefully, the work will be part of the bundled tracker templates in version 2.2, which will likely land in July 2022.

Roundup's development version also compresses web responses to improve performance. A front-end server can compress Roundup's responses, but setting this up is easy to overlook (in fact bugs.python.org did not have this configured). A feature request was filed; now the development code compresses dynamic and static data (CSS, JavaScript, and so on) using gzip. Alternatively, Brotli or zstd compression can be used, but those require packages from PyPI.

Getting involved

Roundup can be installed using pip. Running roundup-demo will install and start a demonstration tracker. If you want to try Roundup without installing it, you can download and unpack the tarball, then run python demo.py. The roundup-users mailing list is the primary resource to get help in deploying and using Roundup. Help is also available on the #roundup IRC channel, on irc.oftc.net.

Anybody can contribute to Roundup. You don't need to be a professional developer. Many improvements have come from Roundup users and administrators. My background is system administration, not development, but in 2002 I needed a tracker to coordinate work at my job and Roundup fit the bill. I decided to implement the multiple notation methods from Request Tracker for Roundup. While learning Python and Roundup, I was able to create this with 20 hours of effort. Since Roundup was at version 0.5 this included debugging problems in the core code.

People with writing skills can help with Roundup documentation. The documentation is extensive but grew organically; while the answer to a question is often in the documentation, finding it is another matter. Efforts to restructure, index, and edit the documentation are more than welcome. For those interested in smaller documentation tasks, look to the Roundup wiki; indexing and copy editing the wiki will improve the user's experience. Roundup currently has translations for 11 languages; additional translations and corrections to existing translations are welcome as well.

The HTML templates are being changed to work with a strict Content Security Policy (CSP). Also, the look and feel of many HTML templates are stuck in the early 2000s. Redesigning the web interface to be responsive and accessible with a modern look and feel would address a common complaint. Extending support for non-SQL databases beyond the current dbm key-value store was suggested a few years ago. Enhancing Roundup to support MongoDB, Couchbase, or LMDB would be a useful addition.

You can browse the issue tracker for starter tickets; many more opportunities of items to work on exist in the Roundup issue tracker. The Roundup development mailing list (roundup-devel) is the place to go for information on developing Roundup. Those looking for an issue tracker for their project should give Roundup a try, while those who are interested in getting involved will find plenty of opportunities within the Roundup project.