Automating font builds

At ATypI 2016 in Warsaw, Marek Jeziorek and Behdad Esfahbod presented the work that the font-tools team at Google has done to automatically build and test fonts from source. The project is increasingly of importance to Google's work on the Noto font project, which is intended to cover every writing system and language on the planet. As one would expect, building and updating fonts that cover so much ground is far beyond what developers can do manually.

Jeziorek started out by describing the team and explaining its mission. There are currently six engineers involved, plus three people who work in a management capacity. The team is responsible for the Noto font family, which means that it must answer to two product teams: Chrome OS and Android. In addition, the team develops all the tools needed to make Noto usable, and those tools are intended to be useful for other projects both inside (for example, to the Google Fonts project) and outside the company.

Noto is an abbreviation for "No more Tofu," Jeziorek explained, a reference to the empty-square fallback character seen when a font does not have the glyphs needed to display the necessary text. The goal is for Noto to provide a quality font for every writing system and language, and to do so in a style that is consistent and looks harmonious across all of those various languages. The main benefit will be to the billion computer users estimated to get online for the first time in the next few years, who do not speak a language written with the Latin, Greek, or Cyrillic alphabets.

The plans right now include more than 100 writing systems and 800 languages, although that could be adjusted. The Noto font family is designed to follow Unicode; if there are additions in future revisions, Noto will add more support to match whatever changes. For each language, there will be a serif font design, a sans serif, and "sans UI" that targets interface design rather than setting long text runs. All of the fonts are open source and are hosted on GitHub.

The big challenge, he said, is coordinating with all of the one-hundred-plus designers that create fonts for their local language. Google works with partners, namely Adobe for Chinese, Japanese, and Korean (CJK) and Monotype for other languages, but those partners frequently employ outside contractors. The result is that the Noto project pulls in work created by a lot of independent type designers.

Each new font project begins with a design proposal sent in as a PDF, Jeziorek said. Several iterations of design feedback tend to follow (focusing on matching the Noto look and feel as well as on readability), after which the designer begins working on the final product. The files delivered when the font is complete are in source form, which is then built and tested for completeness. At the moment, he said, 29 Noto families have been released (each containing several fonts as mentioned earlier, and each including several weight and width variants) that build into 638 TrueType or 638 OpenType font binaries. That amounts to a lot of testing.

Esfahbod then talked about the build pipeline. Three years ago, he said, many existing font-manipulation scripts and tools targeted the Unified Font Object (UFO) format, and he began working on the FontTools Python library to improve its support for UFO. The first task was to build a tool that would create a subset of an existing font, for example. At ATypI 2014, however, he commented that he intended to keep extending FontTools because the other major production pipeline, Adobe's Font Development Kit for OpenType (AFDKO), was closed-source. "Then David Lemon gave me the good news," he said, that AFDKO was being released as an open-source project (with one or two components as small exceptions), as were several related libraries.

So, for the first time, it looked like it was possible to create a full build pipeline using open-source software. When the team next renegotiated its contracts with the type designers, it asked them to provide UFO source files rather than binaries. Surprisingly, though, the replies were "we just switched to using Glyphs." Esfahbod talked to Glyphs developer Georg Seifert about the possibility of providing a scriptable Glyphs-to-UFO converter, and Seifert was receptive to the idea. But, in the end, Esfahbod just wrote the converter himself, since the Glyphs file format is well-documented on GitHub.

That new component made it possible to extract data straight from a Glyphs source file and pipe it into any of several available UFO-related tools, including the ADFKO. New engineers started working on additional pieces of the pipeline. James Godfrey-Kittle developed a library to use UFOs within FontTools; Sascha Brawer wrote a tool to bypass one of the few remaining binary components of AFDKO, which fixed overlapping contours in glyph outlines. Other pieces in the ever-growing Python toolkit include compreffor, which optimizes Compact Font Format (CFF) font tables for size, cu2qu, which converts cubic Béziers from a CFF font into the quadratic Béziers required for TrueType, a tool to do visual diffing between fonts, and much more.

The tool set eventually encompassed everything required to automatically build high-quality binary fonts directly from Glyphs source files. Nevertheless, there was no top-level process to coordinate all of the pieces, until Godfrey-Kittle wrote fontmake. As the name implies, fontmake is a full build system for font sources. The supported inputs include UFOs, Glyphs files, and several auxiliary formats like the designspace files used to define interpolation masters in MutatorMath. That library has been the industry standard to generate large font families from a small set of masters, although Esfahbod noted that the newly released OpenType 1.8 variations font feature may change a lot of workflows.

Up until recently, though, fontmake was useful for the Noto team but was not really friendly to outsiders. "We developers just want to write new code, and maybe fix a bug here and there," he said, not maintain the code. But that has begun to change, he reported. Brawer has developed testing and quality-assurance tools; Godfrey-Kittle and Cosimo Lupo have integrated fontmake with continuous integration, and Godfrey-Kittle has gotten the various dependencies under control, ensuring that fontmake works with a set of "known good" versions of the other libraries.

The result is a fully capable build tool, though Esfahbod noted that it is currently designed only to support Noto's precise build process. Other users will certainly want different or additional options. It is also embarrassingly slow in some places, he added, at least compared to C. "But that's not the point," he said. "The point is that it's faster for development. Font tooling is such a niche that there are not many people interested in developing for it, but Python is what that community wants to use."

More flexibility and stability are still to come, he said, but there are still features to be added as well. Work will need to be done to support the new OpenType 1.8 variations fonts, there are improvements to be made in how CFF operators are used, and there is still one last piece of the AFDKO that has no open-source equivalent: the PostScript auto-hinter used on CFF fonts.

At the pace that the tools team has been working, however, the wait might not be too long at all. For everyone outside of the Noto project, a set of open-source build tools can reap plenty of benefits, from enabling distributions and other font distributors to automate their build scripts to enabling font designers to make use of concepts like continuous integration that other corners of the development community have already found useful.