Chromium and distributions

In 2009, Tom "spot" Callaway famously listed a number of reasons why Chromium was not officially packaged for Fedora. Five years later, that post was the impetus for a talk by Google's Paweł Hajdan, Jr. at LinuxCon Europe in Düsseldorf, Germany. Things have gotten far better since Callaway's post, but there is still more that both Google and the community can do to improve the situation.

The Chrome browser (with its open-source counterpart: Chromium) was announced in 2008 and was met with "excitement in the Linux community", Hajdan said. The "main goal" of his talk "is to connect the two communities"—Chrome/Chromium and Linux. There is "some room for improvement" by both the project and the distributions. Providing the best solution for users is "going to require the cooperation of both communities".

There are roughly 150 open-source packages that go into Chromium, so the project has a "really strong foundation" of openness, he said. Unlike Android, all commits to Chromium go into an open repository. There are also project committers who are not Google employees. Chromium has been open since the first public release, and it is getting more open over time.

Complaints

Callaway's 2009 post is "the basis of my talk", Hajdan said. That article still comes up as one of the first results when searching for information on Chromium and Fedora. Hajdan said that he agrees with many parts of that post, but that quite a few things have changed since 2009.

Back then, Callaway complained that Chromium was not stable. There were no releases of Chromium in those days, Hajdan said, the first was made in May 2010. Now there are regular releases. In 2009, code was pulled directly from the Subversion repository, but since 2011 there have been tarball releases—though, at roughly 200MB, they are rather large.

Beyond that, patches from packagers were not getting merged into the Chromium base, so it might have taken up to two weeks for a packager to make a new release for a distribution. These days, patches from packagers are being merged, so it should only take half an hour or so to package the browser up. Distributions are aiming for fewer patches to apply, but there are some 10 million lines of code in Chromium, so there will likely always be some patches, he said.

The other main complaint from 2009 was about "forked libraries". Chromium uses a lot of third-party libraries, which is not uncommon, but Linux programs tend to assume that packagers will handle any library problems. Chromium is a cross-platform program, though, and there is "no concept of system libraries" on Windows and Mac OS X. In the end, Linux users are a minority, and the libraries have to be in the Chromium tree anyway to support the other operating systems, so Linux was also handled that way.

But the bundled libraries are not placed haphazardly into the Chromium tree. They all live in a separate "third_party" directory where it is easy to find them. In addition, there is metadata attached to each of the libraries to make it easier to determine their origin, license, and so on. To "some degree", there is truth to the statement that Chromium has a lot of bundled libraries, Hajdan said, but partly that perception comes from the fact that the libraries are "more visible than in other projects" because they are all in one place.

According to Callaway, system libraries are not supported by the Chromium upstream, but that is something of a misreading of the situation. Chromium is big and multi-platform; it does not foist problems off on users simply because they are found in some third-party library. "If it affects users, we can and do fix it", he said.

Hajdan noted that, as with other big projects and big companies, there are a variety of opinions on any subject, including library handling for Chromium—both inside Google and the project. He is presenting his, which may well not be the official policy. The various rebuttals that Callaway had heard for his complaints and described in his post are also just opinions. But some of them do have some merit, Hajdan said.

For example, the Chromium project does move fast, which makes it difficult to work with upstreams. Some 200 patches per day land in the repository. He is on the infrastructure team that is supporting that project; "we have to work hard not to crumble under our own weight", he said. But, since the project is moving so fast, it actually tries to minimize the differences it has with upstreams. In general, working better with upstreams is a goal for the project.

Specific libraries

There is also the need to use specific versions of libraries, but that could be handled better in some cases. He gave the "reverse talk" at Google at one point, presenting about how the open-source community works. It turned out that many inside the company did not know that a package can request that the package manager provide a specific version of a library. But there are times when "deep changes" to a library are necessary. Typically, that is done to support the Chromium sandbox that prevents the rendering processes from making system calls that they shouldn't.

He pointed to Gentoo stable as evidence that newer libraries can work just fine for Chromium. The distribution has newer versions of some important libraries (e.g. jinja, libevent, libpng, libxml, and zlib) and the browser "still works". Distributions like Gentoo are helping the Chromium team by doing a lot of testing with these newer libraries; he would like to see the project give recognition to Gentoo and others for that work.

Some notable libraries that are bundled with Chromium include FFmpeg, Hunspell, Mesa, and SQLite. FFmpeg is used to handle HTML5 audio and video for the browser. The Chromium project does work with the FFmpeg upstream and newer versions of FFmpeg "technically could work" for the browser.

On the other hand, Chromium uses an incompatible fork of Hunspell that has a different dictionary format. Some of the changes have made their way upstream, but the Hunspell project is sometimes slow to respond to patches. It would also help if Hunspell would remove some unnecessary obstacles to working with others, he said, including moving away from using CVS as its source code management tool.

The Mesa library is a fork due to some type mismatches for 32-bit systems. Upstream Mesa can be made to work for Chromium on 64-bit systems, he said. At one time, Chromium was using a patched version of the GLEW OpenGL wrapper library, but it doesn't support some features that are needed (e.g. GLES2 and the sandbox).

The Chromium version of SQLite is also an incompatible fork, he said. SQLite is unlikely to make the changes Chromium would need; likewise, Chromium is probably not going to change to use the upstream version.

There is also a handful of libraries included that are not designed to be reused. For example, JavaScript libraries like Polymer and JSZip are meant to be distributed with a project's source. There are libraries that do not provide a stable API, such as libyuv and Skia, which are also meant to be included. The blocking issue for some of these libraries is that the developers "don't have the cycles to do a stable API", he said. By not having to worry about compatibility, those projects can move faster.

There is another handful of libraries that still need more work before they can be unbundled from the Linux version of Chromium. But he listed 17 libraries that had been unbundled over the last few years, including those for bzip2, HarfBuzz, ICU, and Speex.

In fact, there is now a directory in the Chromium tree (build/linux/unbundle) that has tools and information to assist in unbundling libraries from the Chromium builds for Linux. It is currently being used by Arch Linux, Gentoo, and some of the BSDs, but he encourages other distributions to use it as well. The remove_bundled_libraries.py script takes a list of libraries from the third_party directory on the command line and removes them from the build. On Gentoo, there are more than 100 libraries listed for the standard build. In addition, there is a mechanism to generate shim header files that include either the third_party version of the header or the one from the system, which facilitates unbundling.

As usual, contributions are welcome, Hajdan said. There are many contributions from outside Google and he encouraged attendees to "jump into the code base and make it better". There is a contributor agreement, but it is not a copyright assignment; Google does not want it to be a barrier to contributions. So far, FreeBSD and OpenBSD have gotten changes for those systems upstream, as have MIPS developers for their architecture. As he said at the outset, work is needed from both sides to bring Chromium smoothly into the Linux ecosystem.

[ I would like to thank the Linux Foundation for travel assistance to Düsseldorf for LinuxCon Europe. ]