LGM: GIMP's new release, new über-core, and future

GIMP took center stage at Libre Graphics Meeting (LGM) 2012 in Vienna. Day three featured a block of GIMP-related talks that saw the official release of the new stable 2.8 version of the application, plus a look at three new developments that will impact the future of the raster editor in the coming months. The biggest change is that the 2.9 development series has already been ported to the generic graphics library (GEGL) engine — the ease of which reportedly surprised even the developers — but there were interesting revelations about GPU-accelerated image processing and a new take on text handling, too.

FLOSS historians may recall that the very first LGM evolved out of a GIMP developers' summit. Still, in the past few years other application projects have grabbed the spotlight — Blender, Krita, Inkscape, and Scribus, for starters. Part of the reason has been GIMP's slower development cycle over the past few releases; as a long-established project it has a sizable codebase to maintain, and considerable effort in recent years has gone into building GEGL, the next-generation image processing library designed to bring high bit-depth image support, non-destructive operations, and other such niceties. Planning that transition while simultaneously implementing feature requests on the existing core added up to some multi-year waits between major revisions. The previous stable release, 2.6, landed in October 2008.

Thus it was a minor surprise when GIMP maintainer Michael Natterer and GEGL maintainer Øyvind Kolås announced the official release of GIMP 2.8 during their talk. Technically, news of the release had leaked out the day before when the files appeared on the official FTP site, but the announcement was still unexpected good news. Feature-wise most of the new work in GIMP 2.8 was in place when we covered the 2.7.3 development build in November 2011. The headlines include the option of docking all of the tool palettes onto the image editor to function as a single window (with tabs for keeping multiple files open at once), layer groups, on-canvas text editing tools, a new "cage transform" tool, and numerous enhancements to the layout and manipulation of of tool settings.

The GEGL has landed

Upstaging the release of 2.8 was Natterer and Kolås's session showcasing their recent work porting the GIMP development tree to GEGL. GEGL has been slated to replace the legacy GIMP core for close to a decade, and although recent GIMP releases have included an option to activate GEGL for specific functions (such as color transformation), as recently as December 2011 the official plan was still for integrate other work (such as Google Summer of Code projects) into GIMP 2.10, and make the transition to GEGL in GIMP 3.0. The GEGL transplant had been long-planned, the maintainers said, but it took both of them being in the same room at the same time to jump-start it. As they reported, when that happened almost accidentally in March, the port took off, and is now more than 90% complete.

As Natterer explained it, Kolås was in town for a week-long hacking session, and the two decided to attempt some GEGL porting just to verify their planned approach. Kolås added a GEGL feature that used GIMP's existing image-tile storage as its back-end. Natterer patched in the new feature, and it immediately worked so well that he began cutting out other bits of legacy code to replace it with GEGL buffer manipulation. The more legacy code he ported to GEGL, the more fifteen-year-old layers of abstraction in the existing code base simply "collapsed away". In addition to simplifying and shrinking the code, he said, migrating tile manipulation to GEGL buffers made it possible to replace many image filters (such as blurs) with primitive GEGL operations. The two continued to work, and roughly a month later, merged the result into the GIMP trunk. The 2.9 code not only replaces GIMP's core process with a GEGL engine, but it provides a separate GEGL engine for GIMP plugins. In 2.10, the legacy plugin API will be officially deprecated.

In addition to telling the tale of its development, the two also demonstrated the GEGL-backed GIMP 2.9 live. First, they showed how compressing the range of an 8-bit-per-channel image resulted in serious color-banding. Then, they switched to 16-bit-per-channel mode, re-compressed the same image, and restored it to its original look without any discernible banding or quality loss. Another demonstration illustrated how 16-bit-per-channel mode allowed similarly higher-quality painting and gradients. Those examples are easily visible, but in practice the benefits of high bit-depth editing are not so immediate; rather, the errors accumulate over several steps — but they do accumulate, with every operation done in 8-bit mode.

There are still other benefits to the new image pipeline, however. Because GEGL operations are defined on abstract buffers, adding support for an entirely different image format is a matter of writing a new format for babl, the underlying pixel transformation layer. During the GEGL hack-a-thon, Kolås wrote such a back-end for indexed color images (such as you find in the GIF format). Natterer had originally planned to drop support for indexed images, but with the babl format defined, they work just as well as any other format in the GEGL-ified GIMP. GEGL also allows GIMP to use all sorts of painting and filter operations on indexed images (such as smudging and blurring) that are typically not possible on indexed color.

The GEGL transition is still not complete; in particular UI elements frequently mismatch when in high-bit depth mode. For example, the levels and histogram sliders still show 0-255 as the range, rather than 0-100% (or some other bit-depth-neutral label). But the brave can already experiment with GIMP 2.9. More work remains to adapt GIMP's filters and plugins to GEGL, some of which will be tackled by 2012 GSoC students. But in addition to its practical usage for the GIMP project, Natterer pointed out that the ease and success of the GIMP port demonstrates that GEGL is stable, fast, and demonstrably useful for "arbitrarily complex" applications.

Kolås went into more depth on the inner workings of GEGL, which has the potential to serve as the graphics engine for other editors (and indeed there is some work underway on GEGL usage in the MyPaint project). It could also support vector and path operations, or be used in other application classes altogether, such as map transformations. Because GIMP's plugin engine is separate from the main application core, it, too, could be usable by other applications, allowing them to call GIMP plugins without requiring them to adopt GEGL itself.

The modern sounds of the GIMP

The GEGL integration talk joined two other presentations about GIMP development. Peter Sikking and Kate Price of Man Machine Interface Works (MMIworks) presented a glimpse at their still-in-progress work to revamp the editor's text handling capabilities. MMIworks is a user interaction firm that methodically investigates UI challenges and develops solutions; their past work with GIMP includes the new scaling and transformation tools and the single-window-mode implementation.

MMIworks set out to re-design the text manipulation tools for GIMP hoping to unify the sometimes conflicting uses of text in GIMP images: some text elements are part of the design, while some are used as annotations; some text is high-resolution typography destined for print, while other text is small and used for icon creation. In addition, GIMP users frequently apply multiple filters and effects to text, but want it to remain editable, which complicates the implementation.

Further complicating the task is one feature many users want but that is not yet implemented in GIMP: the ability to warp text along a path. Exactly where that warping takes place is another variable: at the top of the line, the bottom of the line, at the midpoint, or perhaps at the x-height. In addition, they said, there is no reason that text-warping could not be applied to the margins or the alignment of a paragraph. Throw in the fact that there are four possible writing directions, and text manipulation amounts to a substantial challenge.

The two did not have a final solution to present, but they did provide hints as to the direction the GIMP is heading. While experimenting with how to re-implement text manipulation from scratch, they decided that many of the parameters associated with text manipulation had features in common with vector or path editing tools. As a result, GIMP's eventual text-manipulation tool will be integrated with its path tools. As they explained it, whether the artist starts with a line of text and warps it to a path, or starts with a path and attaches text to it, the tools should offer the same control.

Victor Oliveira presented his work porting GEGL and GIMP functionality to the OpenCL framework. OpenCL is a cross-platform language for writing code blocks (called kernels) that can be executed on either CPUs or on GPUs (as well as other resources, such as FPGAs). The specification is managed by the Khronos Group (most widely known for OpenGL), and there are Linux implementations for all of the major GPU architectures as well as the Mesa software library.

Previous work has already ported specific GEGL functions to OpenCL; Oliveira ported more functions, including colorspace conversions. He also implemented device-to-device data transfer, wrote a filter API, and benchmarked OpenCL performance. The resulting work is already available, it was merged from the opencl-ops branch in GEGL 0.20 and GIMP 2.8 — Oliveira estimated that it encompassed around 15,000 lines of code.

For systems without a GPU-based OpenCL implementation, the OpenCL core also "automatically" makes GEGL multi-threaded, so there are still performance increases on multi-core CPUs. Some of the biggest gains, he said, come in OpenCL-based color transformations, which were historically a bottleneck. For any GPU-based system, however, the speed-up is quite significant. OpenCL is particularly fast with floating-point arithmetic, which is at the heart of GEGL's high-bit-depth operations. Thus, the more GEGL is integrated with GIMP (such as in the 2.9 series), the faster GIMP will become on multi-core and GPU-accelerated machines.

There are still more operations that could be ported to OpenCL, he said, but the next big step is to get more users to test the OpenCL code on a wider variety of CPU and GPU combinations. Intel, AMD, and Nvidia each have their own implementation, plus the open source Mesa implementation, which adds up to a lot of possible set-ups. Oliveira has already done some caching work to overcome the latency caused by transferring pixels from the CPU to the GPU and back; he suggested that adding an OpenGL output node to GEGL would further speed things up by allowing the image to be rendered directly on the graphics hardware rather than making yet another round trip.

GIMP interpretation

Together, the release of GIMP 2.8, the sudden port of GIMP 2.9 to GEGL, and the immediate availability of GPU-accelerated operations add up to more GIMP news this year than in the previous few LGMs combined. Granted, true GIMP fanatics have probably been running development builds for many months, but for everyone who depends on distribution-provided package, GIMP has taken a big jump forward.

Looking to the future, MMIworks always produces solid tools, and the rebuilt text-manipulation tools suggest that more original features are still in the works — warping text along the x-height line or paragraph-alignment line is certainly a new idea. Taken together with GPU accelerated operations, GIMP is moving forward on the user-facing tools and on the back-end simultaneously. But the most significant news is perhaps Natterer's comments at how much smaller and easier to maintain the GEGL-based GIMP is. As an aside during his talk, he mentioned that the simplified core is easier for new developers to dig into and understand. For an open source project, that is one of the best features one could wish for.

[Thanks to the Libre Graphics Meeting for assistance with travel to Vienna.]