Raw photo development with darktable

Raw images

Digital cameras normally produce their photographs in a convenient and compressed image format, most often JPEG. But, just as our brains perform a great deal of invisible processing to make our eyes work as well as they do, digital cameras do a lot of work between the sensor and the storage device. The data recorded by the sensor does not make for a particularly satisfying image until it has been through exposure compensation, white-balance adjustment, noise removal, sharpness adjustment, and even, with some devices, advanced manipulation like high-dynamic-range processing. The results from today's cameras can be amazingly good, but they invariably encapsulate a set of processing decisions and loses information from the original image.

Professional photographers — and hobbyists with an inflated sense of the quality of their snapshots — often choose to do that post-processing themselves. For such people, cameras can be convinced to write the raw data from the sensor (or something close to it) directly to a file in a format that is normally referred to as "raw" (or "RAW" even though it is not an acronym). Photographers can work with raw files to take full advantage of all of the data collected by the sensor and apply their own preferences — if they can get at that data. Naturally, every camera produces raw data in its own special, proprietary format. Fortunately, the free-software community has proved skilled at understanding these formats and writing decoders for them, so there is support for raw images from most available cameras.

Taking photographs in raw format implies a commitment to put some time into "developing" them into a more useful form; that is what a raw image editor is for. There are several of these editors available, including darktable, which is where your editor chose to start. In the name of "research", an afternoon was taken off work to wander in the local parks in search of a suitable raw image to process with darktable; the result is shown on the right. This image is a JPEG thumbnail created by the camera and stored with the raw image, extracted with dcraw. It is not a great work of art by any means, but it is useful to experiment with: there is a lot of contrast, and the rock formation (known as the "Second Flatiron") is somewhat lost in the haze that was present that day. This picture might just benefit from the sort of manipulation a raw editor can perform.

Running darktable

Launching darktable for the first time is a bit of an intimidating experience. It presents two columns of controls, most of which are hidden behind at least one layer of tabs; there are literally hundreds of things that one can tweak. Fortunately, the project has put some time into its documentation, which includes a set of steps to follow to learn how to make basic edits.

The first step is to import images into the application. Darktable can maintain its own directory full of images, but it is also happy to work with image files stored anywhere in the filesystem. There is a module available to import raw images directly from a USB-connected camera for those who prefer to work that way. In all cases, the end result is the "lighttable" screen, where thumbnails of all of the imported images are displayed.

The next step in the suggested workflow is to assign star ratings to each image in an iterative process, eventually eliminating all except for the best ones. Simply hitting "2", for example, marks the image under the cursor as a two-star photo. Images below a certain number of stars can be hidden (or deleted outright), allowing the photographer to focus their effort on the best prospects. Users should remember that this is not a ride-sharing app, and so the full range of star ratings can be used without ill effect.

Then it is time to pick an image and go into the "darkroom" to process it into its final form. Darktable uses a pipeline model (the "pixelpipe") for this processing; there is a long list of modules that can perform specific tasks in this pipeline. These modules are arranged in a series, with each module operating on the output of the one that came before. As an image is passed through this pipeline, it is transformed from the raw format into (hopefully) what the photographer wants it to be. While there is a logical order in which to use many of the modules, darktable allows them to be ordered arbitrarily, and multiple instances of the same module can appear in different places in the pipeline.

As one studies those modules it becomes apparent that darktable is currently going through a major transition in how the pixelpipe is expected to work. Prior to the 3.0.0 release in at the end of 2019, darktable supported a processing mode called "display-referred"; thereafter, the "scene-based" workflow is used instead. The difference comes down to when the color mapping is performed on the raw image.

The dynamic range of the data in a raw file is greater than can be displayed on the screen or represented in a JPEG file; that is part of why photographers use the raw format. At some point, though, the dynamic range must be reduced so that the image can actually be displayed. The display-referred workflow performs that processing near the beginning of the pipeline, with all subsequent modules working in the display color space. In the scene-based mode, the color mapping is deferred until the end, after other processing has been done.

The advantage of the scene-based mode is that it retains the full dynamic range of the raw image for as long as possible, leading to better results at each step in the pipeline. The change makes sense, but even in the current darktable 3.8.0 release it is clearly incomplete. Many of the provided processing modules only work in one mode or the other, but they are all jumbled together in the interface (and the documentation) so the user must sort them out. If scene-based is the way to go, darktable would do well to simply hide all support for display-referred work by default.

Processing the image

The darkroom view is made up of three columns, with the subject image taking up the middle column. On the right, below the color histogram, is the set of processing modules, which can be displayed in a number of ways; clicking on any given module will open a pane with, potentially, several tabs of adjustments to tweak. The left column contains another set of modules, such as the color picker and image exporter, that don't operate directly on the image.

The first step with a new image is to set the overall exposure. The exposure module can be used to compensate for a photograph that was under- or overexposed in general. Indeed, the darktable documentation suggests slightly overexposing photographs ("exposing to the right") and correcting the exposure in postprocessing.

Adjusting the exposure is a simple matter of moving a slider, but one quickly encounters the complexity of the darktable interface — there are five different ways to adjust a slider. Left-clicking on the range will set the slider cursor to the indicated spot. One can use the mouse scroll wheel to make incremental adjustments; the keyboard arrow keys work in the same way. A right click on the slider, instead, pops up a two-dimensional control that can be used for extreme fine-tuning. Finally, double-clicking on a slider resets it to the default value.

The next step goes to the far end of the pipeline and the filmic rgb module, which performs the color mapping for the display. Here one can set the white and black points for the image, picking the portion of the dynamic range that will be used to create the final result. Many other adjustments can be done here, including contrast tweaking, but the intent isn't to get a final image yet.

Moving back to the earlier parts of the processing pipeline, the color calibration module can be used to correct the white balance of the image if the value chosen by the camera (and stored in the raw file) is incorrect. Setting the white balance can be as simple as clicking on a part of the image that should actually be white; as it happens, pictures with snow in them make this task easy. This module can perform much more sophisticated changes, though. For example, like almost all processing modules, it can use a mask to only operate on part of the image; if one has an image needing a different white balance in different regions, the color-calibration module can be employed twice, with masks, to get the job done. Changes far beyond white balance are possible as well.

Your editor played with these modules for a while, and also brought in the haze removal module to clean up the background, with reasonably good effect. At this point, though, something jumped out: the exposure modifications did a good job of bringing out the detail in the image, but at the cost of severely overexposing the sky, turning the Colorado blue sky white. Going back to the exposure module and playing for a while with parametric masks rectified this problem, in a crude manner at least.

A final step was to use the watermark module to add a tasteful LWN logo to the image. After that, one goes into the export module, which can be used to render the final image to disk. All of darktable's editing is non-destructive; the relevant data is kept in a "sidecar" file next to the pristine raw image. The export module will run the pipeline, then convert the result to the desired image format. Naturally, there is a wide range of options controlling just how the image should be exported. There is also a separate print module for getting images to paper.

After all that work, your editor ended up with the image shown to the right. This result, your editor's first with darktable, bears more resemblance to a preschooler's finger painting than actual art, of course. The color of the rock still is not right, and there are numerous artifacts along the skyline, for example. But it is an illustration of the sorts of things that can be done by even a naive darktable user.

There is, of course, a lot more there for the advanced user. Various modules can perform highly specialized processing in ways that, it seems, that only a user with a PhD in color management can understand. There is a whole mechanism for remembering the configuration of collections of modules that will clearly be useful for photographers who end up doing the same sort of processing repeatedly. If that's not enough, the whole thing can be scripted in the Lua language.

Community and conclusion

Darktable is licensed under version 3 of the GNU General Public License. The program is written mostly in C, with a handful of C++ files tossed in for good measure; it uses the GTK library for the user interface. The most recent release is 3.8.0, which came out in late December; it added a new keyboard shortcut system, a new diffuse/sharpen module, reworked perspective correction, a new demosaic algorithm, support for the Canon CR3 raw format, and more. The project produced one other major release (3.6.0) in 2021 as well, and seems to keep two-release-per-year cadence.

In the time since the 3.4.0 release (just over one year), the project has merged 4,071 changesets from 125 developers, so it does appear to have a reasonably wide contributor base. Of those developers, Pascal Obry was responsible for nearly 22% of those changesets, with Ralf Brown contributing 11%; the top five developers contributed just over half of the changes. The core of the project's community is thus small, as is often the case, but it is definitely not a one-person show.

The project is centered around GitHub, with most of its communication happening there. It does have a development mailing list but makes relatively little use of it; indeed, the project's contact page suggests that "the mailing lists should be your last resort". Discussion on the project's posted issues is active. All told, darktable looks like an active and healthy project, one that a photographer need not worry about depending on.

Darktable itself is a bit of an intimidating program, but it's not clear that it could be any other way. It is a power tool for photographers wanting absolute control over every aspect of their images. Potential users will need to invest a fair amount of time before they can expect to be proficient with it. For many casual photographers, the complexity of darktable will be a firm reminder of how much easier it is to just let the camera do the image-processing work. For those who are willing to put in the time, instead, darktable may well be the tool that puts them firmly in control of their art.