Disney and Sony release open source 3-D modeling utilities

Not one, but two, Hollywood movie studios — Disney and Sony Pictures Imageworks — released open source software this past week. The film industry has had a mixed relationship with the free software movement in the past — on the one hand vehemently opposing it on issues like the Digital Millennium Copyright Act, but on the other making heavy use of free software to save money in visual effects render farms. This week's releases are indeed in the visual effects category; tools for 3-D modeling. Disney's project automates texture mapping, and Sony's is a new shading language. Both could benefit the open source community.

Ptex

Disney's Ptex is a library that simplifies the otherwise time-consuming task of mapping the surface of a 3-D model into two dimensions for the purpose of painting it with a "texture," or 2-D image skin. The traditional method unwraps the surface of the model into a flat image template called a UV map (the name refers to the U and V coordinate axes used on the map to distinguish them from the X, Y, and Z coordinate system of the original model). A UV map can be created automatically, but almost always requires manual tweaking by the 3-D artist in order to minimize awkward seams, overlaps, and other artifacts that are difficult to paint. Furthermore, complex models are typically split into multiple texture files, to simplify the resulting maps where the geometry itself is awkward, and to allow for higher resolution textures in important areas.

Ptex speeds up the process in several ways. First, it stores a separate texture for each face of the original 3-D model, eliminating the need for the user to painstakingly unwrap the model and manually adjust the UV map for optimization. Second, it compactly stores all of the per-face textures for the model in a single file, along with adjacency information, thus reducing the I/O load on the animation system. Third, it allows different resolutions for each face, and allows changing face resolutions, both eliminating the need to split a model into multiple textures and allowing the artist to make adjustments directly within the painting application. Finally, it uses the adjacency data to apply filters on the seams between adjoining faces — even those of different resolutions — resulting in a smooth, seamless final texture.

Disney first used Ptex in the 2008 animated short Glago's Guest, and subsequently on "virtually every surface" of the feature-length film Bolt. A presentation available at the Ptex web site describes the time savings for both computing time and artist workflow. Ptex reduced the number of I/O calls to render one model by a factor of 2,000, and cut the number of CPU cycles on another by a factor of 13.

Just as importantly, however, incorporating Ptex into the animation studio's modeling and painting applications speeds up the creative workflow of the team. The need to manually worry about UV mapping every model forced the 3-D modelers to think about how the model would be painted — which should not be their concern — and forced the painters to both learn and, periodically, adjust the UV maps.

The Ptex team has posted a video to YouTube demonstrating how artists interact with a Ptex texture. It is interesting viewing even for those unfamiliar with 3-D modeling and animation; the narrator points out several steps in the process where Ptex allows him to simply paint the model without concern for the underlying geometry while an older system would necessitate stopping or changing the UV map.

The Ptex source code is available under an MIT-style license through Github. The project's web site also includes documentation of the Ptex library API and a file format specification. A 2008 paper is also provided that goes into considerably more detail on the format and filtering. Of note, when the paper was written Ptex only supported one type of 3-D mesh, Catmull-Clark surfaces that use quadrilateral faces. Since then, Ptex has been extended to support additional mesh types, including non-quadrilateral Catmull-Clark meshes and loop subdivision surfaces.

Open Shading Language

Sony's Open Shading Language (OSL) is a language for writing shaders, algorithms used to calculate high-quality simulated lighting effects in the final rendering stage of 3-D animation. Shaders can do everything from model how a surface reflects and scatters light to how light is transformed as it passes through a volume of space (such as fog). High quality shaders can be of arbitrary complexity, thus the necessity of writing them in a programming language.

The most popular shading language is the RenderMan Shading Language (RSL) developed by Pixar. RSL is supported by a wide variety of free and proprietary tools, and several competing languages use the same basic C-like syntax of RSL.

The OSL language specification [PDF] is hosted at Google Code, and is distributed under the BSD license. OSL does away with several distinctions made in RSL and other shading languages, such as treating light sources as a distinct type of shader and transparency as a distinct property. Instead, OSL shaders calculate a "radiance closure," a symbolic representation of how the surface or volume in question affects light. A light source is merely a closure that emits a positive quantity of light, and a semi-transparent material is merely a closure that permits some quantity of light to pass through it.

Closures are also different from RSL-style shaders because, as an abstract representation of a surface or volume's behavior, they can be evaluated from any direction — RSL-style shaders can only be evaluated to a fixed value that represents the final color as calculated for a particular viewing angle. OSL permits closures to be saved for later evaluation and sampled or referenced without re-rendering. An OSL scene can be described as a connected network of closures that are "integrated" as-needed by the renderer. Depending on the viewing angle and lighting for a particular rendered pixel, some closures may not be evaluated at all, and these dependencies are determined automatically.

The practical upshot of OSL is that shader writers can focus more on modeling the physical properties of a surface or volume, and not explicitly concern themselves with implementation details imposed by other, more structured shading languages. The trade-off is that OSL requires significant changes to the renderer to support its significantly different approach. So far, no such renderer is available outside Sony.

The implementation currently available from the project provides a handful of sample OSL shaders, a compiler for translating OSL shaders into an intermediate bytecode form, a library that interprets OSL shaders, and a sample program called testshade that can execute a shader to produce a single test image.

The OSL Introduction page warns visitors about the pre-production state of the code, but notes that Sony has implemented OSL in its production workflow and is beginning to use it for real projects. OSL is currently slower than its other shaders and the team has isolated some missing functionality that needs to be added to the language.

Nevertheless, the OSL team is confident that it will meet its production goals, and promises to roll out updates to the public code in the coming weeks and months. Further out, it is interested in building alternate back-ends that could translate OSL shaders into code executable on GPUs, and experimenting with the LLVM compiler as an alternative to the present bytecode approach.

Open source integration

Considering the freshness of both projects, it is no surprise that neither has a clear future with existing open source 3-D tools like Blender. Both have attracted considerable attention in the community, though, in discussions at sites like Blenderartists.org and BlenderNation. The reaction has been positive, as is usually the case with a new open source utility.

As one might expect, artists who use Blender seem more excited about the prospect of utilizing Ptex, since it frees them from the decidedly un-artistic step of UV-mapping. Less clear is what to make of OSL, which still has a ways to go before it can be evaluated in real-world circumstances. Sony has demonstrated a good track record with its open source projects, however — it released open source tools for database management, voxel storage, and Python string handling in 2009, and has continued to update its code.

Open source developers who only remember the Hollywood movie studios from their legal assaults on DVD decryption and Bittorrent technology would do well to investigate the moves made by the animation studios. However upper management feels, the animators and tool developers there clearly share common ground with the community, and letting that go to waste would certainly be a mistake.