The familiar CRT monitors or backlit LCD screens on our desks continuously
consume power in order to hold an image. Electronic paper (e-paper) is
different: power is only needed to change the image. Just like paper,
e-paper is able to hold the image permanently without consuming any power.
Displays using CRT, backlit LCD, plasma and OLED technologies are all
emissive, meaning that they have to produce the photons that reach the eye.
This implies that they have to compete in brightness with ambient lighting,
which can result in eye strain. E-paper is the opposite: it is reflective,
which makes it possible to read the display using ambient light even in the
brightness of a hot sunny day.
E-paper is referred to as a bistable or non-volatile technology because of
its ability to hold a specific pixel state without power. There are several
variations of e-paper; they differ in terms of which physical mechanism
is used to achieve the non-volatility of the display. These mechanisms
modulation, bi-stable twisted nematic liquid
crystal [PDF], cholesteric
liquid crystal, and electrophoretic
Interferometric modulation uses the same principle of light wave
interference that results in the rainbow of colors seen with oil floating on
water. Control of wave interference through bi-stable or multi-stable
micro-electro-mechanical systems (MEMS) is what enables electronic control
of the color of a pixel.
In standard twisted nematic liquid crystal displays (TNLCD), the liquid
crystal is sandwiched between two rubbed polymer orthogonal alignment
layers. Bi-stable twisted nematic implementations such as Zenithal liquid
crystal replace the first or both alignment layers in favour of a sub-micron
relief profile that weakens anchoring to the surface and makes it possible
to latch various stable orientations of a liquid crystal pixel using
Cholesteric liquid crystal provides the ability to selectively reflect
various ranges of wavelengths of visible light based on the pitch of the
liquid crystal. The pitch can then be electronically controlled to set
various pixel states.
Electrophoresis describes the fact that particles within a fluid can be
kinetically affected by an electrical field. Basically, applying a voltage
pulse causes pigment particles within a solvent solution to move. This
concept is what is used to control whether a pixel appears black, white or a
shade of gray. This article will focus on electrophoretic displays
since they are relatively easy to obtain.
Traditional display controllers are interfaced to the host using a bus such
as PCI Express or AGP. These controllers have local framebuffer memory or
sufficient internal line buffering to utilize shared host memory; they
expose their framebuffers through memory mappable regions.
Display servers like Xorg or Xfbdev that utilize the kernel's fbdev
interface expect to be able to mmap() the device framebuffer. The
implication is that a driver that implemented only write()/seek() access to
the framebuffer would have limited usage.
Electrophoretic displays require specialized controllers that are capable of
driving suitable waveforms in order to control the display media. This is
because of subtle issues around the behavior of pigment particles within a
solvent. The controller must drive waveforms that result in fast,
reproducible and optimal movement of pigment particles. These waveforms are
a key factor in minimizing pixel update latency, achieving good contrast and
reducing ghosting effects in the output image. Currently, electrophoretic
display updates are significantly slower than CRT or LCD display updates.
For example, a grayscale update of E-Ink's most recent Vizplex display
material can take up to 740ms. This latency has an effect on how hardware is
interfaced with electrophoretic display controllers and how software should
then interact with the display.
One of the electrophoretic display controllers for which Linux support has
been posted (tarball) is
a controller from E-Ink called Apollo. This controller is
interfaced to the host through 8-bit data and 6-bit control over General
Purpose IO (GPIO) interfaces. The implication of the use of GPIO is that it
is not a memory mappable interface. Each pixel of the framebuffer has to be
wiggled to the controller by turning individual GPIO lines on and off. Display
updates on the Apollo with an E-Ink 6" panel with a resolution of 800x600
and 2 bits of grayscale require between 500ms - 1200ms. Given this set of
circumstances, it would have been an option to implement a userspace library
or support code that performed the GPIO wiggling. However, such an
implementation would forfeit support from Xfbdev and other common
fbdev compatible applications.
An early driver implementation has also been posted for an E-Ink controller
This controller interfaces to the host using the Active
Matrix LCD (AMLCD) bus. AMLCD is a 16-bit data bus used to interface LCDs
with CPUs. Normally, the AMLCD bus is used to transfer video display
data only, but, in the case of the Metronome controller, the host transfers a whole
slew of things including waveform, command and image data. The Metronome
becomes a secondary display controller feeding on the output of the primary
display controller on the host. Since AMLCD is an output-only data path, two
GPIO pins are used to retrieve status from the controller.
Many embedded processors provide a built-in LCD controller (LCDC) that is
compatible with the AMLCD interface. For example, the Xscale pxa255 cpu has
an LCDC that has DMA support and is able to pull data directly from host
memory at specified intervals. This type of capability allows drivers to
remap host memory to form an mmap-able framebuffer. However, the Metronome
controller imposes an additional requirement beyond delivering image data
for each display update. This is the need for a specific display update
command that has to be formed and set each time the display is to be
updated. This means that the framebuffer driver needs to know when the
framebuffer has been updated. That is not a trivial task because the nature
of a memory-mapped framebuffer is that the driver is not involved in
changes to the buffer; it is
therefore unaware of when the framebuffer has been written to by a userspace
The three problems described so far can therefore be summarized as follows:
- How to memory map a "non memory mappable" IO interface like GPIO.
- How to mitigate the latency associated with display updates.
- How to cheaply detect when userspace has written to a memory mapped
One early solution to problem 3 was to use a timer and perform framebuffer
differencing to detect the changed pixels. The negative aspects of this
solution are that it requires a large amount of redundant memory and
significant cpu and memory bandwidth consumption every time that framebuffer
differencing is done. Both of these resources are scarce on embedded systems
and, therefore, that solution was not satisfactory.
Deferred IO is an alternative method of solving these problems. The key concept
behind deferred IO is that one can periodically mark an active page of host
memory as read-only in order to catch writes to it. The way it works is quite
straightforward: page table entries for framebuffer pages in host memory are
initially marked as read-only. When the application first writes to any memory
address that maps to any of those pages, a deferred IO specific page fault
handler is reached. This handler schedules a delayed workqueue job. In the interval
before this workqueue is executed, the application can continue to write to
that page with no additional cost.
When the workqueue task executes, it then marks
the page table entry as read-only and then processes the framebuffer data
stored in that page. At that point, the processed data can be delivered to the
device through its native IO interface, which could be GPIO, AMLCD, USB, or
anything else. Since the page was re-marked to read-only, the sequence would
repeat if the application ever rewrote that page. This is somewhat similar
to a writeback cache. Host memory is used as a cache for device memory or
any output destined for the device. The page fault is then used as a
trigger to determine when to actually "writeback" this memory to the device.
This technique solves problem 1 because host memory is used and can therefore be
memory mapped. The output from the application intended for the device is
written to host memory and, unlike hardware supported memory mapped IO,
this output is not transfered to the device for each memory write. It is
only after the driver specified delay has expired that this collected data
is transfered to the device. The fact that the transfer would be through
GPIO or any other mechanism is transparent to the application and requires
The delay between the page fault and the IO is what addresses problem 2. The
application sees only a framebuffer which happens to be in host memory.
Writes to the framebuffer are therefore as fast as writes to any other part
of memory. The display update latency is therefore transparent to the
application. The driver specified interval should be selected to be
appropriate for the latency of the device. For example, if the device has a
one-second display update latency, then a one-second delay would be reasonable.
A longer delay would result in the display being less interactive than what
it was really capable of. A shorter delay would result in host updates
building up since the device would not be able to keep up. Applications that
require display synchronization primitives could use fsync() or the
FBIO_WAITFORVSYNC ioctl depending on their needs.
Problem 3 is solved because the address that is the cause of the page fault
is known. Internally, deferred IO uses the memory management subsystem's
page_mkwrite() callback and page_mkclean() to implement the core of its
functionality. The current deferred IO implementation passes a list of
page structures to the framebuffer driver's deferred IO callback. The driver
can then use page->index to identify which part of the framebuffer was
written to. This provides PAGE_SIZE granularity in identifying the updated
This method works fine in common use cases. For example, if one were to run
xpdf and use page-up to flip through pages, then xpdf would render to the
framebuffer in host memory on each page-up. Then, at the end of each write
induced interval, the driver would deliver the current image to the display.
This would give the effect where one would be seeing the most recent page on
the display rather than every single page that had been flipped through.
This enables the system to be reasonably interactive. Applications like
xclock (an analog clock ticking every second) as well as playback
applications (displaying a slider showing playback position) behave in a
Deferred IO support was merged into the Linux kernel in 2.6.22;
Documentation/fb/deferred_io.txt contains additional information. The driver
for the Apollo controller was also merged in 2.6.22 and is in
drivers/video/hecubafb.c. The driver for the Metronome controller
but not yet complete; it also
includes necessary bugfixes for deferred IO.
The current development focus is on the Metronome controller. It is being
tested with a Gumstix Connex board which has an Xscale pxa255 CPU. The
display media that is being used is an E-Ink Vizplex 6" 800x600 panel with 3
bits of grayscale. The metronomefb driver for this controller uses deferred
IO and is still a work-in-progress but it is capable of running Xfbdev. X
clients such as xclock, xeyes, xlogo and xloadimage have been run without
problems. It is not yet clear how to measure framebuffer performance on such
a system; the reason for this is that most display benchmarks use the time
for a drawing operation to complete as the basis for performance statistics.
On this system, such a benchmark would be merely measuring time to render to
host memory rather than time to deliver to the actual display. It may be
necessary to develop an alternate method of measuring display system
performance for e-paper displays.
All is not yet perfect.
Applications that render images that affect only a small number of pixels
but cross multiple pages because of the framebuffer layout (eg: a thin
vertical image) result in reduced efficiency. This is because the ratio of
changed pixels to the number of written pages is low.
The architectural weakness of deferred IO is that it depends on the system
having an MMU. It may be possible to implement a similar approach using the
lower level memory protection capabilities that are available on some no-MMU
systems. For example, the Blackfin architecture has a Data Cacheability
Protection Lookaside Buffer (DCPLB) that has notions of read/write
permissions on its entries. This will be an interesting area for future
The current implementation only works with framebuffers allocated from
virtual memory. Support needs to be implemented to achieve the same functionality
with memory obtained from kmalloc() or the DMA layer.
There have been suggestions that this technique may be useful in other
areas. One scenario that has been mentioned is optimizing display bandwidth
consumption by switching between DMA and plain memory copies based on the
number of written pages. Another scenario is USB-to-VGA adapters. It may
also be the case that any device connected via a relatively slow bus where
the data flow is primarily output could benefit from a similar approach.
the author is grateful to E-Ink engineers for their extensive support and
hardware help, Peter Zijlstra, Antonino Daplas, Paul Mundt, Geert Uytterhoeven,
Hugh Dickins, James Simmons and others for mm, fbdev, and general help.
to post comments)