Device drivers, in the end, usually do one thing: they communicate with the
hardware by way of a set of memory-mapped I/O (MMIO) registers. So when
one is trying to figure out what a driver is doing - for debugging
purposes, perhaps - it is often interesting to look at the sequence of MMIO
operations the driver performs. If one is trying to reverse-engineer a
driver which is available only in binary form, watching what is done with
MMIO registers may be the only way to figure out how the hardware works.
To this end, the developers behind the Nouveau project developed a tool
called "mmiotrace" which helps them to watch which is going on with
memory-mapped I/O. Now that tool is being fixed up and pushed toward the
Drivers gain access to MMIO regions with ioremap() (or one of the
higher-level functions like pci_iomap()), so that is the logical
place to hook in a tracing infrastructure. So the current mmiotrace patch adds
some new variants of ioremap():
void __iomem *ioremap_cache_trace(unsigned long offset, unsigned long size);
void __iomem *ioremap_nocache_trace(unsigned long offset, unsigned long size);
void iounmap_trace(volatile void __iomem *addr);
These functions perform like ioremap() and
ioremap_nocache(), in that they return a I/O memory pointer which
can be used by the driver to get at MMIO space. What goes on internally,
though, is quite different.
On the x86 architecture (as with most others), I/O memory space is accessed
with memory operations through the page tables in the usual way, so ioremap() just
returns an address which maps onto the desired physical space. The tracing
versions, though, take the extra step of marking the pages within the I/O
region as not being present in the system; as a result, whenever code
attempts to access that space, a page fault will be generated.
Normally, page faults incurred when running in kernel mode will cause a kernel
oops. There are exceptions, though; the functions which copy data between
user and kernel space are one example. The mmiotrace patch adds another
exception which tests faulting addresses against the MMIO region(s) being
traced. Should the address indicate that an MMIO access is being
attempted, the mmiotrace code will:
- Mark the relevant page as being present in memory.
- Set the TF (trace) bit in the faulting thread's processor state mask.
- Invoke a "pre" handler provided by higher-level tracing code.
- Indicate that the fault has been handled and return to the faulting
Once all this has happened, the instruction which originally caused the
page fault will be rerun, successfully this time. But the setting of the
trace bit will cause a new processor trap after that instruction has been
executed. At that point, the page is marked unavailable once again,
the trace bit is reset (assuming it wasn't set elsewhere), the tracing
layer's "post" handler is called, and life continues as normal until the
next fault happens.
The tracing layer really only has one task: figure out what the code was
trying to do in MMIO space and log the action by way of the relay
interface. Figuring things out means learning enough about the instruction
which caused the page fault to determine which address was being accessed,
whether a read or write was being performed, the size of the data being
transferred, and the actual value read or written. So there is a certain
amount of architecture-specific instruction grubbing code involved, which,
for the current patch, is only provided for x86 machines.
Since tracing is enabled by calling a special version of
ioremap(), it is not possible to trace a driver module without
making changes to its source and rebuilding it. That might seem like a strange requirement
for a tool meant to help with reverse engineering (among other things).
The driver being studied by the Nouveau project uses a GPL-licensed shim to
link into the kernel, so making modifications in that case was not a hard
thing to do. A more general solution may eventually need to be found,
though, for situations where that sort of glue layer is not present.
Beyond that, this patch is likely to go through a number of changes before
it finds its way into the mainline. Reviewers have found a number of
things which need fixing, and there's a few too many places in the code
where the comments say (literally) "if this happens, all hell breaks loose." It also
seems likely that mmiotrace will be merged with the recently-posted ftrace tracing mechanism. There is time to
get this work done before the 2.6.26 merge window opens, but the mmiotrace
hackers will need to keep the work moving forward.
to post comments)