Allocation of physically-contiguous memory buffers is required by many
device drivers, but it cannot always be reliably done on long-running Linux
systems. That leads to all kinds of unsatisfying workarounds in driver
recently posted by Michal Nazarewicz are an attempt
to solve this problem in a consistent way for all drivers.
A few years ago, when your editor was writing the camera driver for the
original OLPC XO system, a problem turned up. The video acquisition
hardware on the system was capable of copying video frames into memory via
DMA operations, but only to physically contiguous buffers. There was, in
other words, no scatter/gather DMA capability built into this (cheap) DMA
engine. A choice was thus forced: either allocate memory for video
acquisition at boot time, or attempt to allocate it on the fly when the
camera is actually used. The former choice is reliable, but it has the
disadvantage of leaving a significant chunk of memory idle (on a
memory-constrained system) whenever the camera is not in use - most of the
time on most systems. The latter choice does not waste memory, but is
unreliable - large, contiguous allocations are increasingly hard to do as
memory gets fragmented. In the OLPC case, the decision was to sacrifice
the memory to ensure that the camera would always work.
This particular problem has been faced many times by many developers over the years; each
driver author has tended to go with whatever ad hoc solution seems
to make sense at the time. For some years, the "bigphysarea" patch was
available to help, but that patch was never put into the mainline and has
not seen any maintenance for some time. So the problem remains
unsolved in any sort of general sense.
The contiguous memory allocation (CMA) patches are an attempt to put
together a flexible solution which can be used in all drivers. The basic
technique will be familiar: CMA grabs a chunk of contiguous physical memory
at boot time (when it's plentiful), then doles it out to drivers in
response to allocation requests. Where it differs is mainly in an
elaborate mechanism for defining the memory region(s) to reserve and the
policies for handing them out.
A system using CMA will always need to have at least one boot-time
parameter describing the memory region(s) to use and the policy for allocating
from those regions. The syntax used is rather complex, to the point that a large portion
of the patch is made up of parsing code; see the included Documentation/cma.txt file for the full
details. A simple example of a CMA command-line option would be something
This defines a 10MB region (called "c") and states that allocation requests
from the camera device should be satisfied from this region.
Multiple regions can be defined, each with its own size, alignment
constraints, and allocation algorithm, and memory regions can be split into
different "kinds" as well. The "kinds" feature might be used to separate
large and small allocations, or to put different buffers into different DMA
zones or NUMA nodes. The more complex command lines are reminiscent of
regular expressions, but with less readability. The purpose behind this
complexity is to enable a great deal of flexibility in how memory is
handled without the need to change the drivers which are working with that
memory. Whether that flexibility is worth the cost is not (to your editor,
at least) entirely clear.
A driver can actually allocate a memory chunk with:
unsigned long cma_alloc(const struct device *dev, const char *kind,
unsigned long size, unsigned long alignment);
If all goes well, the return value will be the physical address of the
allocated memory region.
For reasons which are not entirely clear, buffers allocated with CMA have a
reference count associated with them. So two functions are provided to
manipulate that count:
int cma_get(unsigned long addr);
int cma_put(unsigned long addr);
Since reference counting is used, there is no cma_free() function;
instead, the memory chunk is passed to cma_put() and freed
internally when the reference count goes to zero.
CMA comes with a best-fit allocator, but it is designed to work with
multiple internal allocators. So, should there be a need to use a
different allocation algorithm, it's a really straightforward matter to add
it to the system. Naturally enough, the command-line syntax offers a way
to specify which allocator should be used for each region.
In summary: CMA offers a solution to a problem which driver authors have
been dealing with for some years. Your editor suspects, though, that it
will require some changes before a mainline merge can be contemplated. The
complexity of the solution is probably more than is really called for in
this situation, and the whole thing might benefit from some integration
with the DMA mapping infrastructure. But, someday, it would be nice to
incorporate a solution to the large-buffer problem that all drivers can use.
to post comments)