How to use a terabyte of RAM [LWN.net]

How to use a terabyte of RAM

Posted Mar 13, 2008 13:05 UTC (Thu) by davecb (subscriber, #1574)
Parent article: How to use a terabyte of RAM

If the memory filesystem doesn't try to decide
on the order of writes to the underlying filesystem,
but instead simpley creates a queue of operations 
and feeds them to the disk in the order they
are received, then a logging filesystem can
reorder and coalesce them to get the physical
disk update quite quickly.

A colleague did a thesis on that, and found
that reordering and coalescing paid a huge
benefit.

Alternatively, one could have an intermediate
layer which the memory filesystem could pass
operations to, which could do optimizations like
doing all directory operations asap and in-order
and delaying all file writes until close or
a certain large size had accumulated, then
writing teh whole file in one fell swoop. The
latter is *very* effective at increasing performancw
while decreasing seeks and CPU usage.

--dave

(Log in to post comments)

How to use a terabyte of RAM

Posted Mar 13, 2008 15:51 UTC (Thu) by i3839 (subscriber, #31386) [Link]

> then a logging filesystem can
> reorder and coalesce them

The IO scheduler does exactly that already (also for reads).

But the problem is that the writing order is very important quite often.

> Alternatively, one could have an intermediate
> layer which the memory filesystem could pass
> operations to, which could do optimizations like
> doing all directory operations asap and in-order
> and delaying all file writes until close or
> a certain large size had accumulated, then
> writing teh whole file in one fell swoop.

This sounds like the dirsync mount option and the current file caching that happens at the VM
level.

Currently all file operations happen on files cached in RAM. Disk IO is only done for reads
for uncached files and writes explicitly requested by user space (sync), by the VM because of
memory pressure, or because a time out triggered and it's time to write the written data also
to disk. At which point enough data can be gathered so that there is actually anything to
coalesce and reorder by the IO scheduler.

How to use a terabyte of RAM

Posted Mar 13, 2008 18:07 UTC (Thu) by davecb (subscriber, #1574) [Link]

  Dirsync is somewhat similar, except it's 
used to guarantee the write completes
serially and synchronously. It's therefor
a performance pessimization, somewhat like
the sync mount option (;-))

  I was just talking about the need to 
request directory updates as they come in
and in order, as opposed to delaying
and reordering them.

  Ian's thesis actualy went into the
degree to which one could delay and reorder 
directory writes, but I didn't want to add
that complexity to a very short email.

--dave