Optimizing Linux with cheap flash drives [LWN.net]

Optimizing Linux with cheap flash drives

Posted Feb 18, 2011 22:39 UTC (Fri) by jengelh (guest, #33263) [Link] (3 responses)

And lose all the features of a modern POSIX filesystem? Starting with permissions... (unless you go via layers like posixovl)

Optimizing Linux with cheap flash drives

Posted Feb 19, 2011 5:00 UTC (Sat) by eru (subscriber, #2753) [Link] (2 responses)

And lose all the features of a modern POSIX filesystem?

Just resurrect the UMSDOS filesystem!

Optimizing Linux with cheap flash drives

Posted Feb 19, 2011 8:41 UTC (Sat) by arnd (subscriber, #8866) [Link] (1 responses)

It seems that Samsung have done something like this with their RFS (Reliable FAT file system) that they use on a lot of phones with eMMC storage.

Unfortunately, their implementation has a lot of other performance problems and, worse, it's not even available under a GPL compatible license.

Optimizing Linux with cheap flash drives

Posted Feb 22, 2011 17:34 UTC (Tue) by ttonino (guest, #4073) [Link]

My Samsung Galaxy S shows:

On the NAND flash with a software translation layer:
/ rootfs ro,relatime
/mnt/.lfs j4fs rw,relatime
/system rfs ro,relatime
/dbdata rfs rw,relatime
/cache rfs rw,relatime

On the internal SD device:
/data rfs rw
/mnt/sdcard vfat rw,dirsync,noatime,nodiratime
/mnt/sdcard/external_sd vfat rw,dirsync,noatime,nodiratime

Thus, rfs and normal vfat are both used. The rfs description reminded me of the phase tree FAT implementation that was going around a long time ago.

The software translation for the NAND flash is GPL.

Optimizing Linux with cheap flash drives

Posted Feb 18, 2011 23:26 UTC (Fri) by bronson (subscriber, #4806) [Link] (4 responses)

Maybe your partitons are unaligned?

unaligned partitions

Posted Feb 23, 2011 12:11 UTC (Wed) by alex (subscriber, #1355) [Link] (3 responses)

We found that write performance for small files improved by at least a factor of 10 on our embedded SSDs when we fixed the partition alignment. fdisk doesn't help when it tries to work with the fake geometry most SSDs report.

unaligned partitions

Posted Feb 24, 2011 16:14 UTC (Thu) by mgedmin (subscriber, #34497) [Link] (2 responses)

How do you check for partition alignment? fdisk -u -l /dev/sdX?

I'm guessing 63 sectors (of 512 bytes) is not a good alignment?

Are there any tools for fixing partition alignment? Does parted's "move" command shift the data in the partition, or just adjust the boundaries in the partition table?

unaligned partitions

Posted Feb 24, 2011 18:53 UTC (Thu) by meyering (guest, #48285) [Link]

You can make parted list the partition table in units of sectors,
then ensure that (assuming 512-byte sectors) each partition's
start sector is divisible by some round number, like 2048
if you want them to be 1MiB-aligned.

For example, here all partitions are MiB-aligned, except
for the first one, which is only 32KiB-aligned. But since it's
only for grub, that is ok:

$ parted -m -s -- /dev/sdb u s p free
BYT;
/dev/sdb:117231408s:scsi:512:512:gpt:ATA OCZ-VERTEX2;
1:34s:63s:30s:free;
1:64s:4095s:4032s:ext2:_grub_bios:bios_grub;
2:4096s:1048575s:1044480s:ext3:_/boot:boot;
3:1048576s:12582911s:11534336s:ext4:_/:;
4:12582912s:16777215s:4194304s:linux-swap(v1):_/swap:;
5:16777216s:37748735s:20971520s:ext4:_/usr:;
6:37748736s:52428799s:14680064s:ext4:_/var:;
7:52428800s:52449279s:20480s:ext4:_/full:;
8:52449280s:117229567s:64780288s:ext4:_/h:;
1:117229568s:117231374s:1807s:free;

Please do not use parted's "move" command. It is risky since it tries
to be smart and is file-system aware. In addition to moving the partition
it may try to move an embedded file system, too, but its built-in FS-aware
code is so old and unreliable that it is slated to be removed altogether.

If you try to use that sub-command (or e.g., mkpartfs which is in the same
boat), recent versions of parted will emit a big warning telling you some
of the above.

unaligned partitions

Posted Feb 25, 2011 0:15 UTC (Fri) by jnh (subscriber, #69758) [Link]

It depends on the physical sector size of the device. If you have physical sectors of 512 bytes, then partitions measured in 512 logical sectors are aligned regardless of where they start; with larger physical sectors, starting a partition at LBA 63 isn't going to be aligned. Annoyingly, many SSDs do not correctly report their true internal topology, so even modern partitioning tools which can use that information may need to be given hints, but that said, it isn't immediately clear to me exactly what an SSD should report its topology as given the current interfaces.

I recommend reading Martin K. Petersen's advanced storage papers from
http://oss.oracle.com/~mkp/

Optimizing Linux with cheap flash drives

Posted Feb 19, 2011 8:29 UTC (Sat) by roblucid (guest, #48964) [Link] (1 responses)

IIRC early OEM drives may have poor controllers (like the Jmicron) with things like the so called "write stutter" problem. Once you'ld run out of virgin pages (segments?). A "secure erase" utility would be required to restore original performance.

There should be some good articles on SSD performance issues (mainly or wholly used with Windows but still should be relevant enough), and you could perhaps see if an SSD upgrade to more modern firmware, might be beneficial.

Optimizing Linux with cheap flash drives

Posted Feb 19, 2011 8:31 UTC (Sat) by roblucid (guest, #48964) [Link]

Whoops, I left out the site name AndandTech http://www.anandtech.com/

Optimizing Linux with cheap flash drives

Posted Feb 19, 2011 8:47 UTC (Sat) by arnd (subscriber, #8866) [Link] (7 responses)

The CF card I mentioned in the article is similar to a lot of the really cheap SSDs. It can have three 4 MB segments open for linear writing and one for random access, which is apparently just not enough for ext3 with journal.

Aligning the partition to 4 MB and changing to btrfs solved it for me for that card. I have not yet done thorough testing to find out what the specific requirements of the possible file systems are, but it's worth a try.

I'd also be interested to see what flashbench shows about this drive. If you can create an empty 4 MB aligned partition on it, please run it and send the results to flashbench-results@lists.linaro.org.

Optimizing Linux with cheap flash drives

Posted Feb 19, 2011 10:30 UTC (Sat) by aleXXX (subscriber, #2742) [Link] (6 responses)

Very nice article, but what is a bit unclear to me is whether these issues mostly apply to CF- and SD-cards and USB memory sticks or do they also apply to SSD drives ?

Alex

Optimizing Linux with cheap flash drives

Posted Feb 19, 2011 10:40 UTC (Sat) by arnd (subscriber, #8866) [Link] (5 responses)

The cheapest SSD drives are basically CF cards in a different form factor, or nowadays with a PATA-SATA converter. This will show the exact same behavior as good SD cards.

High-end SSDs come with significant amounts of RAM that can be used to hide most of the nasty effects, or to do something much smarter altogether, such as implementing the entire drive as a log structured file.
The caching unfortunately makes it a lot harder to reverse-engineer the drive through timing attacks, so it's much harder to tell what it really does.

What we know is that the underlying NAND flash technology is very similar, so in the best case, an SSD will be able to hide the problems, but not completely avoid them. If I were to design an SSD controller, I'd do the same things that I'm suggesting in https://wiki.linaro.org/WorkingGroups/KernelConsolidation...

Optimizing Linux with cheap flash drives

Posted Feb 19, 2011 18:23 UTC (Sat) by aleXXX (subscriber, #2742) [Link] (1 responses)

You mention read/write speed around 15 MB/s.
How does that fit together with the number between 150 and 350 MB/s which are listed for SSD drives e.g. on alternate.de ?

Actually I can remember that when writing to raw NAND we had also rates somewhere in the 10 to 15 MB/s range.

Alex

Optimizing Linux with cheap flash drives

Posted Feb 19, 2011 20:03 UTC (Sat) by arnd (subscriber, #8866) [Link]

15 MB/s is typical for good SD cards (e.g. Class 6), which are limited by design to 20-25 MB/s anyway (UHS-1 SDHC will be faster, but is still rare today). High-end SSDs can be much faster for a number of reasons:

* SATA is a much faster interface than SD/MMC
* NCQ and write caching allows optimizing the accesses by reordering and batching NAND flash accesses
* Using SLC NAND instead of MLC improves raw accesses
* Using multiple NAND chips in parallel gives a better combined throughput
* Expensive microcontrollers on the drive can use smarter algorithms

All of these cost money, so you don't find them on the low end drives that I analyzed.

Optimizing Linux with cheap flash drives

Posted Feb 20, 2011 9:12 UTC (Sun) by alonz (subscriber, #815) [Link] (2 responses)

Actually, according to information in this AnandTech article, some high-end controllers use even weirder techniques... (They mention specifically real-time compression and real-time deduplication, and there's likely a lot more)

Optimizing Linux with cheap flash drives

Posted Apr 6, 2011 18:36 UTC (Wed) by taggart (subscriber, #13801) [Link] (1 responses)

The compression and deduplication of the Sandforce controller show big benefits over the controllers that don't have them. But those benefits are lost if your data isn't compressable/redundant like if it's encrypted :(

Live CD

Posted Apr 20, 2011 19:12 UTC (Wed) by dmarti (subscriber, #11625) [Link]

What about a live CD that you boot from, type "yes I want to trash my flash drive" and it automatically tries different partition schemes, runs benchmarks, and tells you which one is fast? Don't trust what the drive says, just try it a bunch of possible ways and see what works for real. (I'd pay $14.95 for the iso assuming the underlying code was Free.)