| |||||||||||||
Log-structured file systems: There's one in every SSDLog-structured file systems: There's one in every SSDPosted Sep 20, 2009 23:36 UTC (Sun) by dlang (✭ supporter ✭, #313)In reply to: Log-structured file systems: There's one in every SSD by drag Parent article: Log-structured file systems: There's one in every SSD
it is very expensive to run all the wires to connect things via a parallel bus, that is why drive interfaces have moved to serial busses
trying to map your flash storage into your address space is going to be expensive, and it also isnt very portable from system to system.
it's already hard to get enough slots for everything needed in a server, dedicating one to flash is a hard decision, and low-end systems have even fewer slots.
there is one commercial company making PCI-E based flash drives, their cost per drive is an order of magnitude higher than the companies making SATA based devices, they also haven't been able to get their device drivers upstream into the kernel so users are forced into binary-only drivers in their main kernel. this is significantly worse than the SATA interface version because now mistakes in the driver can clobber the entire system, not just the storage device.
sorry, I don't buy into the 'just connect the raw flash and everything will be good' reasoning.
(Log in to post comments)
Log-structured file systems: There's one in every SSD Posted Sep 21, 2009 0:38 UTC (Mon) by drag (subscriber, #31333) [Link]
Ya. I was just trying to show what it would be like to try to access a large amount of flash memory in a 'raw mode'.
People are kinda confused, I think, about the whole block vs flash thing.
The way I see it Flash memory, in a lot of ways, is much more like memory then block devices. Instead of thinking it as block devices with no seek time... think of it more like memory with pecular requirements that makes writes very expensive.
Log-structured file systems: There's one in every SSD Posted Sep 21, 2009 1:03 UTC (Mon) by dlang (✭ supporter ✭, #313) [Link]
the thing is that even main memory access isn't really 'memory like' anymore.
on modern systems, it's impossible to directly access a particular byte of ram. the memory chips actually act more like tape drives, it takes a significant amount of time to get to the start position, then it's cheap to do sequential read/writes from that point forward.
your cpu uses this to treat your ram as if it was a tape device with blocks the size of your cpu cache lines (64-256 bytes each)
In addition, if you want to have checksums to detect problems you need to define what size the chunks of data are that you are doing the checksum over.
Log-structured file systems: There's one in every SSD Posted Sep 21, 2009 9:10 UTC (Mon) by nix (subscriber, #2304) [Link] on modern systems, it's impossible to directly access a particular byte of ram. the memory chips actually act more like tape drives, it takes a significant amount of time to get to the start position, then it's cheap to do sequential read/writes from that point forward. your cpu uses this to treat your ram as if it was a tape device with blocks the size of your cpu cache lines (64-256 bytes each)That's almost entirely inaccurate, I'm afraid. Ulrich Drepper's article on memory puts it better, in section 2.2.1. The memory is necessarily read in units of cachelines, and it takes a significant amount of time to load uncached data from main memory, and of course it takes time to latch RAS and CAS, but main memory itself has a jagged access pattern, with additional delays from precharging and so on whenever RAS has to change. But that doesn't make it like a tape drive, it's still random-access: it takes the same time to jump one row forwards as to jump fifty backwards. It's just that the units of this random access are very strange, given that they're dependent on the physical layout of memory in the machine (not machine words and possibly not cachelines), and are shielded from you by multiple layers of caching.
Log-structured file systems: There's one in every SSD Posted Sep 21, 2009 8:30 UTC (Mon) by butlerm (subscriber, #13312) [Link]
PCI-E really isn't a parallel bus - it is multi-lane differential serial bus.
You can use a single lane if you want to - 500 MB/sec per lane today, 1 GB/s soon.
The good thing about PCI-E for this application is that you can use simple
Flash is not exactly a byte addressable memory technology, btw, so you still
Log-structured file systems: There's one in every SSD Posted Sep 25, 2009 22:32 UTC (Fri) by giraffedata (subscriber, #1954) [Link] it is very expensive to run all the wires to connect things via a parallel bus, that is why drive interfaces have moved to serial busses That's not why drive interfaces (and every other data communication interface in existence) have moved to serial. They did it to get faster data transfer. But I'm confused as to the context anyway, because the ancestor posts don't mention parallel busses.
Log-structured file systems: There's one in every SSD Posted Sep 25, 2009 22:57 UTC (Fri) by dlang (✭ supporter ✭, #313) [Link]
they described handling flash as if it was just ram with special write requirements.
that (at least to me) implied the need for a full memory bus (thus the lots of wires)
by the way, parallel buses are inherently faster than serial buses, all else being equal.
if 1 wire lets you transmit data at speed H, N wires will let you transmit data at a speed of NxH.
the problem with parallel buses at high speeds is that we have gotten fast enough that the timing has gotten short enough that the variation in the length of the wires (and therefor the speed-of-light time for signals to get to the other end) and the speed of individual transistors varies enough to run up against the timing limits.
Log-structured file systems: There's one in every SSD Posted Sep 28, 2009 15:31 UTC (Mon) by giraffedata (subscriber, #1954) [Link] They described handling flash as if it was just ram with special write requirements. But the post described doing that with a PCI Express card. PCI-E is as serial as SATA. by the way, parallel buses are inherently faster than serial buses, all else being equal. It sounds like you consider any gathering of multiple wires to be a parallel bus. That's not how people normally use the word; for example, when you run a trunk of 8 Ethernet cables between two switches, that's not a parallel bus. A parallel bus is where the bits of a byte travel on separate wires at the same time, as opposed to one wire at different times. Skew is an inherent part of it.
crosstalk, not wire length Posted Oct 2, 2009 0:21 UTC (Fri) by gus3 (guest, #61103) [Link] > if 1 wire lets you transmit data at speed H, N wires will let you transmit data at a speed of NxH.That is true, when the bus clock speed is slow enough to allow voltages and crosstalk between the wires to settle. However, as clock speeds approach 1GHz, crosstalk becomes a big problem. > the problem with parallel buses at high speeds is that we have gotten fast enough that the timing has gotten short enough that the variation in the length of the wires ... and the speed of individual transistors varies enough to run up against the timing limits. Wire length on a matched set of wires (whether it's cat5 with RJ-45 plugs, or a USB cable, IDE, SCSI, or even a VGA cable) has nothing to do with it. The switching speed on the transmission end can accomplish only so much, but there has to be some delay to allow the signal to settle onto the line. The culprit is the impedance present in even a single wire, that resists changes in current. The more wires there are in a bundle, the longer it takes the transmitted signal to settle across all the wires. By reducing the number of wires, the settling time goes down as well. Related anecdote/urban legend: On the first day of a new incoming class, RAdm Grace Hopper would hold up a length of wire and ask how long it was. Most of the students would say "one foot", but the correct answer was "one nanosecond."
crosstalk, not wire length Posted Oct 2, 2009 17:15 UTC (Fri) by giraffedata (subscriber, #1954) [Link] That's good information about transmitting signals on electrical wires, but it doesn't distinguish between parallel and serial protocols. Crosstalk is a phenomenon on bundled wires, which exist in serial protocols too: each wire carries one serial stream. This wire configuration is common and affords faster total data transmission than parallel with the same number of wires. Signals having to settle onto the line also happens in serial protocols as well as parallel. Is there some way that crosstalk and settling affect skew between wires but not the basic signal rate capacity of each individual wire?
crosstalk, not wire length Posted Oct 7, 2009 5:47 UTC (Wed) by gus3 (guest, #61103) [Link] I had to think about the crosstalk vs. skew issue for a bit, but I think I can explain it. (N.B.: IANAEE; I Am Not An Electronics Engineer. But I did work with one for a couple years, and he explained this behavior to me.)Take an old 40-conductor IDE cable, for example. Typically, it's flat; maybe it's bundled. Each type creates its own issues. A flat cable, with full 40-bit skew, basically means that the bit transmitted on pin 1, can't be considered valid until the bit on pin 40 is transmitted, AND its signal settles. Or, with an 8-bit skew, bits 1, 9, 17, 25, and 33 aren't valid until bits 8, 16, 24, 32, and 40 are transmitted. (IIRC, an 80-conductor cable compensated for this, using differential signaling, transmitting opposite signals on a pin pair, using lower voltages to do so. This permitted less crosstalk between bits, while speeding the signal detection at the other end. But I could be wrong on this.) A bundled 40-conductor cable is a little better. Think about an ideal compaction: 1 wire in the center, 6 wires around it, 12 around those, 18 around those, and 3 more strung along somewhere. From an engineering view, this could mean bit 1, then bits 2-7 plus settling time, then bits 8-19, plus settling time, then bits 20-37 plus settling time, then bits 38-40 plus settling time. (This from an iterative programmer's mind-set. A scrambled bundle might be better, if an EE person takes up the puzzle.) Now, consider a SATA bus. Eight wires: ground, 2 differential for data out, ground, 2 differential for data in, ground, and reference notch. Three ground lines, with the center ground isolating the input and output lines. Add to this the mirror-image polarity between input and output; the positive wires are most isolated from each other, while the negative wires are each next to the middle ground wire. The crosstalk between the positive input and positive output drops to a negligible level, and the negative lines, near the center ground, serve primarily for error checking (per my best guess). I hope my visualization efforts have paid off for you. Corrections are welcome from anyone. Remember, IANAEE, and my info is worth what you pay for it. This stuff has been a hobby of mine for over 30 years now, but alas, it's only a hobby.
|
|||||||||||||