banks? [LWN.net]

banks?

Posted Sep 22, 2007 4:33 UTC (Sat) by sbishop (guest, #33061)
Parent article: What every programmer should know about memory, Part 1

For such a detailed and well-written article, I'm surprised that banks weren't mentioned. In short, a memory chip contains 4 or 8 banks, which each contain rows, which each contain columns. With regards to the timing constraints mentioned regarding rows, each bank is almost completely independent. Behaviourly, this means that you can activate, read, write and precharge each bank concurrently. (There's only one shared data bus, of course, but this allows you to talk to one bank while another is busy internally.)

I work for a memory manufacturer, but I don't know much about how our stuff is actually used. So maybe this doesn't actually help, speed-wise, and that's why Ulrich didn't mention it.

I have a few other comments/suggestions:

I wouldn't bother mentioning half-latencies. DDR only had one, 2.5, but DDR2 and beyond don't have any.
Speaking of CAS latencies, they're only getting worse. Higher clock speeds means higher latencies, period. You could run DDR parts at CAS latency 2, 2.5, or 3. DDR2 latencies go from 3 up to 7, I believe.
I don't understand the foot-note part of this quote: "As explained in Section 2.1.2, DRAM cells must constantly be refreshed. This does not happen completely transparently for the rest of the system. At times when a row {Rows are the granularity this happens with despite what [highperfdram] and other literature says (see [micronddr]).} is recharged no access is possible."

I don't know what the literature says--Can we get links to that stuff?--but all banks have to be precharged during a refresh event. (To be specific, this is an "auto-refresh"--"self-refresh" is a low-power mode.)

Sam Bishop

to post comments

No banks, thnx.

Posted Sep 22, 2007 10:21 UTC (Sat) by khim (subscriber, #9252) [Link] (1 responses)

While there are subtle differences between two-banks DIMM and two single-bank DIMMs (related to refresh as you've correctly pointed out) from software point of view they are more-or-less identical. I think this is why it was omitted from article. And the question "what the hell does latency 2.5 mean?" is common (less so today when DDR is almost history and DDR2 is mainstream, but still) so I think few lines are worth it...

No banks, thnx.

Posted Sep 22, 2007 14:10 UTC (Sat) by sbishop (guest, #33061) [Link]

You're confusing the terms "banks" and "ranks."

Ulrich goes into quite a bit of detail about how an address is broken into a "row" and "column" component, but there is a third component, bank, internal to the DRAM.

A DIMM[1] can be single-rank or dual-rank. When you write or read a single-rank DIMM, you talk to every memory component on the DIMM, every time. Dual-rank DIMMs have double the number of components as a single-rank DIMM, and when you write or read (activate or precharge, for that matter) you only talk to half of the components.

What any of this has to do with software, I don't know. I work with some absolutely lousy programmers who know quite a bit more about memory than I do. :)

[1] I know this holds for unbuffered DIMMs, but I don't know about fully-buffered DIMMs. I don't work with those.

CAS

Posted Sep 22, 2007 14:39 UTC (Sat) by knan (subscriber, #3940) [Link]

And currently available DDR3 modules looks to have CAS latencies of 5 to 9.

banks?

Posted Sep 24, 2007 13:52 UTC (Mon) by vapier (guest, #15768) [Link] (2 responses)

unfortunately, the memory management in Linux is completely transparent to userspace which means the advantages of placing your data in different banks is totally not doable.

it's really only possible when you're writing your own bare-metal applications as then you have full control over the physical memory layout. your custom linker scripts would place the data such that you make sure specific pieces of data lie in specific banks.

of course, this is beyond the stated scope as it isnt Linux :)

banks?

Posted Sep 24, 2007 16:54 UTC (Mon) by sbishop (guest, #33061) [Link] (1 responses)

Now you've got the Gentoo crowd double checking their linker man page... :)

Seriously, I've heard that the translation from a physical address to a particular bank/row/column combination is specific to the memory controller. Which makes perfect sense.

Also, let me make this clear: my bank comment was in regards to the "more details than you really need to know" section anyway. With Linux and/or random, commodity hardware, you can't call out a specific row or column either. But if you're going to talk about rows and columns, you ought to at least mention banks, the third address component of a modern DRAM. ("Modern DRAM" here means, approximately, anything I'm familiar with: Sync-DRAM, DDR, DDR2, etc.)

banks?

Posted Sep 24, 2007 20:18 UTC (Mon) by vapier (guest, #15768) [Link]

i might point out that linker scripts arent documented in the linker man page

i obviously cant read minds, but i think the discussion of being able to scan rows without re-issuing a CAS command will lead into the idea of accessing data in a certain fashion (in sequential lines) rather than much more randomly ... whereas with banks, exhibiting certain behavior at runtime wont matter since in general, being in different banks is random