Another round of speculative-execution vulnerabilities [LWN.net]

Another round of speculative-execution vulnerabilities

Posted Aug 9, 2023 2:07 UTC (Wed) by dxin (guest, #136611)
Parent article: Another round of speculative-execution vulnerabilities

When DEC said they will scale Alpha to 1000x processing power in the future, it was considered quite ambitious.
But they actually guessed it right: things gets really sketchy after 1000x.
We have scaled really well on number of cores, but barely made it past 10x clock speed, and on the way of 10x IPC we shot out feet on every single step.

to post comments

Another round of speculative-execution vulnerabilities

Posted Aug 9, 2023 2:19 UTC (Wed) by willy (subscriber, #9762) [Link] (7 responses)

What's your baseline for 10x IPC? I was just reading that 486 had 1.8 CPI, half that of the 386. We're up to, what, 6 IPC now? So that's about a 25x improvement in IPC relative to the i386

Another round of speculative-execution vulnerabilities

Posted Aug 9, 2023 5:35 UTC (Wed) by ibukanov (subscriber, #3942) [Link]

This is the optimal case in absence of pipeline stalls. Typically it does not happens unless one runs carefully written calculation-type code. Which explains why the startup time for modern apps is much slower than what one can expect based on IPC.

Another round of speculative-execution vulnerabilities

Posted Aug 9, 2023 9:16 UTC (Wed) by paulj (subscriber, #341) [Link] (5 responses)

One issue is that there is a parallelism limit in common codes, of about 4 to 10 IPC, according to a '93 DEC WRL tech report by David Wall. Also, even if you increase IPC and can avoid pipeline stalls there, memory speed - latency particularly - isn't keeping up.

There is a good argument to be made that the increasing transistor count budgets could be better spent on adding more, simple, compute elements ("cores") rather than adding ever more complex speculative execution logic to ever more complex compute elements. That this would be more efficient overall.

I.e., rather than trying to make 1 (or a very small) number of parallel paths of execution very fast with speculative execution, we should just provide many more paths of execution with simpler cores. The simpler cores might each have to stall more waiting on memory latency, but if you have many of them you can get more throughput - they will not waste cycle or energy on misplaced speculative execution.

These are not new ideas, they go back a long way, and we're slowly going down that path it seems. GPUs are kind of part of that vision, CPUs have gone many-core, but still with very complex speculative logic to fulfil desire for good single-thread benchmark results. Old blog of mine, but the references are still good to read: https://paul.jakma.org/2009/12/07/thread-level-parallelis...

Using all of those cores

Posted Aug 10, 2023 16:05 UTC (Thu) by DemiMarie (guest, #164188) [Link] (4 responses)

How can one beat the parallelism limit you mentioned?

Using all of those cores

Posted Aug 10, 2023 17:19 UTC (Thu) by farnz (subscriber, #17727) [Link] (2 responses)

You can't, easily. Much of the parallelism limit is inherent to the way we perceive the problem domain, and it's simply not possible to have more parallelism without radical new understandings of the problems we're trying to solve.

Some problems, such as graphics rendering and neural network modelling, do have a higher inherent parallelism, and we have an alternative type of processor, called a GPU for historical reasons, which is designed to be faster than a CPU on problems with lots of parallelism; it achieves this by sacrificing single thread performance in favour of running a large number of concurrent threads, complete with hardware support for launching a very large number of threads and multiplexing them onto a smaller number of executing threads.

Not everything with parallelism is suitable for GPUs

Posted Aug 10, 2023 22:01 UTC (Thu) by DemiMarie (guest, #164188) [Link] (1 responses)

GPUs have other limitations, though. For instance, the SIMT model means that GPUs are terrible at workloads with lots of non-uniform control flow. That isn’t a huge limitation for math or graphics, but it is a serious limitation for what I call “business logic” workloads, where a significant part of the problem is figuring out what to do next. This includes e.g. web applications, which have a huge amount of parallelism but lots of conditional branches and non-uniform memory accesses.

Not everything with parallelism is suitable for GPUs

Posted Aug 10, 2023 22:08 UTC (Thu) by farnz (subscriber, #17727) [Link]

They're no more terrible at non-uniform control flow than CPUs are - in the worst case, you just use one SIMD lane per GPU core, get a much lower throughput, but still have the large number of threads. It's just that we look at GPUs differently to CPUs, so we see the slowdown from using only one SIMD lane as a big deal on a GPU, but we don't see it as a big deal that we only use scalar instructions on CPU cores with the ability to process 8 (AVX2) or 16 (AVX-512) 32-bit values in parallel, despite the fact that this is the same class of slowdown.

Using all of those cores

Posted Aug 11, 2023 9:48 UTC (Fri) by paulj (subscriber, #341) [Link]

Parallelism for a code for a specific problem?: You have to find a more parallel algorithm. If that is even possible.

Making efficient use of compute resources, in a world where the codes you want to run have limited parallelism? Run many different codes together on the same compute elements, and switch between them to keep memory bandwidth and compute occupied. No single code will run faster, but at least you maintain throughput in the aggregate.

This is kind of where computers have gone anyway. From your phone, to your desktop, to servers running containers running jobs in the cloud - they've all got many many dozens of jobs to run at any given time. If one stalls, switch to another.

Another round of speculative-execution vulnerabilities

Posted Aug 9, 2023 8:13 UTC (Wed) by Wol (subscriber, #4433) [Link] (30 responses)

The problem with CPU speed is we have now hit what I'll call the "Cray Limit". Cray Supercomputers were always relatively small, because the bigger the computer the longer it took the various components to talk to each other. This puts an absolute upper limit on the GHz, because if the chip speed times the size of the computer is faster than the speed of light, well, things ain't going to work!

At roughly 1ft/ns, this means your typical ATX mobo cannot operate faster than 500MHz. Knock a nought off that, to give a 3cm chip, and you've stuck a nought on your chip speed, 5GHz. Careful placement of components will nudge that speed up, but if components need to communicate "across chip", you're stuffed ...

Cheers,
Wol

Another round of speculative-execution vulnerabilities

Posted Aug 9, 2023 9:16 UTC (Wed) by joib (subscriber, #8541) [Link]

> The problem with CPU speed is we have now hit what I'll call the "Cray Limit". Cray Supercomputers were always relatively small, because the bigger the computer the longer it took the various components to talk to each other.

Cray's were about the same size as other mainframe sized computers of the days. Going much bigger wasn't really useful, because neither software nor hardware at the time was ready for massive parallelism. Today it is, and thus we have warehouse sized supercomputers that can run (some, obviously not all) HPC style problems utilizing all that parallelism.

> At roughly 1ft/ns, this means your typical ATX mobo cannot operate faster than 500MHz. Knock a nought off that, to give a 3cm chip, and you've stuck a nought on your chip speed, 5GHz. Careful placement of components will nudge that speed up, but if components need to communicate "across chip", you're stuffed ...

That matters insofar as you require everything to be synchronous, with a signal traversing from across a wire within one clock cycle. The existence of CPU cores within your CPU running at different frequencies, not to mention long distance high speed network transmission, suggests that it's possible to design things without such synchronicity requirements.

Another round of speculative-execution vulnerabilities

Posted Aug 9, 2023 9:57 UTC (Wed) by malmedal (subscriber, #56172) [Link] (6 responses)

> because if the chip speed times the size of the computer is faster than the speed of light, well, things ain't going to work!

No, chip designers face many problems, but that one is solved. They are making complicated networks, called clock-trees, or meshes, that ensure each clock edge arrives at all components in the clock-domain simultaneously.

The problems they can't seem to solve include power and the fact that wires are getting slower faster than they get shorter in recent processes.

Another round of speculative-execution vulnerabilities

Posted Aug 9, 2023 11:53 UTC (Wed) by Wol (subscriber, #4433) [Link] (5 responses)

> No, chip designers face many problems, but that one is solved. They are making complicated networks, called clock-trees, or meshes, that ensure each clock edge arrives at all components in the clock-domain simultaneously.

That's the wrong problem! Making sure all the signals arrive together is easy. The problem is that if your edges do not share a common "light cone", you're stuffed!

Increasing the frequency reduces the size of the light cone. If the light cone is to encompass the entire chip, then the upper limit on frequency is 5GHz. As others have said, if you're only interested in communicating internal to a single core, then LOCALLY you can increase the frequency further, because everything you're interested in fits into a smaller light cone.

You can have all the fancy clock-trees you like, but if your components are that far apart that you physically require faster-than-light information transfer, you're stuffed.

Cheers,
Wol

Another round of speculative-execution vulnerabilities

Posted Aug 9, 2023 12:34 UTC (Wed) by malmedal (subscriber, #56172) [Link] (3 responses)

Signal propagation speed in a wire is much slower than light speed. The chips are already much larger than the "light-cone". It gets progressively harder but there is no hard limit at a specific place.

The actual speed was about 1mm per nanosecond over ten years ago. It is probably less than 0.05mm/ns now.

Another round of speculative-execution vulnerabilities

Posted Aug 9, 2023 13:18 UTC (Wed) by Wol (subscriber, #4433) [Link] (2 responses)

So what you're saying is the speed of light slows down massively as the wire size shrinks. Ouch. Although I know the speed of electron propagation is much much slower. So it's a case of the fewer electrons fit on the wire, the closer the speed of light down the wire approximates to the speed of electrons in the wire? Painful!

Cheers,
Wol

Another round of speculative-execution vulnerabilities

Posted Aug 9, 2023 13:36 UTC (Wed) by malmedal (subscriber, #56172) [Link]

Hmmm, I've understood the issue to be explained by the telegraph equations: https://en.m.wikipedia.org/wiki/Telegrapher%27s_equations, but I suppose at the latest processes they may possibly need to account for individual elections. Which would be an extra complication.

Another round of speculative-execution vulnerabilities

Posted Aug 9, 2023 14:37 UTC (Wed) by joib (subscriber, #8541) [Link]

With my physicist pedantery hat on, no, I don't think that's exactly right. Now, my specialty was never high speed small scale circuits so I could talk out of my ass, but anyway; The speed of light in a conductor (that is, the velocity with which the electromagnetic wave propagates in the medium) is about 2/3 of the speed of light in vacuum, including for very narrow conductors. That is, very very fast. Similarly, the electron drift velocity also remains as it is for larger conductors, that is very very low, on the order of um or mm/s.

What changes then is that for these very small conductors you'll find on modern deep submicron integrated circuits, the relative capacitance of the conductor starts to rise (and the resistance doesn't go down as well as you'd like either). This leads to a phenomenon where when you apply a voltage on one end of the conductor, it takes longer until the voltage/current rises enough on the other end to be registered as a 0->1 flip. So in effect it appears as if the speed of signal propagation drops. I'm not sure how well the telegrapher's equations mentioned by malmedal in the sibling post applies to multi-GHz signals propagating in these very narrow conductors, but something like that is the gist of it. I don't think you need to apply quantum mechanics or study the behavior of individual electrons per se to understand this phenomena.

Another round of speculative-execution vulnerabilities

Posted Aug 9, 2023 22:41 UTC (Wed) by magnus (subscriber, #34778) [Link]

You can always add pipeline stages to transfer the data over several cycles to overcome this (at the cost of latency of course).

Another round of speculative-execution vulnerabilities

Posted Aug 9, 2023 19:01 UTC (Wed) by flussence (guest, #85566) [Link]

We're already seeing that in RAM speeds where there's a tug-of-war over having it on-CPU/off-CPU, each new DDR version has a huge jump in latency that needs to be papered over with more cache, DIMMs on large boards need buffer chips, DDR5 (iirc) now *requires* ECC to survive normal operation…

I wouldn't be surprised if a few years from now we start seeing hard NUMA become mainstream. Back to the days of having an empty slot close to the CPU because the manufacturer was too cheap to populate it!

Another round of speculative-execution vulnerabilities

Posted Aug 9, 2023 19:45 UTC (Wed) by willy (subscriber, #9762) [Link] (20 responses)

I told you why this argument was crap last time you espoused it. Knock it off.

Another round of speculative-execution vulnerabilities

Posted Aug 9, 2023 20:58 UTC (Wed) by Wol (subscriber, #4433) [Link] (19 responses)

Are you telling me we can communicate faster than light? Other people have explained the problem is worse than I thought, but from what I remember of your explanation you were saying that c wasn't a problem. (Well, it does seem it's not THE problem, but it does place a hard upper limit on chip frequency - of 5GH on a 3cm chip ...)

Cheers,
Wol

Another round of speculative-execution vulnerabilities

Posted Aug 9, 2023 21:40 UTC (Wed) by willy (subscriber, #9762) [Link] (4 responses)

You have two misunderstandings relevant to your argument. They're opposite in sign, so they come close to cancelling each other out.

The first is that things need to happen in a single cycle. An instruction that needs data from L3 cache can and will stall for hundreds of cycles. During that time the CPU will execute some of the other dozens of instructions that it has ready. It's something like six clock ticks to retrieve data from L1. Data in registers is ready to operate on and incurs no delay.

The second is that the speed of communication between different parts of the CPU have anything to do with the speed of light. Speed of electrons in copper is much slower. That's the part other people are telling you that you have wrong.

(There are other problems with your argument, but those are the big two)

Another round of speculative-execution vulnerabilities

Posted Aug 9, 2023 21:59 UTC (Wed) by farnz (subscriber, #17727) [Link] (3 responses)

Note that the speed of electrons is irrelevant; the voltage change that represents a change in state moves much faster than the electrons do, typically at around 60% to 70% of the speed of light in a copper conductor.

But the point about things not needing to happen in a single cycle is key; I can design my logic to account for propagation delays in the circuit, and have it work perfectly. This is what the timing diagrams that are part of any digital logic chip datasheet (and in every CPU datasheet since the 4004) are all about - how do I connect up the entire system's worth of logic such that the system's timing constraints are met?

Another round of speculative-execution vulnerabilities

Posted Aug 10, 2023 7:59 UTC (Thu) by Wol (subscriber, #4433) [Link] (2 responses)

So, as farnz says, you appear to have completely mis-understood my argument and are arguing against a straw man.

Signals are carried by photons (or em waves, same(ish) thing) so the speed of light IS relevant, although from what others have said the telegraph effect is probably more important, and

My argument has repeatedly been prefixed with "IF components need to communicate" so okay, I'm not necessarily talking about clock cycles, but a single communication cycle has that upper limit. I'm not always clear in what I say, I know that, but if you make no attempt to understand me, I can't understand you either. So IFF a communication cycle equals a clock cycle, 5GHz is the maximum clock possible between two random components in a chip. Of course, splitting a communication clock cycle into multiple clock cycles can speed OTHER stuff up, but it makes no difference to the speed at which a signal travels across a chip.

(And of course, without communication a chip can't work.)

Cheers,
Wol

Another round of speculative-execution vulnerabilities

Posted Aug 10, 2023 15:03 UTC (Thu) by farnz (subscriber, #17727) [Link]

A corollary of your argument is that Starlink satellites (communication clock rate of around 230 kHz) can be no higher than 1.3 km above the receiver, and Sky TV satellites (communication clock rate of 22 MHz or above) can be no higher than 13 metres above the receiver.

Another round of speculative-execution vulnerabilities

Posted Aug 10, 2023 16:23 UTC (Thu) by malmedal (subscriber, #56172) [Link]

Nobody is misunderstanding you, it is very easy to understand what you are saying. It's just that it is wrong.

However you seem to be unable to understand what people are saying, please read more carefully.

For instance there is no "telegraph effect" the "telegrapher's equations" are just Maxwell's equations applied to signals in a wire.

If you wish to be able to say anything intelligible about chips you need to understand what "pipelines" are in this context. This appears to be a major gap in your knowledge, you completely ignore it when people bring this up. It is not just a word, it is one of the fundamental concepts.

Already the 8088 had a pipeline, it is not a new concept.

Another round of speculative-execution vulnerabilities

Posted Aug 9, 2023 21:57 UTC (Wed) by malmedal (subscriber, #56172) [Link] (13 responses)

As I said, there is no hard limit, with a speed of 0.05mm per nanosecond would imply that a 3cm wide chip could only manage less than 2MHz. Yet the speeds of the latest processors are above 5GHz.

Another round of speculative-execution vulnerabilities

Posted Aug 9, 2023 22:04 UTC (Wed) by willy (subscriber, #9762) [Link] (11 responses)

To further emphasize this point, the speed of a PCIe gen 6 link is now 64GHz. And the maximum length of a trace is considerably longer than 3cm.

Another round of speculative-execution vulnerabilities

Posted Aug 9, 2023 22:21 UTC (Wed) by mathstuf (subscriber, #69389) [Link] (2 responses)

Another way to think about it: these signals are largely one-way communication. If it were round-trip, the speed of light would matter. But all that really matters is how accurately you can sample the wire for a signal (PCIe gen 6 can apparently do so 64 billion times a second).

And another way to sanity check things: if the size of space between communication endpoints limited your processing rate, we'd probably still be waiting for the first (quality) images from various Mars rovers.

At least I think that's somewhat closer than what Wol has as a model.

Another round of speculative-execution vulnerabilities

Posted Aug 10, 2023 8:00 UTC (Thu) by Wol (subscriber, #4433) [Link] (1 responses)

No. I did say the model is based on communication *between* components, ie there-and-back.

Cheers,
Wol

Another round of speculative-execution vulnerabilities

Posted Aug 10, 2023 20:28 UTC (Thu) by rschroev (subscriber, #4164) [Link]

If I were to send you a good old-fashioned letter, it would take a day or 4 (or somewhat less or more, I don't actually know; the exact value doesn't matter for this discussion); a reply from you to me would take 4 days too; there and back is then 8 days. Does that mean that I can only send you a letter once every 8 days? Only when my letter needs information from your reply; in that case I have to wait until I get your letter. But in all other cases, I can easily send new letters while old ones are still in transit. The distance between you and me sets a lower limit on latency, but does not affect bandwidth. It's the same for communication in computer systems.

Another round of speculative-execution vulnerabilities

Posted Aug 9, 2023 22:23 UTC (Wed) by malmedal (subscriber, #56172) [Link]

It's amazing how far they have pushed the technology, just ten years ago I would have said it was impossible to get that high outside of a lab-setting.

Another round of speculative-execution vulnerabilities

Posted Aug 10, 2023 10:49 UTC (Thu) by james (guest, #1325) [Link] (6 responses)

To further emphasize this point, the speed of a PCIe gen 6 link is now 64GHz.

I'm pretty sure this isn't technically correct, at least when talking about how far the signal propagates before the next signal is generated. PCIe 6.0 uses

PAM4 (Pulse Amplitude Modulation with 4 Levels) [...] a multilevel signal modulation format used to transmit data. [...] It packs two bits of information into the same amount of time on a serial channel. The utilization of PAM4 allows the PCIe 6.0 specification to reach 64 GT/s data rate and up to 256 GB/s bidirectional bandwidth via a x16 configuration.

It's basically the same concept as MLC versus SLC in flash.

This is the key difference between PCIe 5.0 (which used NRZ, or one bit per cycle) and PCIe 6.0. Both run at 32 billion signals per second: it's just with PCIe 6.0 each signal conveys two bits.

Your main point is correct, though -- this isn't what limits the length of a PCIe 6.0 connection.

Another round of speculative-execution vulnerabilities

Posted Aug 10, 2023 15:39 UTC (Thu) by kpfleming (subscriber, #23250) [Link] (5 responses)

And those 32 billion transfers per second are spread across 16 parallel lanes... so each lane is nowhere close to '64 GHz' :-)

Another round of speculative-execution vulnerabilities

Posted Aug 10, 2023 16:44 UTC (Thu) by malmedal (subscriber, #56172) [Link] (4 responses)

> And those 32 billion transfers per second are spread across 16 parallel lanes...

No. Each lane separately transmits 64 Gigabits per second.

Standard terminology is 64GT and and 32GHz.

Another round of speculative-execution vulnerabilities

Posted Aug 23, 2023 5:28 UTC (Wed) by JosephBao91 (subscriber, #157211) [Link] (3 responses)

Well, I think this statement is still not correct.
PCIe Gen5 is 32GT/s, with a frequency of 16GHz (tranfer data both posedge and negedge), and Gen6 uses PAM4 instead of NRZ, it transfers 2bits each time, and the frequency is still 16GHz, but the speed is 64GT/s.
And for hardware design, PAM4 16GHz is more difficulty compared with NRZ 16GHz.

Another round of speculative-execution vulnerabilities

Posted Aug 23, 2023 11:34 UTC (Wed) by malmedal (subscriber, #56172) [Link] (2 responses)

Do you have a reference for this? I haven't read the actual specs myself, but all the articles I've read say 5.0 is 32GHz e.g. https://www.tomshardware.com/reviews/pcie-definition,5754...

Another round of speculative-execution vulnerabilities

Posted Aug 23, 2023 12:13 UTC (Wed) by excors (subscriber, #95769) [Link] (1 responses)

I think that may just be a different meaning of "frequency": the sampling rate is 32GHz, while the Nyquist frequency (basically the frequency of the sine wave corresponding to the worst-case signal 1010101...) is 16GHz. Same as describing audio CDs as 44kHz (the sampling rate) or 22kHz (the highest audio frequency that can be encoded without aliasing) - both are reasonable in different contexts.

For example https://blog.samtec.com/post/why-did-pcie-6-0-adopt-pam4-... describes the Nyquist frequency of PCIe 5.0/6.0 as 16GHz. (The sampling rate is also the same in both, the difference is that in 6.0 each sample encodes 2 bits, so it's 16GHz Nyquist frequency with 32GHz sampling rate and 64GT/s data rate.)

Another round of speculative-execution vulnerabilities

Posted Aug 23, 2023 13:14 UTC (Wed) by malmedal (subscriber, #56172) [Link]

Thank you, sounds plausible.

Another round of speculative-execution vulnerabilities

Posted Aug 9, 2023 22:26 UTC (Wed) by farnz (subscriber, #17727) [Link]

For a very clear example, a geostationary TV satellite is typically transmitting at 22 MHz or higher symbol rates; if the signal has to propagate all the way from the satellite to the receiver before the satellite can start the next symbol, then geostationary orbit has to be no higher than 14 meters above the satellite dish. In practice, everything is designed to handle this delay, and thus it's fine.

If you insist on two-way communication, Starlink's signal has been partially reversed engineered, and has a symbol time of 4.4 µs; this corresponds to 1.3 km path length in free space. And yet, a Starlink satellite is around 550 km above the Earth's surface, for a propagation delay of around 1,800 µs - significantly more than the symbol time.