Thoughts on software-defined silicon [LWN.net]

Thoughts on software-defined silicon

Posted Feb 18, 2022 23:48 UTC (Fri) by Cyberax (✭ supporter ✭, #52523)
In reply to: Thoughts on software-defined silicon by NYKevin
Parent article: Thoughts on software-defined silicon

The thing is, the larger die is supposed to be more expensive. So the fact that you can just sell the fully-loaded die as an "economy" SKU means that you don't have competitors that would just undercut you on price or features.

This was very much the case up to about 5 years ago in case of Intel on servers.

These days? It just seems stupid.

to post comments

Thoughts on software-defined silicon

Posted Feb 19, 2022 2:00 UTC (Sat) by khim (subscriber, #9252) [Link] (23 responses)

> So the fact that you can just sell the fully-loaded die as an "economy" SKU means that you don't have competitors that would just undercut you on price or features.

Nope. You are forgetting an elephant in the room: price of set of photolithography masks and design cost for these masks. To make one set of masks you have to spend millions of dollars (singular millions). But to develop completely new set price can go to $100 million for relatively small chips and I wouldn't be surprised to know that with monsters like AMD/nVidia/Intel are producing they can go to $1 billion or more.

When non-recurring costs are so high it may make sense to produce transistors which are destined to be disabled and it would still be cheaper than to produce many physically different SKUs.

> These days? It just seems stupid.

Nope. As price of development grows higher and higher and relative cost of unused silicone lower and lower more and more companies would want to do that.

That's simple math.

I don't know under what kind of rock you were sitting, but just recall that endless disable/enable AVX512 saga: apparently it's cheaper for Intel to go with already prepared masks and just disable AVX512 in firmware rather then redo the production process!

Thoughts on software-defined silicon

Posted Feb 19, 2022 8:47 UTC (Sat) by Wol (subscriber, #4433) [Link] (11 responses)

I had a 3-core AMD a while back (just scrapped it), and I understood that a lot of these chips were actually 4-cores with a core disabled. Especially with a new design, if these things can be disabled by blowing a fuse, surely it makes sense to just disable stuff that fails QA and sell the resulting chip at a lower price point.

Cheers,
Wol

Thoughts on software-defined silicon

Posted Feb 19, 2022 11:41 UTC (Sat) by smurf (subscriber, #17840) [Link] (9 responses)

That makes economic sense when the silicon is broken, and already routinely used with clock speeds or (in the embedded realm) on-die Flash memory.

Selling perfectly working CPUs at a bargain and then charge for the "upgrade" is a slightly different kettle of fish, and frankly I can't contribute much to that discussion beyond "don't like it and will go to some pains not to use CPUs with this kind of anti-feature".

The non-turnoff-able IME is bad enough.

Thoughts on software-defined silicon

Posted Feb 19, 2022 12:15 UTC (Sat) by khim (subscriber, #9252) [Link] (8 responses)

> That makes economic sense when the silicon is broken, and already routinely used with clock speeds or (in the embedded realm) on-die Flash memory.

Except it wasn't broken. The majority of sold chips had perfectly functional additional cores you can enable (back when all it took to enable them was a pencil).

And I'm sure the majority of Ryzen 5 5600X sold today actually have 8 working cores, too. They have fully functional 32MB cache designed for all 4 cores and if your approach was the reason for their existence then we would have had some version of Ryzen with reduced cache and all 8 cores enabled. There are nothing like that because it doesn't make any economic sense: you couldn't fit these between Ryzen 5 5600X and Ryzen 7 5800X. They would be weird side-cousin to Ryzen 5 5600X which would just make buyers confused.

> Selling perfectly working CPUs at a bargain and then charge for the "upgrade" is a slightly different kettle of fish.

It's exactly the same only now you couldn't sell small percent of chips which are actually defective. But have less SKUs to manage.

I'm not sure if it would work or not, but it makes perfect economic sense.

> don't like it and will go to some pains not to use CPUs with this kind of anti-feature

Yeah, that's what stops these incentives. PR backlash. But as R&D prices for CPUs go up and prices of unused silicone goes down the incentive to switch to that model becomes more and more acute.

Actually the problems with Dark silicon almost guarantee that said model would become the norm eventually.

When you couldn't power all the transistors on the chip simultaneously for thermal reason the ability to pick between features A, B, and C (either of which can be enabled but not all simultaneously) make such scheme pretty attractive. You couldn't do that with millions of SKUs but can easily achieve with million of [potential] licenses.

And in that case the ability to enable features without license would become actively harmful: enabling all features simultaneously would just fry the chip and it would be pretty hard to prove in warranty service that this happened because of customer irresponsibility, not because of customer's misuse.

Thoughts on software-defined silicon

Posted Feb 19, 2022 12:58 UTC (Sat) by Wol (subscriber, #4433) [Link] (1 responses)

Going back many years, to microcode running on processors with speeds in KHz arena ...

50-series Pr1mes actually came with a microcode update if you were running INFORMATION (aka Pick) on them, it added a whole bunch of instructions specially optimised for handling strings, to make the database more efficient.

THAT would be an interesting feature on modern silicon :-)

Cheers,
Wol

Thoughts on software-defined silicon

Posted Feb 26, 2022 5:17 UTC (Sat) by flussence (guest, #85566) [Link]

We *almost* had a chance to see that: Zen2 chips emulate a few instructions (BMI2 set) in microcode, and they're pitifully slow to the point of being better to hand-roll in C. If they could've fixed it in an update it would've been interesting news… but maybe they just didn't care for such a niche thing.

Thoughts on software-defined silicon

Posted Feb 19, 2022 15:06 UTC (Sat) by mfuzzey (subscriber, #57966) [Link] (3 responses)

>enable features without license would become actively harmful: enabling all features simultaneously would just fry the chip and it would be pretty hard to prove in warranty service that this happened because of customer irresponsibility, not because of customer's misuse.

Surely this case could be handled in hardware by only accepting configurations enabling at most any 2 of features A, B, C (or whatever other thermal / power constraints exist). I don't see why this would require a license based system to be safe

Thoughts on software-defined silicon

Posted Feb 19, 2022 15:10 UTC (Sat) by khim (subscriber, #9252) [Link] (2 responses)

This would only work if you can, somehow, determine which combinations are actually safe and which may do harm based solely on some simple calculations.

If you need to do any kind of testing, then license would be perfect way to ensure that everything works perfectly.

That's minor issue, ultimately. Economic need to differentiate markets drives the effort to a much larger degree than technical needs.

Thoughts on software-defined silicon

Posted Feb 20, 2022 10:20 UTC (Sun) by NYKevin (subscriber, #129325) [Link] (1 responses)

Meh, from a technical perspective, there's no reason the licenses have to be sold individually. You could instead publish one set of "licenses" that everyone is allowed to download and copy freely, but they're all signed so you can't modify them. Maybe you also publish a separate signing key that allows users to make their own private "licenses" with untested combinations of features, with the proviso that "this might brick your hardware, don't ask for your money back if it does." Regardless of the specifics, an "open" version of this sort of thing could exist, if Intel wanted to make it.

The point is, you can decouple the technical aspect from the economic aspect, at least to some extent. Locked hardware exists because it is economically favored for it to exist. You can't "solve" the "problem" of locked hardware; it is not a technical problem in the first place. As long as those economic incentives continue to exist, it is inevitable that Intel, and other chip manufacturers, will produce and sell locked hardware.

Thoughts on software-defined silicon

Posted Feb 20, 2022 12:00 UTC (Sun) by khim (subscriber, #9252) [Link]

> You can't "solve" the "problem" of locked hardware; it is not a technical problem in the first place.

Yes, but you can go in the other direction: use solution designed to solve economic problem to solve technical problem, too.

> As long as those economic incentives continue to exist, it is inevitable that Intel, and other chip manufacturers, will produce and sell locked hardware.

True, but they would use technical need to keep hardware from breaking as justification for what they are doing.

Thoughts on software-defined silicon

Posted Feb 21, 2022 1:06 UTC (Mon) by nix (subscriber, #2304) [Link]

> When you couldn't power all the transistors on the chip simultaneously for thermal reason the ability to pick between features A, B, and C (either of which can be enabled but not all simultaneously) make such scheme pretty attractive. You couldn't do that with millions of SKUs but can easily achieve with million of [potential] licenses.
> And in that case the ability to enable features without license would become actively harmful: enabling all features simultaneously would just fry the chip

Yes, but... modern chips have been past this point for at least a decade. It wasn't solved with a licensing system: it was solved by having power management circuitry on the CPU that adjusted things (usually the operating frequency and voltage, but it is perfectly possible to imagine it also adjusting semi-invisible microarchitectural features like the number of execution ports) such that your code would keep running, just slower. Boost mode etc is the same thing: the fewer cores busy, the faster they're run, and even with lots of cores busy you can run fast briefly until the power management system turns down the CPU frequency to keep things cool enough. This is obviously *vastly* more efficient and flexible than some clunky licensing system would be: it allows for dynamic adjustment, which is something no licensing system like this could ever handle.

No, this is all about getting to make one SKU and sell it as several and allow upselling lower models to higher without needing hardware replacement. Shame that doing so requires cryptographic locks in the chip. (I doubt that anticircumvention measures are meaningful here: modern CPUs are almost impossible to analyze at the level you'd need to to crack this open anyway, or people would already have extracted much more significant private keys for firmware signing etc. Nobody has.)

Thoughts on software-defined silicon

Posted Feb 21, 2022 9:01 UTC (Mon) by marcH (subscriber, #57642) [Link]

> When you couldn't power all the transistors on the chip simultaneously for thermal reason the ability to pick between features A, B, and C ...

This problem has already been solved and it has not been solved by turning off features:

https://en.wikipedia.org/wiki/Intel_Turbo_Boost

Thoughts on software-defined silicon

Posted Feb 26, 2022 5:09 UTC (Sat) by flussence (guest, #85566) [Link]

> I had a 3-core AMD a while back (just scrapped it), and I understood that a lot of these chips were actually 4-cores with a core disabled.

I had one of those too. Stable as a rock with the extra core enabled for 12 years now, and it could even handle overclocking on top of that. The weird BIOS dance to unlock it put me off ever trying to run Coreboot on the thing though.

Thoughts on software-defined silicon

Posted Feb 19, 2022 9:12 UTC (Sat) by Cyberax (✭ supporter ✭, #52523) [Link] (10 responses)

> Nope. You are forgetting an elephant in the room: price of set of photolithography masks and design cost for these masks.

Sure. But this works only if you don't have competition. Because it's easy to undercut others on features otherwise. After all, it doesn't cost you anything extra to enable a feature that your competitor doesn't have.

We're already seeing this with AMD, it's steadily eating into Intel's server marketshare that had been unassailable up until ~3 years ago: https://www.tomshardware.com/news/intel-amd-4q-2021-2022-...

Thoughts on software-defined silicon

Posted Feb 19, 2022 11:57 UTC (Sat) by khim (subscriber, #9252) [Link] (9 responses)

> But this works only if you don't have competition.

If you have competition then it becomes even more important.

> After all, it doesn't cost you anything extra to enable a feature that your competitor doesn't have.

It would cost you a lot. If you would stop selling $299 Ryzen 5 5600X (which is, essentially, $449 Ryzen 7 5800X with two cores disabled) and would start selling Ryzen 7 5800X for $399 instead of having these two… you would lose both on people who were ready to spend $449 and on ones who were ready to buy something for $299.

I took example from AMD book, not Intel book to show that as competition grows more acute the need to disable features and sell crippled product grows, not diminishes. Market segmentation is powerful tool.

When AMD had no ability to make powerful CPUs (for many years AMD's CPU before Ryzen were awful) — because it was hard to sell even CPUs with even all features enabled (and it was losing money as the result). When AMD made something for the top tier — it started marked segmentation games (and immediately become profitable).

> We're already seeing this with AMD, it's steadily eating into Intel's server marketshare that had been unassailable up until ~3 years ago

And this has nothing to do with the fact that AMD presented 7nm EPYC Rome two years ago while Intel still couldn't make 10nm Xeons (10nm of Intel is more-or-less the same as 7nm of TSMC), but everything to do with the fact that EPYCs are less segmented? Dream on.

Thoughts on software-defined silicon

Posted Feb 19, 2022 15:11 UTC (Sat) by jhoblitt (subscriber, #77733) [Link] (3 responses)

It isn't just an issue of competition in the market. Fab capacity of each company and the overall market is a factor. Currently, the entire world is maxed out on wafer starts. So while it probably saves on masks, validation, etc. to cripple a perfectly functional 16core die to be sold as an 8core die, that means there is an 8core die area of silicon that is now lost. In a situation where every cpu produce can be sold, that lost real estate represents a potential 100% increase in gross margin.

Thoughts on software-defined silicon

Posted Feb 19, 2022 15:19 UTC (Sat) by khim (subscriber, #9252) [Link] (2 responses)

If we have lived in a world where every CPU can be sold then we would have seen similar craziness to GPU market where prices are 2x-3x recommended price.

And GPU makers don't, actually, embrace that craziness, they fear it because they know what comes next: governments would say that cryptocurrency mining is a criminal activity, GPU sales would drop through the floor and they would be selling them at loss for some time.

Fab capacity is strained not because it's impossible to build mode fabs, but because it's impossible to do that profitably: build cycles are many years, investment are measured in billions and unused fabs are not aging well.

Thus no, your reasoning doesn't make sense long-term. And CPU/GPU manufacturing is long-term process, it takes many years to develop CPU from scratch and year or two just to do minor alterations.

Thoughts on software-defined silicon

Posted Feb 19, 2022 16:10 UTC (Sat) by jhoblitt (subscriber, #77733) [Link] (1 responses)

I was just quoted a 26+ week lead time on any zen3 epyc cpu with >= 32cores from a major manufacturer. I ended up accepting zen2 cores in order to cut the estimated lead time in half. However, the price will start to float at the market rate 90days from the date the quote was issued. I won't even know the final cost until I'm invoiced for the shipment. If this isn't a CPU shortage, I don't know what one would look like.

The GPU and CPU makers are all bidding on the same fab capacity. When AMD reserves wafers at TSMC, that's capacity that is denied to Apple/Nvidia/Intel/etc. and vice versa.

The ability to build new fabs is not unlimited. Bleeding edge fabs need equipment from ASML who reports that they and their supply chain are already maxed out. The world is essentially already building new fab capacity at the maximum rate they can get lithography equipment.

The rumors are that TSMC is booked out *years* in advance at the 5, 4, and 3nm nodes. Are you arguing that AMD is going to produce 128core zen4s and then cripple perfect dies down to 16c core parts instead of taping out multiple designs that use the exact same logical blocks, when all that lost real estate could have instead been used for GPUs or ~5-7 additional CPUs?

Thoughts on software-defined silicon

Posted Feb 19, 2022 17:12 UTC (Sat) by khim (subscriber, #9252) [Link]

> I was just quoted a 26+ week lead time on any zen3 epyc cpu with >= 32cores from a major manufacturer. I ended up accepting zen2 cores in order to cut the estimated lead time in half.

That's nice piece of data. Let's try to decipher it, shell we?

You asked for latest-and-greatest cores and got promise to receive them when they would be made (you may not know that, but half-year or more is not atypical if we are talking about manufacturing of complex 7nm chip: they require almost hundred masks and application of one mask takes full day or more, depending on how much load fab experiences right now). The exact same thing happened with surprisingly good R300 back when it was hot — yet somehow back then noone was bawling their eyes out and complained about shortage of fabs. Everyone accepted that it was simple management miscalculation.
When you asked for less “hot” chips made by the exact same manufacturer in the exact same lab by the exact same producer you got shorter times because, apparently because there are surplus of these in production.
Yet it's very easy to buy consumer-grade chips with Zen3 core. The full nomenclature from lowly Ryzen 5 5600X (which my friend bought in India two days ago) to Ryzen 9 5950X (which I bought at the same time). Prices are less than what recommended prices are. Many of these are artificially crippled.

Doesn't look like shortage of CPUs to me, sorry.

> If this isn't a CPU shortage, I don't know what one would look like.

Indeed, you don't know that. When you need to pay 10x or 100x price to receive 250nm chip (like some automotive chips are selling for right now) and situation stays that way for years — then you can say there are shortage of chips.

Till then it's normal reaction of market on changes in demand and supply for something that takes years to produce coupled with customers who, naïvely, expect to buy the same thing with lead times measured in weeks.

JIT-manufacturing, both good and bad sides of it.

> The rumors are that TSMC is booked out *years* in advance at the 5, 4, and 3nm nodes.

Why do you say these are rumors? That's the reality. Latest nodes are always booked years in advance. Because fabs are extremely expensive yet they can only ask for extra-premium prices for latest nodes for a few years… nobody builds spares. You don't even need rumors to confirm something that was always true.

> The GPU and CPU makers are all bidding on the same fab capacity.

Nope. That's not true because of, as you have said yourself: fab capacities are booked years in advance. Essentially they are booked when labs are built or maybe a bit later. If they are already booked then there are no competition between customers.

> When AMD reserves wafers at TSMC, that's capacity that is denied to Apple/Nvidia/Intel/etc. and vice versa.

No, they just build more fabs.

> Are you arguing that AMD is going to produce 128core zen4s and then cripple perfect dies down to 16c core parts instead of taping out multiple designs that use the exact same logical blocks, when all that lost real estate could have instead been used for GPUs or ~5-7 additional CPUs?

No, I'm saying that if you don't want shortages you don't disrupt markets by printing trillions of unbacked money and using all tricks you can imagine to avoid 40-50% inflation… mechanism where you have to calculate number of chips you need to order five years in advance but where buyers expect lead times measured in weeks works if you can predict number of buyers, but if you break that mechanism… it stops working. This is completely unrelated to selling crippling CPUs. Ryzen 5 5600X and Ryzen 9 5900 are still being produced and sold despite the fact that you can sell the exact same chips as Ryzen 7 5800X and Ryzen 9 5990X.

Turning 128 core chip into 16 core chip wouldn't make any sense because AMD embraces chiplets architecture thus you can just use more or less chiplets. But if you want to shave off 2 cores or 4 cores... AMD does that and is happy to sell these at discount prices.

Thoughts on software-defined silicon

Posted Feb 19, 2022 19:07 UTC (Sat) by Cyberax (✭ supporter ✭, #52523) [Link] (4 responses)

> It would cost you a lot. If you would stop selling $299 Ryzen 5 5600X (which is, essentially, $449 Ryzen 7 5800X with two cores disabled) and would start selling Ryzen 7 5800X for $399 instead of having these two… you would lose both on people who were ready to spend $449 and on ones who were ready to buy something for $299.

Except that you can undercut Intel and sell Ryzen 7 5800X for $299 and immediately crush Intel. Otherwise Intel will just push you out with cheaper and faster CPUs.

In reality, cores are usually locked because they fail internal QA - this is indeed a perfectly valid strategy.

> I took example from AMD book, not Intel book to show that as competition grows more acute the need to disable features and sell crippled product grows, not diminishes. Market segmentation is powerful tool.

Not following. Once the competition starts biting, there's a pressure to unlock more and more features on the low-end of the spectrum.

Thoughts on software-defined silicon

Posted Feb 20, 2022 0:40 UTC (Sun) by khim (subscriber, #9252) [Link]

> Except that you can undercut Intel and sell Ryzen 7 5800X for $299 and immediately crush Intel.

Seriously? Immediately crush Intel? Weren't you celebrating 25% market share which AMD achieved in four years after introducing CPU which totally kicked Intel's ass?

If you would start selling Ryzen 7 5800X for $299 the only thing you would achieve would be stiffing your own R&D. Which would mean that your next CPU would be worse that CPU of competitors and you would “crush” yourself instead.

> In reality, cores are usually locked because they fail internal QA - this is indeed a perfectly valid strategy.

How do you know? We know for a fact from the times when it was possible to enable them that they were perfectly functional back then. We know from sales figures today that it's, most likely, still true today (Ryzen 5 5600X sales are much higher than Ryzen 7 5800X). Nope, these cores are not locked because they fail QA. They are locked to be able to sell CPUs at different price points.

> Once the competition starts biting, there's a pressure to unlock more and more features on the low-end of the spectrum.

Where would that pressure come from? It's zero-sum game. We only have two competitors (in mobile space there are more, but still not that many). If you stop offering CPUs at different price points then you wouldn't, suddenly, get more money from selling more CPUs because you couldn't produce more CPUs at the snap of fingers. You pay a lot to ramp up capacity and it happens slowly. If you would deprive yourself from receiving more money they you would just fail to produce enough CPUs, leave money on the table and would lose in the next round of competition.

That's how we ended up with just two manufacturers of x86 CPUs BTW.

Thoughts on software-defined silicon

Posted Feb 20, 2022 0:44 UTC (Sun) by excors (subscriber, #95769) [Link]

> Except that you can undercut Intel and sell Ryzen 7 5800X for $299 and immediately crush Intel.

Why would you want to crush Intel in the lower-end market segment where (per the economy seat analogy) their profit margin is approximately zero? The production cost of your 8-core chip will be similar to their 8-core-with-2-perfectly-good-ones-disabled chip, so you'll make no profit either. And now you can't do any market segmentation yourself, because you're already selling your most powerful chip for no profit. That seems much worse than copying Intel's segmentation strategy and getting a small share of high-margin segments.

Thoughts on software-defined silicon

Posted Feb 21, 2022 4:35 UTC (Mon) by timrichardson (subscriber, #72836) [Link] (1 responses)

IN pricing theory, you want to extract the maximum value. If you sell a product at only one price, you are forced to compromise at both ends: there are potential customers who would pay above your marginal price, so are potentially profitable but don't buy because your offered price doesn't meet the value they see in the product, and you leave money on the table from customers who have more value in your product than what you charge; they would have paid more if you asked, but you didn't.

The conventional answer to is create differentiated products at different price points. Intel does this, nothing new. It is commonly accepted that this is something like an happy accident of the variation in how CPUs are made. A comment above says that this is greatly exaggerated but even if not, the distribution of different working cores is not a random accident: it would be a deliberately chosen manufacturing strategy affected by how the production process is configured. I doubt that Intel or AMD is very surprised by the output they get, and I expect they could tweak their production process to avoid nearly all locked cores, although at the cost of lower total output ( I have some manufacturing experience behind that comment, but it think it is not a controversial statement). The difference between accepting binned manufacturing output or achieving the same thing with software seems really invisible to me. I find it ironic that a computer science community is having trouble with the concept of abstracting hardware into software.

Thoughts on software-defined silicon

Posted Feb 21, 2022 13:40 UTC (Mon) by gnb (subscriber, #5132) [Link]

Whether the difference is really invisible depends a lot on the implementation: are the feature enablements being sold liable to expiry or revocation by the vendor? If so the difference between that and actually owning the feature seems pretty clear-cut. I suspect what is making a lot of commenters on this article uneasy is a suspicion that this is part of a move to a rental model.