Stable kernels to address Zenbleed released [LWN.net]

Stable kernels to address Zenbleed released

Posted Jul 24, 2023 17:56 UTC (Mon) by wsy (subscriber, #121706) [Link]

I just tried to run the POC on a Zen2 cloud VM. Got overwhelmed by the amount of data it spilled.

Stable kernels to address Zenbleed released

Posted Jul 25, 2023 12:43 UTC (Tue) by whoami (guest, #166229) [Link] (3 responses)

My CPU is listed as affected. It says in the documentation that microcode can be built into the kernel image:

https://docs.kernel.org/arch/x86/microcode.html#builtin-m...

So I added "amd-ucode/microcode_amd_fam19h.bin" in CONFIG_EXTRA_FIRMWARE, but on boot it still says "Zenbleed: please update your microcode for the most optimal fix". Later on in dmesg it says patch_level=0x08600103. I double checked CONFIG_MICROCODE_AMD=y.

So apparently building a monolith kernel and loading AMD microcode during boot does NOT work as of 6.4.6

Stable kernels to address Zenbleed released

Posted Jul 25, 2023 15:21 UTC (Tue) by wsy (subscriber, #121706) [Link] (1 responses)

zen2 is family 17h

Stable kernels to address Zenbleed released

Posted Jul 25, 2023 15:37 UTC (Tue) by whoami (guest, #166229) [Link]

I know that. It's pointed out at this Gentoo Wiki article:

https://wiki.gentoo.org/wiki/AMD_microcode

Same result. I've looked at the sources but haven't found a way to fix this yet.

Stable kernels to address Zenbleed released

Posted Jul 27, 2023 15:25 UTC (Thu) by MarcB (guest, #101804) [Link]

Note that there is no microcode update for Ryzen/Ryzen PRO yet. Only EPYC got the firmware fix, so far.

Stable kernels to address Zenbleed released

Posted Jul 25, 2023 17:03 UTC (Tue) by alspnost (guest, #2763) [Link] (3 responses)

Confused.com - because it says "Zen 2", but then the list includes some Zen 3 chips, like my Ryzen 5 5600. I've just installed the 6.4.6 kernel, and Ubuntu's amd-microcode package update, and rebooted. There is no change to the output during boot, which still shows "microcode: CPU0: patch_level=0x0a50000d" on all cores. Am I still affected?

Stable kernels to address Zenbleed released

Posted Jul 25, 2023 19:03 UTC (Tue) by abatters (✭ supporter ✭, #6932) [Link] (1 responses)

> the list includes some Zen 3 chips

https://arstechnica.com/information-technology/2023/07/en...

"But AMD's habit of mixing-and-matching processor architectures in recent CPU generations means that there are some Zen 2 chips sprinkled across the Ryzen 4000, 5000, and 7000 lineups as well, affecting some new systems as well as older ones."

Stable kernels to address Zenbleed released

Posted Jul 25, 2023 21:55 UTC (Tue) by alspnost (guest, #2763) [Link]

Yes, confusing indeed. I really wish AMD didn't do this, but marketers seem to have the upper hand!

Stable kernels to address Zenbleed released

Posted Jul 25, 2023 19:21 UTC (Tue) by willmo (subscriber, #82093) [Link]

The 5600 (which is the Vermeer SoC) has the Zen 3 core as you say, so it’s not affected.

In some product lines, AMD has reached Intel levels of naming confusion, and the way they listed the affected products in the advisory didn’t help. In the 7000 mobile series they’ve changed schemes and we’ll see if it helps at all. So for example:

* 5300U/5500U/5700U (Lucienne SoC): Zen 2 core, affected
* 5400U/5560U/5600U/5800U and 5xxxH(S/X) (Cezanne SoC): Zen 3 core, not affected
* 7320U/7520U (Mendocino SoC): Zen 2 core, affected
* Other 7000-series products: generally Zen 3 or 4 cores, not affected

Basically, just consult Wikipedia or other preferred source to figure out which core you have, and if it’s Zen 2 then it’s affected. :-)

Stable kernels to address Zenbleed released

Posted Jul 25, 2023 18:22 UTC (Tue) by NightMonkey (subscriber, #23051) [Link]

I confined the exploit (wow) and the fix (yay) on Gentoo on my Acer Aspire 5. Anyone know if MX Linux is working on a kernel and micro code upgrade to fix this? Thanks.

Microcode version check vs code update

Posted Jul 25, 2023 22:35 UTC (Tue) by garloff (subscriber, #319) [Link] (10 responses)

Hmm,
The zenbleed message and the MSR setting happens before the microcode update happens (with early initrd).
So, unless the ucode update is done by the BIOS, we'll always use the MSR workaround on zen2?
Unless of course the updated ucode also ignores the MSR setting.
(I expect the MSR setting to cost a few % performance, that's why this question seems relevant to me.)

Microcode version check vs code update

Posted Jul 26, 2023 8:19 UTC (Wed) by bof (subscriber, #110741) [Link] (7 responses)

I just updated microcode, by placing the updated file from linux-firmware.git into /lib/firmware/amd-ucode/ and rebuilding initrd.

BEFORE:
> there:~ # dmesg|egrep '(microcode updated early|Zenbleed)'
> [ 0.047559] Zenbleed: please update your microcode for the most optimal fix
> [ 3.187620] microcode: microcode updated early to new patch_level=0x08301072

AFTER:
> there:~ # dmesg|egrep '(microcode updated early|Zenbleed)'
> [ 3.187753] microcode: microcode updated early to new patch_level=0x0830107a

(EPYC 7452 processors)

So I guess it's All Good.

Microcode version check vs code update

Posted Jul 26, 2023 9:13 UTC (Wed) by garloff (subscriber, #319) [Link] (5 responses)

That looks like magic to me.
The zenbleed handling code can look 3s into the future, knowing that a fixing microcode update will be installed?

Microcode version check vs code update

Posted Jul 26, 2023 11:07 UTC (Wed) by gmgod (guest, #143864) [Link] (2 responses)

Depending on how your initrd is generated, it might literally start by loading the microcode and then start itself.

Microcode flashing is supposed to happen in very early boot, before initrd is even loaded. All recent bootloaders allow you to flash microcode first, then proceed to initrd. Unified kernels do that automatically too (if they are built with microcode).

Microcode version check vs code update

Posted Jul 26, 2023 14:05 UTC (Wed) by whoami (guest, #166229) [Link] (1 responses)

If by unified kernel you mean an image with initramfs built-in, I've been doing that for years. Will try adding in microcode there, thank you.

Microcode version check vs code update

Posted Jul 26, 2023 14:12 UTC (Wed) by gmgod (guest, #143864) [Link]

Yep that's it! :)

Microcode version check vs code update

Posted Jul 26, 2023 13:10 UTC (Wed) by bof (subscriber, #110741) [Link] (1 responses)

It seems, from cursorily reading of the code, that the microcode/amd.c early loader simply avoids printk'ing, instead it just puts the patch level into a global, and later on, that's what triggers the output at 3 seconds. The intel loader, on the other hand, does it right away. Why - no idea.

Microcode version check vs code update

Posted Jul 26, 2023 13:36 UTC (Wed) by garloff (subscriber, #319) [Link]

Hi bof,
thanks for this insight. So just the printk happens later (and after the zenbleed check/printk), not the microcode loading.
That would explain the magic.
I have no theory about the reason for the delayed printk -- console output is limited before certain hardware init has happened, but that's why the kernel can do some buffering ... maybe the code comes from a time when ucode loading was done before printk was safe to use?
@all: I use the early_initrd for microcode updates as everyone else does. This does not change the fact that the zenbleed message happens 3s before the ucode update message.

Microcode version check vs code update

Posted Apr 6, 2024 22:05 UTC (Sat) by akostadinov (guest, #48510) [Link]

Hi. Did you observe a positive performance improvement?

After some fiddling I managed to install a new microcode on a Ryzen 5 3600 but I actually see a slight (1%-1.5%) negative performance effect on multi-thread performance (according to geekbench).

It was tricky to find my CPUID, funnily at the end I just looked it up on http://instlatx64.atw.hu/

Afterwards downloading a new microcode from CPUMicrocodes [1] and merging it into `/usr/lib/firmware/amd-ucode/microcode_amd_fam17h.bin` with amd-ucodegen and amd_ucode_info was more straightforward.

$ sudo dmesg | grep -i microcode
[ 1.125508] microcode: Current revision: 0x08701033
[ 1.125510] microcode: Updated early from: 0x08701030

I wrote a blog to describe how exactly to update [2].

[1] https://github.com/platomav/CPUMicrocodes/tree/master/AMD
[2] https://rboci.blogspot.com/2024/04/zenbleed-microcode-upd...

Microcode version check vs code update

Posted Jul 26, 2023 11:03 UTC (Wed) by gmgod (guest, #143864) [Link]

The modern, recommended, way to apply microcode is in early boot (i.e. by your bootloader, even before any software is ran, so even before initrd).

This is what is done automatically in unified kernels as well.

If you don't do that, and unless the kernel is reworked to start by loading the microcode, data will leak.

Microcode version check vs code update

Posted Jul 26, 2023 13:19 UTC (Wed) by whoami (guest, #166229) [Link]

Correct.

You can try adding in "load_ucode_bsp();" or "load_ucode_ap();" right above "zenbleed_check(c);" on line 1133 of arch/x86/kernel/cpu/amd.c (and don't forget to add in "#include <asm/microcode.h> somewhere at the top). I'm not sure which one is correct so please share results if you want.