The real realtime preemption end game

Posted Nov 18, 2023 5:36 UTC (Sat) by mirabilos (subscriber, #84359)
In reply to: The real realtime preemption end game by mjg59
Parent article: The real realtime preemption end game

It works in practice on BSD, dmesg shows the previous run’s messages as well.

In practice here means x86 hardware like Thinkpads and other assorted PCs and servers whose BIOS will not overwrite the entire memory during warm reboot, as well as SPARCstations whose OpenBoot will similarily not clear the high-up memory used for the kernel log buffer.

to post comments

The real realtime preemption end game

Posted Nov 19, 2023 3:20 UTC (Sun) by Paf (subscriber, #91811) [Link] (7 responses)

“ It works in practice on BSD, dmesg shows the previous run’s messages as well.”

And surely this is only possible through the retention of data in memory over reboot! What other magic could do this?

Sorry, but I’d lay a lot of money this is done with storage.

The real realtime preemption end game

Posted Nov 19, 2023 5:15 UTC (Sun) by mirabilos (subscriber, #84359) [Link] (6 responses)

How much money are you willing to hand over?

https://mbsd.evolvis.org/cvs.cgi/src/sys/kern/subr_log.c?...
(I’m using a somewhat beefier mirror here to not get the main server slashdotted)
look for initmsgbuf near the beginning of the file, which gets a pointer to the RAM region.

It is called for SPARC from:
https://mbsd.evolvis.org/cvs.cgi/src/sys/arch/sparc/sparc...
(initmsgbuf called with an almost fixed (only the oldest systems avoid the first page) address…)

For i386, the call is at…
https://mbsd.evolvis.org/cvs.cgi/src/sys/arch/i386/i386/m...
… where msgbufp comes from…
https://mbsd.evolvis.org/cvs.cgi/src/sys/arch/i386/i386/p...
(the __OpenBSD__ ifdef) which sets the virtual address. The physical address (MMU mapping) is done somewhere between locore.s and there, and it looks to me like its location depends on the size of the kernel image, so you’d only get the log messages if you boot the same or a very similar-sized kernel after warm reboot.

And yes, it’s purely memory-based. It helps immensely in copying e.g. the remainder of a ddb(4) session (in-kernel debugger) out if you don’t have a serial console.

The real realtime preemption end game

Posted Nov 19, 2023 5:17 UTC (Sun) by mirabilos (subscriber, #84359) [Link]

Heh, and of course just as I posted this, the hoster fell off the ’net (including not only the hoster’s own homepage but also their status page, which is not hosted at a different site as is usual).

Oh well, it’ll be back at some point.

The real realtime preemption end game

Posted Nov 19, 2023 18:29 UTC (Sun) by kreijack (guest, #43513) [Link] (4 responses)

My understanding is that (at least in the x86 world), the memory is cleaned during the reboot.

So the problem is not to find a fixed area where store the data, but avoid that this area is cleaned up during a reboot.
And this cannot be done in a generic way.

The kind of reboot that I am talking, is the one that allow you to exit from a "crash", so I think that we are talking about an hard reboot. And an hard reboot implies the memory cleanup.
Think if this wouldn't exists: this would allow to extract from the memory some secret with a simple reboot at the "right time"; it would be a giant security hole.

The pstore back-ends in the x86 are mostly two: the first one relies on the UEFI variable storage; the second one relies on the ACPI-ERST, which is like a flash memory.

The real realtime preemption end game

Posted Nov 19, 2023 18:35 UTC (Sun) by mirabilos (subscriber, #84359) [Link] (3 responses)

I’m afraid your understanding has always been wrong, then.

The real realtime preemption end game

Posted Nov 19, 2023 18:40 UTC (Sun) by mirabilos (subscriber, #84359) [Link] (2 responses)

Hm, perhaps a bit more elaborating.

Yes, it’s not a persistent storage like the BIOS (or EFI) settings.

No, a boot does not imply memory cleaning (except for memory used during boot, of course). It usually does imply some kind of memory test, and several kinds of memory amount probing by different places in the boot process, but these are often nōn-intrusive enough to keep the memory contents.

A cold boot does have empty memory simply because the memory had no power and the memory controller likewise did not refresh the memory banks.

A warm reboot does not have a period of such, so the memory is *usually* retained.

A hard reboot can fall into either category, depending on how it is executed and wired. The usual power button long-press will be a poweroff followed by a mostly-cold boot; a watchdog reboot, or if the kernel crashed but is still able to reboot-ish (even if just by causing a triple-fault) can be warm reboots (this mostly depends on the memory controller to continue refreshing the memory during that, and of course the firmware not overwriting it).

The real realtime preemption end game

Posted Nov 20, 2023 19:57 UTC (Mon) by kreijack (guest, #43513) [Link] (1 responses)

> A warm reboot does not have a period of such, so the memory is *usually* retained.

I think that the key word is "*usually*". On my UEFI system I build a UEFI program that dump the first 4 bytes of the following address:
- 3GB
- 7GB
- 14GB

Then it sets these bytes to a specific value, and then it dump again.

What I saw is:
1) the first time that I run the program, I saw "random values", like 0 and other non 0 values.
2) the 2nd time that I run the program, I saw the same values that I set in the first iteration.

This proof that UEFI doesn't reset the memory between different program invocation.

Then I "warm rebooted" the system, and I saw the "random values" at 1). So it seemed that in my system the memory is cleared between the reboot.

What I'm telling is that at least some bios clears the memory. In may case (a ASUS B550 desktop mainboard) it seems that the BIOS clear the memory.

What I found is that it is possible to force the BIOS to not clear the memory after a reset [1]. But again this is not typically what happens after a crash; after a crash you push the reset physical buttons.

[1] https://stackoverflow.com/questions/36608101/does-a-soft-...

The real realtime preemption end game

Posted Nov 21, 2023 23:31 UTC (Tue) by mirabilos (subscriber, #84359) [Link]

Yes, that’s precisely what I meant with “usually”: a sufficient amount of systems keeps sufficient amounts of memory alive to make this feature worth being in existence, even if counter-examples exist and no spec supports this usage.

The reset button as the only way out of a crash is such a PC thing though. Some machines have watchdogs, and some have something like ddb(4) on BSD or SysRq on Linux that allow for warm reboots even in the face of a crash.

The real realtime preemption end game

Posted Nov 19, 2023 5:40 UTC (Sun) by mjg59 (subscriber, #23239) [Link] (1 responses)

It works as long as your firmware behaves in a specific way, something no specification requires of it.

The real realtime preemption end game

Posted Nov 19, 2023 18:22 UTC (Sun) by mirabilos (subscriber, #84359) [Link]

Yes, of course. Which is why I said that “it works in practice”: it works on a sufficiently large array of machines that OpenBSD (and probably NetBSD before) chose to implement it so, and even if it doesn’t work on one machine it’s no big loss.