LWN: Comments on "OpenBSD kernel address randomized link"

OpenBSD kernel address randomized link

cesarb — Wed, 19 Jul 2017 14:50:22 +0000

One complication of doing something similar for Linux, is that the Linux kernel depends on the link order of its object files. A long time ago, the order in which built-in drivers were initialized was defined by listing them in a central C file; this was changed so each driver adds a pointer to its initialization function into a special section of the object file, with the linker merging it all together. The initialization order is no longer explicit in the code; it's now the order in which the objects are linked together, defined by the makefiles.

OpenBSD kernel address randomized link

ikm — Mon, 17 Jul 2017 17:32:52 +0000

As far as I remember ext4 remounts the filesystem read-write to replay the journal in this case (and subsequently remounts it back r/o), or refuses to mount the filesystem otherwise. I'd expect XFS to do the same, even though I'm just guessing at this point. I's just that putting journal checks thoughout all hot paths of the fs code just to implement read-only mounts doesn't sound very practical. It'd also be quite brittle/error prone.

OpenBSD kernel address randomized link

Jonno — Mon, 17 Jul 2017 03:32:01 +0000

> Throughout this process, the system stays atomically in one of two states:

Actually, that is not true. After step 4 the file will be safely on disk, but the directory will not be. If power fails after you did the rename but before the directory is synced the directory listing can be anything. As far as POISIX is concerned, a directory is a file whose content is an implementation defined mapping from filename to inode. In step 4 you are editing that content in-place. The dangers of doing this is exactly the same as the dangers of editing any other file in-place.

As a practical matter, if the directory is no larger than a single block, writes are going to be atomic (at least at the interface level, what the drive does internally is another matter), but for large directories all bets are of. This race condition can not be entirely avoided, but can be shortened by using fsync on the directory immediately after step 4.

OpenBSD kernel address randomized link

farnz — Sun, 16 Jul 2017 12:44:34 +0000

To be completely pedantic, POSIX is fine with your system exploding in a shower of sparks after power loss or a hard crash. However, if you've implemented the concurrent access rules POSIX requires, and you've implemented the data integrity after sync rules that POSIX requires, then assuming that you've implemented any sort of reasonable behaviour after a power loss or hard system crash, and that the hardware does not fail, you'll see atomicity.

OpenBSD kernel address randomized link

lsl — Sat, 15 Jul 2017 21:46:42 +0000

> Throughout this process, the system stays atomically in one of two states:

AIUI, the atomicity guarantees are with respect to other processes inspecting the file system concurrently. POSIX guarantees nothing whatsoever about the state of the system following a hard system crash or power loss. Finding your kernel image replaced by a picture of Rick Astley should be perfectly fine as far as POSIX is concerned.

OpenBSD kernel address randomized link

rahvin — Sat, 15 Jul 2017 00:08:30 +0000

For full chain Secure boot signing I thought you already had to sign every kernel.

The shim loads the microsoft signed key, has the bios validate and loads the bootloader, but to retain a secure boot codepath then the bootloader needs to be signed by a key the shim knows, then the bootloader loads the kernel which is signed by a key the bootloader knows, etc, etc, etc. Secure boot arguably isn't much use if you don't sign all the way down the chain to the boot kernel at least. Then if you want to be really secure you have to sign modules, programs and everything after the kernel with a key the kernel can validate. Fully implemented you shouldn't be able to get anything executed during boot that isn't cryptographicly signed and presumably unmodified.

The problem's always been that everything's got to be signed (and presumably the private key isn't available on the disk for someone to start signing stuff) so you can verify nothing's been tampered with. This signing is a non-trivial task. The cool thing is if you can get something like this working you've got a lot of confidence up to that point that nothing unapproved is loaded (although this doesn't protect you from bugs at all). The hard part is getting this simple enough that it's not a 2 day effort to patch security vulnerabilities. I believe they're working towards this goal with the future of secure boot and TPM.

OpenBSD kernel address randomized link

rahvin — Fri, 14 Jul 2017 23:55:29 +0000

Presumably the current linux kernel drivers can adequately handle XFS in it's preferred order (I would assume so as I use XFS on a partition on my home server). Given this I believe the parent poster was simply lamenting that Grub2 doesn't just use this same behavior and has implemented a driver that's not following the appropriate standard leading to problems.

Without knowing anything about Grub's code I would say it shouldn't be that difficult to change the Grub2 behavior to the proper methodology without significant speed impacts because the kernels already got all that sorted out. The problem is likely that the Grub2 project isn't sexy, has limited contributors and it's not a popular issue, the single greatest weakness of getting any problem addressed in open source when you can't fix it yourself. Feel free to correct me if I'm wrong.

OpenBSD kernel address randomized link

Nahor — Thu, 13 Jul 2017 17:34:15 +0000

Or the "kernel image" could be the linker + .o files. The bootloader would remain unchanged and load everything in RAM. Then the linker would move the .o files around and link them then execute the normal kernel code.

I think the kernel already has a linker (for modules).
And this also works for IoT devices, where the filesystem is likely RO.

DAX would be an issue though.

All that said, the solution seems so obvious that I'm sure I'm missing something critical...

OpenBSD kernel address randomized link

tialaramex — Thu, 13 Jul 2017 13:22:35 +0000

Hmm, this may be the wrong place to ask but

Does XFS have read-only mounts? How do they interact with this... let's call it an optimisation?

Historically it seems like XFS log is not very big, so a correct implementation could read the entire thing into RAM, and consult it throughout all further operations. Obviously that's going to be both slow and error-prone, because it's basically taking a lot of situations where there's a happy path and splitting them into two similar but different happy paths one of which is very difficult to test. Is that really how all bootloaders designed to work with XFS do it?

I presume that the XFS authors focused instead on, as you say, "replaying the log at mount time", converting the mount to read-write on the fly which is cool but obviously there is no promise that's _possible_ let alone a good idea.

OpenBSD kernel address randomized link

farnz — Thu, 13 Jul 2017 13:20:20 +0000

That sort of buggy behaviour is precisely the sort of bug I was thinking of when I said "Assuming OpenBSD implements the expected POSIX behaviours"; it sounds like the combination of GRUB2 and Linux does not implement the expected POSIX behaviours.

And I'd agree with you on the blame; while I can see why GRUB2 wouldn't want to do a log replay itself, it should at least check the log first, then the backing store if the log has no recent changes - that way, it won't read something unexpected.

OpenBSD kernel address randomized link

nix — Thu, 13 Jul 2017 12:07:21 +0000

Note that exactly this behaviour is likely to cause failure with the combination of XFS and GRUB2, because GRUB2 reads the filesystem without replaying the log (XFS journal), but XFS (IMHO sensibly) assumes that once something has hit the log it is committed to disk, so speeds up shutdown by not doing a full log replay until mount time. So GRUB2 reads an undefined, uncommitted God-knows-what which probably doesn't include the latest writes, and implodes.

This is, IMHO, entirely GRUB2's fault, for implementing an XFS filesystem reading driver that doesn't actually read the filesystem in the same way that XFS itself has (and *always* has: this is not a recent optimization).

OpenBSD kernel address randomized link

matthias — Thu, 13 Jul 2017 12:03:47 +0000

When I read the text I first thought: good idea and I saw no problems with secure boot until the rc script came into play. Then I thought O crap.

For secure boot I would design sth. like a boot loader and linker (opposed to just a boot loader). In this scenario, a kernel is a signed tar.gz (or sth. equivalent) containing a buch of .o files. The boot loader and linker would
- be signed itself (signature verified by secure boot)
- check the signature of the kernel
- untar the kernel
- link the kernel
- execute the kernel

Probably this can be achieved with some initramfs and the kexec mechanism. To avoid some overhead (and problems with kexec not always being available), one could implement a grub module. Actually, this is roughly what I was thinking of when starting to read the article.

There could be one additional problem: Where to get good random data in very early boot? Apart from this problem, my solution should be cleaner. E.g., it would also work for embedded systems, without writeable permanent storage. However, it will be more work to implement.

OpenBSD kernel address randomized link

dskoll — Thu, 13 Jul 2017 10:32:36 +0000

How would this interact with so-called Secure Boot? I guess you'd have to sign each new kernel? That could be slightly annoying because you probably don't want a plaintext signing key lying around.

OpenBSD kernel address randomized link

farnz — Thu, 13 Jul 2017 09:09:42 +0000

Assuming OpenBSD implements the expected POSIX behaviours (and I'd be surprised if it didn't - this is one of the saner bits of POSIX), this is easy to arrange:

Boot from kernel.
Link new kernel into same filesystem, under the name kernel.new.
Use fsync and/or sync to ensure that all data is written out to disk.
Use rename to atomically replace kernel with kernel.new.

Throughout this process, the system stays atomically in one of two states:

kernel is the currently running kernel, not yet replaced by kernel.new. kernel.new may or may not exist, and may or may not be a bootable kernel at this point.
kernel is the newly built kernel.

You can increase safety by first using link to ensure that kernel.booted is the last kernel to boot as far as running the linker, and teaching the bootloader to try it if kernel fails to boot for any reason - that way, you get a second chance if the new kernel is mislinked due to a bug.

OpenBSD kernel address randomized link

mjthayer — Thu, 13 Jul 2017 05:04:12 +0000

Half off-topic; if I remember correctly (that means I am too lazy to check it) Intel recently introduced processor functionality to protect against ROP attacks (or return address manipulation generally) by caching function return addresses in a safe place and checking that they matched the address on the stack before returning. Couldn't something similar be achieved by duplicating the return address at function entry at the bottom of the local variable area, where it would be harder to overwrite, and comparing it before return? Of course, it would not be as strong as what Intel does in hardware, and it would be possible to overwrite the return address and the duplicate from the frame below with some care.

OpenBSD kernel address randomized link

lkurusa — Thu, 13 Jul 2017 04:04:30 +0000

Why? Could you not just boot into an older kernel via GRUB?

OpenBSD kernel address randomized link

ikm — Thu, 13 Jul 2017 00:26:51 +0000

> After each boot, an rc script links a new kernel in the background and installs it.

Sounds scary. That would require the installation to be fully atomic, as otherwise an unfortunate power loss at this moment could leave the system in a permanently unbootable state.