Files with negative offsets [LWN.net]

Every open file on a Linux system has an associated offset - the current read or write position within that file. The virtual filesystem code, when dealing with file positions, performs some basic checks, such as ensuring that the position is not negative. After all, what sense does it make to talk about a file position before the beginning of the file?

As it turns out, there is a situation where a negative file position makes sense. Special files (such as /dev/mem and /dev/kmem) provide a window into the system's main memory. The "position" within these files corresponds to the address of the memory of interest. The interesting thing is that, on the x86_64 platform, addresses can be negative numbers.

This comes about as follows: this architecture currently uses a 48-bit address space. The hardware sign-extends the uppermost bit, however, so any address with that bit set will turn into a negative number. The x86_64 Linux port uses the upper bit to mark kernel space, so kernel addresses are, in fact, negative. A quick look at /proc/kallsyms confirms this:

    ffffffff80100000 T startup_32
    ffffffff80100100 T startup_64
    ffffffff801001a0 T initial_code
    ffffffff801001a8 T init_rsp
    ffffffff801001b0 T early_idt_handler
    ...

The end result is that using /dev/kmem on an x86_64 system is difficult; any attempt to seek into kernel space will yield an error.

The clear fix is to modify the VFS layer to let negative file positions be passed through to the underlying filesystem or device driver. The problem with doing that in a general way, however, is that not all code (especially in drivers) is prepared to deal with a negative offset. Suddenly exposing that code to negative offsets could open up no end of bugs and security problems. So the real solution, as worked out by Al Viro and Linus Torvalds, is to add a new flag for the file structure called FMODE_ANY_OFFSET. This flag can only be set within the kernel; user space has no access to it. So the /dev/kmem driver will be able to set the flag and work with the full range of offsets, but, for the rest of the system, nothing will change.

Index entries for this article
Kernel	Filesystems
Kernel	x86_64

to post comments

Files with negative offsets

Posted Jun 2, 2005 7:09 UTC (Thu) by xoddam (subscriber, #2322) [Link] (8 responses)

A negative offset from the beginning of a file really is meaningless, and
a cleaner solution to this problem would be to make the offset type
unsigned so that addresses with the high bits set are simply high
positive integers. After all, very very very very large files might
conceivably have meaningful contents past offset 2^63 (not).

Unfortunately the semantics of the loff_t type require a signed integer
because it may also describe an offset from the current position in the
file, or from the end of the file.

Short of defining a new unsigned type for the current offset into a file,
Linus' solution is about as correct as can be in this situation.

Files with negative offsets

Posted Jun 2, 2005 11:29 UTC (Thu) by brother_rat (subscriber, #1895) [Link] (7 responses)

I think in this case, I think negative numbers can be taken to mean "from the end of the address space", so to say they are "entirely meaningless" isn't really true.

Files with negative offsets

Posted Jun 2, 2005 22:22 UTC (Thu) by giraffedata (guest, #1954) [Link] (6 responses)

Nobody said that negative numbers or negative addresses are entirely meaningless. The statement was that negative file offsets are meaningless, and it's hard to argue with that. Besides defying common sense, they defy POSIX. The goal of this interface, even for the special files, is to implement the POSIX standard and reap the benefits of presenting a uniform interface to application programs.

While it's nice that Linus and Al have saved users of all the normal files from this assault on common sense, it would have been nicer to spare the users of ALL files. There are plenty of interfaces the kernel could provide that access a memory location given an address, but this one is about making memory look like a file. A file doesn't have data before its beginning.

There are plenty of ways to map a 48 bit address space into a 63 bit file offset space. It's a kmem device driver problem, not a VFS problem. VFS was already as right as it could be.

Incidentally, another reason that loff_t has to be signed is that it is a return value from the lseek system call and C library function. As a positive number, it is a file offset; as a negative number, it indicates a failure. In the very special case of /dev/kmem, this turns out not to be critical because the particular negative numbers that would be used for failures are not addresses that Linux uses.

Files with negative offsets

Posted Jun 3, 2005 12:23 UTC (Fri) by farnz (subscriber, #17727) [Link] (5 responses)

The pain here is that AMD64 defines a 64-bit virtual address space; Opteron and Athlon64 only have a 48-bit virtual address space (40-bits physical), and define their own mappings from 48-bits to 64-bits. The choice is therefore to break the offset semantics and keep to AMD's documentation, or to define a new mapping for 48-bit to 63-bit, and have to rework it for a later processor with a 64-bit virtual address space.

Files with negative offsets

Posted Jun 3, 2005 15:09 UTC (Fri) by giraffedata (guest, #1954) [Link] (4 responses)

The choice is therefore to break the offset semantics and keep to AMD's documentation, or to define a new mapping for 48-bit to 63-bit, and have to rework it for a later processor with a 64-bit virtual address space.

I think those aren't the only options. For a 64 bit signed address space, I'd probably go with two files -- /dev/kmem where offset = address and /dev/kmem_minus where offset = -address .

Note that the negative offset solution Linus chose is not compatible with the full architectural specification either, because it doesn't work for addresses -1 - -4095, and not only the architecture but actual existing CPUs have such addresses.

(Reminder: -1 - -4095 don't work because in the loff_t type, those are error codes. I'm told it's not an immediate issue because Linux doesn't use those addresses).

gnikniht drawkcab

Posted Jun 7, 2005 7:20 UTC (Tue) by xoddam (subscriber, #2322) [Link] (1 responses)

> /dev/kmem_minus where offset = -address .

Now that's a bit silly. You'd have to *reverse* the contents of the
memory in the device driver when reading and writing!

offset = -address silly

Posted Jun 7, 2005 17:09 UTC (Tue) by giraffedata (guest, #1954) [Link]

Well, OK. Then: /dev/kmem_upper, where offset = 2^63 + address .

Files with negative offsets

Posted Jun 7, 2005 7:31 UTC (Tue) by farnz (subscriber, #17727) [Link] (1 responses)

Which CPUs currently have a 64-bit virtual address space? To the best of my knowledge, all AMD64 CPUs to date have a 48-bit virtual address space.

Two files is dangerous in terms of future compatibility, as AMD64 sign-extends addresses; thus, at the moment, I can read from 0xffffxxxxxxxx, where x is any hex digit, and get kernel addresses. In future, this becomes a userspace address (when the virtual space is extended to 49 or more bits), and my code suddenly breaks on me.

Linus's current solution works fine for as long as AMD64 processors have (or use) no more than 52-bits of virtual address space. I think the only long terms solution is to switch to 128-bit offset types, but that's going to be painful.

Files with negative offsets

Posted Jun 7, 2005 17:31 UTC (Tue) by giraffedata (guest, #1954) [Link]

Which CPUs currently have a 64-bit virtual address space?

No one said any CPU currently has a 64-bit virtual address space.

I did say current AMD64 CPUs have addresses -1 - -4095. All of them do.

The reason this Linux architectural gap doesn't matter for the /dev/kmem case is that Linux chooses not to use the address range -1 - -2^20. (I'm not sure why -- I just looked up a memory map just now to verify what I was told about Linus' solution working in spite of the -1 -4095 hole is true).

Two files is dangerous in terms of future compatibility, as AMD64 sign-extends addresses; thus, at the moment, I can read from 0xffffxxxxxxxx, where x is any hex digit, and get kernel addresses. In future, this becomes a userspace address (when the virtual space is extended to 49 or more bits), and my code suddenly breaks on me.

It's the other way around. The sign extension is what makes it work the same regardless of the virtual address space size. (In fact, it's evident to me that that's the whole point behind AMD64's negative address weirdness/innovation). The 48 bit address 0xffff fffffff0 is the address -16, which is identical to the 52 bit address 0xfffff fffffff0 or the 64 bit address 0xffffffff fffffff0. When you read address -16, you will never be reading user space.

In your C program, you should be using a pointer data type, which is 64 bits encoded in two's complement pure binary, which means that even today, your pointer is encoded with the 64 bits 0xffffffff fffffff0. Nothing would change.

I don't see 128 bit file offsets ever happening. /dev/kmem isn't what the POSIX file interface is for; what it is for won't need 128 bit offsets enough to justify the cost of the change.

Files with negative offsets

Posted Jun 9, 2005 16:25 UTC (Thu) by landley (guest, #6789) [Link]

Actually, when I first read the headline I thought they were going to be talking about a really simple way of doing extended attributes.

Negative offsets would be a way to attach data to a file that's completely ignored by everybody who uses the API in a conventional manner, but really easy to access using the exact same API calls if you know it's there. The only new thing you'd need is another SEEK_END to get to the most negative offset. (An extra stat field to show the extended attribute size would be nice too, of course...)

Structuring this data would not be the operating system's problem...

Rob