_FILE_OFFSET_BITS default

Posted Mar 15, 2014 21:58 UTC (Sat) by khim (subscriber, #9252)
In reply to: _FILE_OFFSET_BITS default by kleptog
Parent article: Glibc feature test macros

Perhaps I'm being insufficiently imaginative, but I can't think of a way that compiling an old program with _FILE_OFFSET_BITS=64 could lead to a buffer overrun. Do you have an example?

Is this a joke? Something like this will overflow just fine with maliciously crafted file of size 2G+2*maxreqsize:
  int mail_size = lseek(fd, 0, SEEK_END);
  if (mail_size > MAX_MAIL_SIZE) {
    syslog(LOG_MAIL, "Mail is too big!\n");
    return;
  }
  lseek(fd, 0, SEEK_SET);
  read(fd, mail_bug, mail_size);
  …

Is it sloppy programming? Yeah, sure. But even if so it's safe with _FILE_OFFSET_BITS=32 and unusafe with _FILE_OFFSET_BITS=64.

In my experience programs rarely do anything with fseek/ftell.

Indeed. Most of the code is probably out there will probably work fine with _FILE_OFFSET_BITS=64. Unfortunately you only need one piece of code which does something sloppy with longs/ints/size_ts to have an exploit.

fseek/fteel use long with or without _FILE_OFFSET_BITS=64 but without _FILE_OFFSET_BITS=64 the actual size always fit in int while as with _FILE_OFFSET_BITS=64 it's easy to get values which are negative when stored in int but positive when stored in size_t. Hilarity ensues.

_FILE_OFFSET_BITS default

Posted Mar 15, 2014 22:41 UTC (Sat) by kleptog (subscriber, #1183) [Link]

Nice example, thanks for that. I wouldn't write it that way, but I can imagine that kind of code being written.

In this example the reading in of 2G of data will probably fail but I can see now that it's not the quick win I think it was.

_FILE_OFFSET_BITS default

Posted Mar 15, 2014 23:00 UTC (Sat) by giraffedata (guest, #1954) [Link] (1 responses)

I was not as imaginative as khim; I was just thinking of data structures that have 4 bytes for the file offset.

Like struct stat. If you used struct stat in a binary interface, and then recompiled with _FILE_OFFSET_BITS=64, your program would be using a struct stat64 while your communication partner would still be using stat and the file size field, not to mention the whole structure, would overrun.

If you just had your own data structure - maybe a database index - with a 4 byte offset field and assigned an off_t value to it, you'd end up with an arithmetic overrun, which is almost as bad as a buffer overrun. (And that's even before considering the signedness issue khim raised).

I'm sure there are people saying it was never right to assume the width of off_t or struct stat, and therefore essentially nobody did it, but that's hindsight. There was a time when the concept of a file larger than 2 GB was too ridiculous, and the nature of such a system too unpredictable, to justify mucking up your code with an overflow test. One could easily have reasoned that if files ever did get that big, the designers of the new interface could deal with the 32 bit assumption then (which they did).

_FILE_OFFSET_BITS default

Posted Mar 17, 2014 12:12 UTC (Mon) by roblucid (guest, #48964) [Link]

Indeed, I remember when I got my hands on the first 1GB IBM SCSI disk, which seemed unbelievably large, given ones just 3 years older at 300MB were in boxes the size of midi-tower PCs. Files, reaching sizes of 2GiB, seemed to be just in perpetuity broken.. we knew the UNIX FS imposed limits. In fact who wants to open a 2GiB text logfile with vi(1)? Even today with current RAM sizes, back then 32MiB was very large and text files were "small" and loaded into memory. BSD introduced nvi(1) which handled large files better.

_FILE_OFFSET_BITS default

Posted Sep 8, 2014 22:22 UTC (Mon) by wahern (subscriber, #37304) [Link]

The compiler will complain loudly about the narrowing.

And 64-bit programs have been around since the early 1990s. The first full-blown Linux distribution supporting 64-bit Alpha came out in 1995.

Compared to Windows, 64-bit issues on Linux have been few and far between. Part of that is because back then FOSS developers tended to be battle-hardened portability experts who took these issues seriously. And at least with 64-bit, that cautionary attitude still lingers.

In today's FOSS world where portability is looked down upon as a needless chore, where there's x86 and ARM and nothing else, and where IBM actually ships a Power chip in little-endian mode because it's apparently too difficult for their Linux customers to not conflate representation and value, I can't imagine we'd be able to make such a transition so smoothly again.