Classic buffer overflow bug; another YES for segmentation

Today I helped to debug mysterious crashes in a code roughly like the following:

void f() {
  char buf[SIZE], *ptr = buf, logmsg[128];

  while(some_condition) {
    /* do something with ptr */
    sprintf(logmsg, "something", ...)
  }
}

This code crashed. This was within a multithreaded program and GDB has shown very dubious stack trace (some function calls with many unrecognized addresses in between), and it has shown very strange address in ptr. An illuminating piece of information, provided by the author of the code, is that the loop runs fine for the first time.

Since all uses of ptr in that loop are pretty innocent and couldn't set it to some really wild value that GDB has shown, my suspicion fell to the sprintf. The "something" within the sprintf was a long text which itself (without interpolated values) overflowed the logmsg buffer. Since stack grows downwards on the AMD64 architecture, the overflown buffer overwrote the ptr pointer.

What made it relatively hard to debug are two things;

1. GDB showing dubious stack trace (this is understandable after the fact, since the whole stack was thrashed), and


2. error manifesting itself (program crash) at a different point in the program (next time the ptr pointer is used) rather than at the point where it was made (sprintf overwriting the buffer).
   

Such buffer overflows could be prevented by using the segmentation mechanism present on the IA32 architecture. All memory accesses to segments that are less than 1MB in size are limit-checked in the hardware, with byte granularity. However, nobody uses these feautres for several reasons:

• They are not portable (i.e. tied only to IA32 architecture),


• No widely-used 32-bit compiler supports FAR pointers. MSVC ignores the far keyword, and gcc never did support it. AFAIK, the only compiler that honours it is the Watcom compiler.


• Loading segment registers is relatively costly operation, and, what's worse, Win32 reserves the FS register for itself and other segments must not be changed. So if you have more than one buffer to access, you need to reload the segment register all the time.

The final blow to segmentation came when AMD decided to effectively kill it. The descriptors are still present for system management purposes, but segment base and limits are forced to flat 64-bit address space in hardware.

IMO, it's a shame to see a good technology getting killed because of lack of imagination on the side of programmers. And some "holy grail" of portability. If it were used properly, and OS'es designed the way they should have been, the IA32 architecture could have been almost 100% immune to buffer overflows which are to blaim for many insecurities in applications. And without all of the "NX", "virus protection", etc. hype.