My guess is that the X driver is getting too much mmio space mapped, and accidentally writing into whatever's next; the kernel panics regardless, but it's only particularly notable if the ethernet driver happens to have just done the special ritual to start writing to the eeprom (which it's doing to reprogram it correctly), and then the graphics driver happens to hit it.