Kernel debugging via the net

One nice feature that was quietly slipped into the 2.6.0-test4-mm6 release is the kgdb-over-ethernet patch, by Robert Walsh and San Mehat. As described in the included documentation, kgdbeth makes it frighteningly easy to hook into a running Linux kernel over the network and prowl around in it. It's really just a matter of setting four boot parameters:

• gdbeth=number the device number of the ethernet interface to use for debugging. Usually zero for eth0.

• gdbeth_remoteip to set the IP address of the machine which is able to hook in with gdb.

• gdbeth_remotemac to set the remote system's MAC address.

• gdbeth_localmac to tell the kgdb stub what the local system's MAC address is.

As one would expect, the target system will only respond to debugger traffic coming from the system designated by the boot-time arguments. Once you've booted a kernel with the kgdbeth patch and the proper parameters, hooking in with gdb is simple. Here's a (slightly cleaned up) log from a quick session done here at LWN Labs:

gdb ./vmlinux
    (gdb startup stuff...)
(gdb) target remote udp:victim:6443
warning: The remote protocol may be unreliable over UDP.
warning: Some events may be lost, rendering further debugging impossible.
Remote debugging using udp:victim:6443
do_IRQ (regs=
      {ebx = -1069465600, ecx = -1054087008, edx = -216755, esi = 624384, 
       edi = -1072664576, ebp = 581632, eax = 0, xds = 123, xes = 123, 
       orig_eax = -251, eip = -1072652202, xcs = 96, eflags = 582, 
       esp = -1072652057, xss = 0}) at arch/i386/kernel/irq.c:514
warning: shared library handler failed to enable breakpoint
(gdb) print ioport_resource
$2 = {name = 0xc0362e75 "PCI IO", start = 0, end = 65535, flags = 256, 
      parent = 0x0, sibling = 0x0, child = 0xc03a2a80}
(gdb) print *ioport_resource->child
$3 = {name = 0xc035d94f "dma1", start = 0, end = 31, flags = 2147483648, 
      parent = 0xc03a40e0, sibling = 0xc03a2a9c, child = 0x0}
(gdb) c
Continuing.

For anybody who has wanted to be able to use gdb on a running kernel, but who has never gotten around to setting up the requisite serial lines and such, kgdbeth promises to make things easier than ever.

Matt Mackall has noticed that a number of patches - including Ingo Molnar's network console code and kgdbeth - each provide their own low-level ethernet functions. Code which hooks into the kernel at such a fundamental level needs to be able to send and receive packets without involving the entire networking subsystem. As a way of addressing this duplication of code and effort, Matt put together and posted a netpoll API. The patch came accompanied by new versions of netconsole and kgdbeth, both of which are somewhat cleaned up and significantly reduced in size. An added bonus is that netpoll supports almost all interfaces out there without the need for any driver changes. As of this writing, netpoll has not found its way into an -mm release, but that could change.

Of course, Linus's feelings on kernel debuggers are well known, so kgdbeth, while potentially useful for developers, is unlikely to find its way into the 2.6 mainline. So Andrew Morton will have to keep this one in -mm. At least, until Linus hands off the 2.6 kernel - to Andrew.

---

(Log in to post comments)