Killing processes that don't want to die

Posted May 29, 2018 4:35 UTC (Tue) by mrons (subscriber, #1751)
Parent article: Killing processes that don't want to die

These days I would use systemd cgroups to kill these rouge fork bombs.

Back in the day, on a computer science teaching system, it would be sport for the students to try to make fork() bombs that were hard to kill (for the sys admin (me)).

The example used in this article, where the PID is rapidly changing, was one such technique used by students. We used to call such fork bombs "comets".

One amusing way I found to kill a comet was to use the limit of max users processes (ulimit -u). I would create a standard fork bomb, run as the rouge user, and exhaust the max number of processes the user could run. The comet would then no longer be able to fork(). Then I could killall the user processes to recover.

So using a fork bomb to kill a fork bomb.

Killing processes that don't want to die

Posted May 31, 2018 13:26 UTC (Thu) by fanf (guest, #124752) [Link]

I call these kinds of processes "fork rabbits". I got into a sticky situation with one of them once...

I was hacking on a production server (I didn't have an adequate test environment). I had a daemon that was supposed to re-open its log files etc. when it got a signal. In order to cope with slow cleanup of the old file descriptors, it would fork and reopen the new file descriptors in the child, allowing the parent to clean up at leisure.

I refactored the signal handling code, and screwed it up.

When the daemon received a signal, it became a rabbit.

It was running as root on a production server.

I couldn't use `kill -KILL -1` and I couldn't reboot the machine. (I might have been able to kill by pgid, but I didn't think of that at the time.)

Fortunately this machine did not have randomized pids, so I could anticipate the future pid of the rabbit a few seconds in advance and run a `while :; do kill $pid; done` loop. Of course the rabbit raced right through the trap.

I rewrote the killer in C, and tried again, but the rabbit kept winning the race. So I tried running multiple concurrent killers targeting several adjacent pids. Eventually this worked!

(The side effect would have been a number of failed FTP connections...)