Even further from the edge...

Posted Nov 27, 2006 15:36 UTC (Mon) by SiliconSlick (guest, #39955)
In reply to: Notes from the not-so-leading edge by nicku
Parent article: Notes from the leading edge

I learned and needed to type
rm /var/lib/rpm/__db*
on my (non-rawhide) Fedora boxes for well over one year now.

Indeed... we've experienced the same problem on CentOS(/RHEL) 4.3/4.4 so often that we have a script to quickly remedy the situation.

$ cat /usr/local/sbin/reset_rpm_db.sh

#!/bin/sh
# resetrpm.sh - reset RPM DB when RPM deadlock occurs - 27mar2006
# run as root/sudo

# Make sure nothing is going to start rpm/yum...

/etc/rc.d/init.d/cfexecd stop
/etc/rc.d/init.d/yum stop

# while current yum/rpm/cfagent sessions...
let MOREPIDS=1
while [ $MOREPIDS -ne 0 ]
do
  let MOREPIDS=0 # assume no more
  # kill them
  for PID in `ps axfu | grep -e yum -e rpm/rpm -e cfagent | grep -v grep | cut -b10-15 | sort`
  do
    let MOREPIDS=1 # there were more
    echo -n "Killing "
    ps axfu | grep $PID | grep -v grep
    kill -9 $PID
  done
done
echo ""
echo "Finished killing hung processes..."

# clean up DB

rm -f /var/lib/rpm/__db.00?

# do a quick query

rpm -q glibc

# restart daemons

/etc/rc.d/init.d/yum start
/etc/rc.d/init.d/cfexecd start
# EOF

(note, we make extensive use of yum/rpm in cfengine).

A lot of our problems _seem_ to stem from problems with NFS mounts, but even when NFS is working flawlessly (and all mounts are available/reliable), the RPM DB still manages to cause hangs on occasion.