Use gdb to get backtraces of hung/spinning sites

Sometimes code in a Plone site spins and eats CPU, or hangs, and you really need to get a backtrace...

Purpose

to get a backtrace of all the Python threads of your plone site to see who is hanging/spinning.

Overview

With the right tool, this is actually pretty easy.

Step by Step

  1. Change directories to the instance directory (i.e. where there is a bin/zopectl and a var/Z2.pid, etc.) (If you're running a load balanced setup, you'll need to do this in each of your zeoclient instance directories).
  2. Run my zopebt.sh shell script. It should show you where in the python code all of the threads of your Zope instance are running.

It uses gdb to get a non-intrusive python trace of all the running threads. This has been only tested on Linux, but has been tested with several Zope releases. It should work on other Python daemons, with suitable tweaks to find the right process-id.

More details

Thanks to jeremy over at python.org for the clues on how to get this to work nicely.