Python crashes on Mac OS X (lxml related)

If you are on Mac OS X, and you have a package that uses the lxml library, you may fall victim to an incompatibility between recent versions of lxml and the versions of libxml2 and libxslt that come installed with Mac OS X. This causes a segmentation fault when certain lxml commands are run, typically manifesting a Python crash (and a dialog box to that effect) when Plone starts up.

This problem should be fixed for new installations, because the lxml team now build binary eggs that should work on OS X. However, if you are unlucky, you may still find yourself with an egg built on your own machine that gives difficult-to-debug crashes.

There are two ways to fix this.

Use a binary egg

This is the preferred solution. lxml comes with binary eggs for Windows and OS X that bundle the correct versions of libxml2 and libxslt. You may have missed these eggs if you installed lxml manually, or if you have an old version installed.

To make sure you have the latest build, you should remove any cached version of the egg or the downloaded source tarball from your cache and re-run buildout.

1. Find out which version of lxml is being used. You can find this out by running buildout with the -v option, or by looking at one of the generated scripts. For example, if your instance script is called bin/instance, look for a line like '/Users/optilude/.buildout/eggs/lxml-2.2.1-py2.4-macosx-10.5-i386.egg'. This tells you that buildout has used version 2.2.1 and built or downloaded an egg that's now at the given path.
2. Verify that there indeed is a binary egg for this version, by checking PyPI. You may want to use a [versions] block in buildout.cfg to pin to a version that works.
3. Delete the egg directory/file. If you have a shared eggs cache set up in~/.buildout/default.cfg, it will be there. Otherwise, it will be in the eggs folder under your buildout.
4. Remove the source download from your download cache, if it's there. You may have a shared download cache set up in ~/.buildout/default.cfg, or you may have a download directory in your buildout. In either case, look for a directory there called dist and remove any files related to lxml, in particular any .tgz file.
5. Re-run buildout. Use the -v option to check for output. If you see a line related to "Building lxml", you may have missed something in steps 3 to 4.

Build your own static version

This approach works by instructing buildout to build its own version of lxml.

First, add an lxml part to buildout.cfg as the first part.

[buildout]
parts =
    lxml
    ...

It's important that this goes first, to avoid buildout downloading and using an lxml source distribution prematurely.

Then add the following part:

[lxml]
recipe = z3c.recipe.staticlxml
egg = lxml
force = false

This will make your buildout take a bit longer, as it will download and compile libxml2 and libxslt. However, it should then build lxml against these versions.