Where to keep Products

ZEO clients that use the same server for storage need to have the exact same products installed. It's easier to just maintain one directory of products and make sure that all clients can see it. Accordingly, I create a $BASE/products directory and create the Products directory within each client as a symlink to the directory[1]:

    $BASE/
        products/
            CMFCore/
            CMFPlone/
            # etc.
        client0/
            Products@ -> ../products
            # etc.
        client1/
            Products@ -> ../products
            # etc.
        # etc.

This directory is where the contents of the Plone-x.y.z tarball and all other add-on products should be placed. To create this structure, download the Plone Core source tarball and place it in /tmp. Then execute the following commands:

    cd /tmp
    tar zxf Plone-x.y.z.tar.gz
    cd $BASE
    mkdir products
    mv /tmp/Plone-x.y.z/* products
    rm -r /tmp/Plone-x.y.z*
    cd client0
    rm -r Products
    ln -s ../products Products
    # repeat the last three steps for all Zeo clients/Zope instances

How to organize and maintain Products

I use subversion to keep this directory of products under version control for the following reasons:

• You can start with a base set of products that are common to several customers or sites and create branches where customer or site-specific products are added or included as shown below.
• You can use subversion's externals feature to include other products that are maintained in separate subversion trees. This will be even more useful as the Collective is moved to subversion. [2]
• You can tag the directory each time you deploy the products to the production server so that you have a repeatable deployment. This makes trouble-shooting a production installation locally as easy as making a new Zope instance, checking out the products directory with the appropriate tag, and copying over the production Data.fs. It also simplifies cluster installations.
• If you need to make changes to products that you did not write but are unable or unwilling to contribute them back to the authors, you can keep them under version control and use svn merge to merge them into newer versions as they are released [3]

I generally check out the products directory into $BASE as a working copy on my development machines, but use svn export to get just the files without the svn metadata on the production and staging machines. This also helps to enforce the rule that no changes should be made in production/staging without being tested in a development environment, as you are unable to check in new files or changes to existing files from an exported copy. (Of course it doesn't stop someone from changing a file and not recording that fact in version control, but the thought of doing that just makes me sick, so let's move on.) On your development machines you will want to configure subversion to ignore .pyc files.

Permissions and ownership

It is important that the products directory and files and directories contained within it have the proper permissions and ownership. The user that Zope runs under needs to be able to write to the directories in order to create the byte-code (*.pyc) files when the products are first installed. However, the source files should not be writable by the Zope user. The solution is to have the products directory and everything under it owned by another user, to change the group ownership of the directories to the zope group, and to turn on the group write bit:

    # The first command is not necessary if you have checked out 
    # or otherwise populated the products directory yourself
    sudo chown -R $youruser:$yourgroup products
    find products -type f|xargs chmod g+r
    find products -type d|xargs chgrp zope
    find products -type d|xargs chmod g+rxw

You will have to run the last two chgrp and chmod commands as root or via sudo if you are not a member of the Zope user's group.

Test your installation

Once your products are in place and the permissions are set, test your installation by starting up (or restarting) Zope and adding a new Plone site from the ZMI. Check to make sure that the *.pyc files are written in the product directories. If everything looks good you can move on to configuring Apache as a front-end.

[1] You can also specify the Products directory explicitly in the zope.conf file for each client/instance.

[2] Make sure to use an URL for the external Product that represents a tag rather than a branch or the trunk, otherwise you may get updates that you haven't tested

[3] See the discussion of vendor branches in the svn book for more information