This tutorial shows how to install Plone 3 into a buildout, and how to use that buildout when working on a software project that extends Plone. A buildout is a self-contained environment where you can manage the dependencies (including Zope and Plone and any third-party products or libraries you need) and custom code for your project. Even if you are not planning on writing any custom code, the buildout approach is an easy way to install Plone in a robust, well-tested manner.

Prior to Plone 3.0, most developers and users who did not use a GUI installer, would set up a Zope instance, drop in a few products into the Products folder, and be done with it. Unfortunately, this approach has a few problems:

• Plain old Zope instances are not very well equipped to deal with packages distributed as python eggs or using setuptools namespace packages. Many new packages in Plone 3 are made in this way, and more and more third party modules will be as well.
• Without access to the metadata that is held in eggs, developers may find it too time-consuming or confusing to factor their work into multiple packages that are more re-usable, preferring monolithic products that are impossible to re-use outside Zope.
  
• Without any further tools, it is cumbersome to repeat a setup across different environments.

As eggs become more important, developers should look to employ more appropriate tools for managing their code. zc.buildout, hereafter referred to only as "buildout" is one such tool. This tutorial shows how to use buildout for day-to-day development as well as deployment.

Terminology

Before we begin, let us get a few bits of terminology straight:

Zope installation

What you get by downloading and compiling Zope, or using one of the binary Zope installers.

Software home ($SOFTWARE_HOME environment variable)

Inside the Zope installation, the lib/python directory contains all the Python code that makes up the core of the Zope application server, known as the software home. The various Zope 3 packages are distributed here.

Zope instance

One Zope installation can support multiple instances. The mkzopeinstance.py script from the Zope installation is used to create a new instance. Each instance has a control script for starting and stopping the server, a zope.conf configuration file, a products directory, and a Data.fs file containing the ZODB. In this tutorial, we will let buildout create and configure a zope instance for us.

Instance home ($INSTANCEHOME environment variable)

The path to the current Zope instance.

Python path ($PYTHONPATH environment variable, aka sys.path)

The Python interpreter will look for modules in one or more folders, known as the Python path.  When Zope is running, this typically includes the global Python modules making up the standard library, the interpreter's site-packages directory, where third party "global" modules and eggs are installed, the Zope software home, and the lib/python directory inside the instance home. In this tutorial, we will see how buildout adds a number of specific eggs to the pythonpath at run time as well.

Python package

A general term describing a redistributable Python module. At the most basic level, a package is a directory with an __init__.py file and some Python code.

Zope product

A special kind of Python package used to extend Zope. In old versions of Zope, all products were directories inside the special Products directory of an zope instance, and would have a Python module name beginning with "Products". For example, the core of Plone is a product called CMFPlone, known in Python as Products.CMFPlone.

Python egg

A way to package and distribute Python packages. Eggs contains a setup.py file with metadata such as the author's name and email address and licensing information, as well as information about dependencies. setuptools, the Python library that powers the egg mechanism, is able to automatically find and download dependencies for eggs that you install. It is even possible for two different eggs to concurrently use different versions of the same dependency. Eggs also support a feature called entry points, a kind of generic plug-in mechanism. We will not cover extension points in great detail here, but you can read more about them (and other features of eggs) at the PEAK website.

The Cheese Shop (aka PyPI, the Python Package Index)

The Cheese Shop is an index that hosts thousands of Python packages. You can browse this if you are looking for a particular package. More importantly, setuptools (and buildout, and the easy_install script) can query this index to download and install eggs automatically.

 easy_install

A console script which can be used to search the Cheese Shop and install packages into the global Python environment. We will only use this to install a few global packages, since buildout manages eggs for us in a way that is local to each buildout project, avoiding global version ambiguities.

Namespace package

A feature of setuptools which makes it possible to distribute multiple, separate packages sharing a single top-level namespace. For example, the packages plone.theme and plone.portlets both share the top-level "plone" namespace, but they are distributed as separate eggs. When installed, each egg lives in its own directory (or possibly a compressed archive of that directory). Without namespace packages, we would have had to distribute one giant plone package, with a top-level plone directory containing all possible children, e.g. plone/theme and plone/portlets.

The magic Products namespace 

When Zope finds a "product", it will create an entry in Control_Panel/Products in the root of the ZMI, and run the initialize() method, found in the product's root __init__.py file, each time Zope starts up. Not every package used in a Plone context needs to be a product, but "productness" is required for:

• GenericSetup profiles
• Skin directories being installed as layers in the portal_skins tool (but not for Zope 3-style browser views)

The easiest way to create a product is to use Paster/ZopeSkel to create an egg-ready package in the Products.* namespace using the basic_namespace template:

$ paster create -t basic_namespace Products.myproduct
Selected and implied templates:
  ZopeSkel#basic_namespace  A project with a namespace package

Variables:
  egg:      Products.myproduct
  package:  productsmyproduct
  project:  Products.myproduct
Enter namespace_package (Namespace package (like plone)) ['plone']: Products
Enter package (The package contained namespace package (like example)) ['example']: myproduct
... accept defaults to end

If you're using buildout, create your package in the src directory, and add references to it in the develop and instance/eggs sections of buildout.cfg:

develop =
    src/Products.myproduct
...
[instance]
...
eggs =
    ${buildout:eggs}
    ${plone:eggs}
    Products.myproduct

Run bin/buildout and you'll be set up to develop your egg-ready product in the src directory. Turn it into a distribution egg when complete.

It is possible to use packages (including egg-distributed ones) outside the Products namespace/directory as Zope 2 products. Many developers prefer this approach, feeling it unnatural to keep everything in a single, "flat" namespace.

Extra steps are required for this. Prior to Zope 2.10.4, this is also required for products in the Products namespace

We must add a line like the following to the package's configure.zcml:

<configure 
    xmlns="http://namespaces.zope.org/zope"
    xmlns:five="http://namespaces.zope.org/five">

    <five:registerPackage package="." initialize=".initialize" />

</configure>

• Another package's configure.zcml file.
• Zope's site.zcml, the root of all ZCML files, which is found in the etc directory in the instance home.
• A ZCML slug, a one-liner created in the zope instance's etc/package-includes directory, with a name like my.package-configure.zcml.

In all cases, the syntax is the same:

<include package="my.package" file="configure.zcml" />

If you have meta.zcml or overrides.zcml files, you can add <include /> directives for these as well. If you are using slugs, it must be named accordingly, e.g. my.package-meta.zcml or my.package-overrides.zcml. A slug can not contain more than one line.

Later in this tutorial, we will show how buildout can manage slugs for us automatically.

Before we can create a buildout to manage Zope and Plone, there are a few prerequisites to take care of.

First, you will need an appropriate Python interpreter, if you do not have one already:

• Install Python 2.4 for your platform, and add it to your system PATH. It is easiest if Python 2.4 is what you get when you type python -V on a command line.
• If you installed Python using an operating system package (e.g. an RPM), make sure you get the development package (e.g. python-devel) as well. This includes Python header files that we will use later to compile Zope. If you installed from source, or used the Python Windows installer, you should already have these.
  
• Install PIL, the Python Imaging Library into this Python interpreter.

Then, download ez_setup.py and run it with:

$ python ez_setup.py

This will download and install setuptools and the easy_install script. Watch the console output to understand where easy_install is installed. If this is not in your system PATH, you should add this directory to the path as well.

Finally, use easy_install to get ZopeSkel, a collection of skeleton templates for Zope and Plone development:

$ easy_install -U ZopeSkel

This will get Paste Script and various other dependencies. If you added the Python console scripts directory (where easy_install was placed) to your system path, you should now be able to run the paster command. You can test it with:

$ paster create --list-templates
Available templates:
  basic_namespace:          A project with a namespace package
  basic_package:            A basic setuptools-enabled package
  basic_zope:               A Zope project
  nested_namespace:         A project with two nested namespaces.
  plone:                    A Plone project
  plone2.5_theme:           A Theme for Plone 2.5
  plone2_theme:             A Theme for Plone 2.1 & Plone 2.5
  plone3_buildout:          A buildout for Plone 3 projects
  plone3_theme:             A Theme for Plone 3.0
  plone_app:                A Plone App project

Your output may differ slightly, but make sure you have the plone3_buildout and plone templates at least.

Additional installation steps for Windows

If you are using Windows, there are a few more things you need to do.

First, get and install the Python Win32 extensions for Python 2.4.

If you intend to compile Zope yourself, rather than using a binary installer, or if you ever need to compile an egg with C extensions, you will need the mingw32 compiler. Make sure you choose the "base" and "make" modules at a minimum when the installer asks. By default, this installs into C:\MingW32. Inside the installation directory, there will be a bin directory, e.g. C:\MingW32\bin. Add this to your system PATH.

Finally, you need to configure Python's distutils package to use the mingw32 compiler. Create a file called distutils.cfg in the directory C:\Python24\Lib\distutils (presuming Python was installed in C:\Python24, as is the default). Edit this with Notepad, and add the following:

[build]
compiler=mingw32

We are now ready to create a new buildout. The "buildout" is a directory containing all the parts that make up a project, including a Zope instance, the Plone sources, custom configuration options, and your our project's source code. Create one like this:

$ paster create -t plone3_buildout myproject

This will ask a series of questions. If you want to use an existing installation of Zope rather than have buildout download and compile one for you, specify an absolute path as the zope2_install. Similarly, if you do not want buildout to download the core Plone products, you can point it to an existing directory containing all the products (it will still download Plone 3's eggs, but as we will see later, it is possible to share an eggs directory among multiple buildouts). You will need to enter a Zope administrator username and password, and you may want to turn debug mode and verbose security on during development.

Now, enter the newly created myproject directory, and run the buildout bootstrap script:

$ cd myproject
$ python bootstrap.py

This will create a number of directories and scripts and dowload the latest version of the zc.buildout egg. This step should be needed only once.

To get started straight away, run:

$ ./bin/buildout

This reads the generated buildout.cfg file and executes its various "parts", setting up Zope, creating a Zope instance, downloading and installing Plone. We will explain this file in more detail shortly.

You will need to run ./bin/buildout again each time you change buildout.cfg. If you do not want buildout to go online and look for updated versions of eggs or download other archives, you can run it in non-updating, offline mode, with;

$ ./bin/buildout -No

To start Zope, run:

$ ./bin/instance fg

The instance script is analogous to zopectl as found in a standard Zope instance. You can use ./bin/instance start to run Zope in daemon mode. It can also be used to run tests:

$ ./bin/instance test -s plone.portlets

Directories in the buildout

Before we dive into buildout.cfg, let us take a quick look at the directories that buildout has created for us:

bin/

Contains various executables, including the buildout command, and the instance Zope control script.

eggs/

Contains eggs that buildout has downloaded. These will be explicitly activated by the control scripts in the bin/ directory.

downloads/

Contains non-egg downloads, such as the Zope source code archive.

var/

Contains the log files (in var/log/) and the file storage ZODB data (in var/filestorage/Data.fs). Buildout will never overwrite these.

src/

Initially empty. You can place your own development eggs here and reference them in buildout.cfg. More on that later.

products/

This is analogous to a Zope instance's Products/ directory (note the difference in capitalisation). If you are developing any old-style Zope 2 products, place them here. We will see how buildout can automatically download and manage archives of products, but if you want to extract a product dependency manually, or check one out from Subversion, this is the place to do so.

parts/

Contains code and data managed by buildout. In our case, it will include the local Zope installation, a buildout-managed Zope instance, and Plone's source code. In general, you should not modify anything in this directory, as buildout may overwrite your changes.

You can check in a buildout directory to a source code repository to share it among developers. In this case, you should ignore the directories bin/, eggs/, downloads/, var/, and parts/. Each developer can run bootstrap.py to get these back, and will normally need local copies anyway. All your configuration should be in the buildout.cfg file, and all custom code in src/ or products/.

buildout.cfg is the most important file in your new buildout environment. Here is how it looks:

[buildout]
parts =
    plone
    zope2
    productdistros
    instance
    zopepy

# Add additional egg download sources here. dist.plone.org contains archives
# of Plone packages.
find-links =
    http://dist.plone.org
    http://download.zope.org/ppix/
    http://download.zope.org/distribution/
    http://effbot.org/downloads

# Add additional eggs here
# elementtree is required by Plone
eggs =
    elementtree
    
# Reference any eggs you are developing here, one per line
# e.g.: develop = src/my.package
develop =

[plone]
recipe = plone.recipe.plone

[zope2]
recipe = plone.recipe.zope2install
url = ${plone:zope2-url}

# Use this section to download additional old-style products.
# List any number of URLs for product tarballs under URLs (separate
# with whitespace, or break over several lines, with subsequent lines
# indented). If any archives contain several products inside a top-level
# directory, list the archive file name (i.e. the last part of the URL, 
# normally with a .tar.gz suffix or similar) under 'nested-packages'.
# If any archives extract to a product directory with a version suffix, list
# the archive name under 'version-suffix-packages'.
[productdistros]
recipe = plone.recipe.distros
urls =
nested-packages =
version-suffix-packages = 

[instance]
recipe = plone.recipe.zope2instance
zope2-location = ${zope2:location}
user = admin:admin
http-address = 8080
debug-mode = on
verbose-security = on

# If you want Zope to know about any additional eggs, list them here.
# This should include any development eggs you listed in develop-eggs above,
# e.g. eggs = ${buildout:eggs} ${plone:eggs} my.package
eggs =
    ${buildout:eggs}
    ${plone:eggs}

# If you want to register ZCML slugs for any packages, list them here.
# e.g. zcml = my.package my.other.package
zcml = 

products =
    ${buildout:directory}/products
    ${productdistros:location}
    ${plone:products}

[zopepy]
recipe = zc.recipe.egg
eggs = ${instance:eggs}
interpreter = zopepy
extra-paths = ${zope2:location}/lib/python
scripts = zopepy

Let us walk through this file step-by-step:

The main [buildout] section

The [buildout] section is the starting point for the file. It lists a number of "parts", which are configured in separate sections later in the file. Each part has an associated recipe, which is the name of an egg that knows how to perform a particular task, e.g. build Zope or create a Zope instance. A recipe typically takes a few configuration options.

Our global settings are as follows:

[buildout]
parts =
    plone
    zope2
    productdistros
    instance
    zopepy

find-links =
    http://dist.plone.org
    http://download.zope.org/ppix/
    http://download.zope.org/distribution/
    http://effbot.org/downloads

eggs =
    elementtree
    
develop =

This specifies that the parts plone, zope2, productdistros, instance and zopepy will be run, in that order. Then, we tell buildout that it can search one of a number of URLs when it is looking for eggs to download. In addition, it will always search the Cheese Shop.

Next, we can list any eggs that buildout should download and install for us. This may include version specifications. For example, if you want sqlalchemy 0.3,  but not 0.4, you could list;

eggs = 
    elementtree
    sqlalchemy>=0.3,<0.4dev

Finally, we can list development eggs, by specifying a directory where the egg is extracted in source format. For example:

eggs =
    elementtree
    my.package

develop = 
    src/my.package

This presumes that there is an egg called my.package in the src/ directory. We will learn how to create such eggs a little later in this tutorial. Notice how we must also list my.package as an actual egg dependency: development eggs are not automatically added to the "working set" of eggs that are installed for Zope.

The [plone] section

This is very simple - it just uses plone.recipe.plone to download Plone's products and eggs.

[plone]
recipe = plone.recipe.plone

It will use the latest release available. Version numbers for plone.recipe.plone correspond to version numbers for Plone itself. Therefore, to make sure you always get a 3.0.x release, but not a 3.1, you can do:

[plone]
recipe = plone.recipe.plone>=3.0,<3.1dev

When the recipe is run, Plone's products will be installed in parts/plone. The eggs are made available via buildout variable ${plone:eggs}, which we will reference in the [instance] section later, and the URL of a "known good" version of Zope is available in the variable ${plone:zope2-url}.

The [zope2] section

This part builds Zope 2, using plone.recipe.zope2install. If you specified an existing Zope installation, you will not have this part. Otherwise, it looks like this:

[zope2]
recipe = plone.recipe.zope2install
url = ${plone:zope2-url}

Here, we reference the download location for Zope as emitted by the [plone] part. This ensures that we always get the recommended version of Zope. You could specify a download URL manually instead, if you wanted to use a different version of Zope.

When the recipe is run, Zope 2 is installed in parts/zope2. The Zope software home becomes parts/zope2/lib/python.

The [productdistros] section

This uses the plone.recipe.distros recipe, which is able to download distributions (archives) of Zope 2 style products and make them available to Zope. It is empty to begin with:

[productdistros]
recipe = plone.recipe.distros
urls =
nested-packages =
version-suffix-packages =

However, you can list any number of downloads. The recipe is also able to deal with archives that contain a single top-level directory that contains a bundle of actual product directories (nested-packages), or packages that have a version number in the directory name and thus need to be renamed to get the actual product directory (version-suffix-packages).

For example, here is how you may download CacheFu 1.1:

[productdistros]
recipe = plone.recipe.distros
urls =
    http://plone.org/products/cachefu/releases/1.1/CacheFu-1.1.tgz
nested-packages =
    CacheFu-1.1.tgz
version-suffix-packages = 

You can specify multiple downloads on separate lines. When the recipe is run, the product directories for downloaded products are found in parts/productdistros.

The [instance] section

The instance section pulls it all together: It configures a Zope instance using the plone.recipe.zope2instance script. Here is how it looks:

[instance]
recipe = plone.recipe.zope2instance
zope2-location = ${zope2:location}
user = admin:admin
http-address = 8080
debug-mode = on
verbose-security = on
eggs =
    ${buildout:eggs}
    ${plone:eggs}
zcml = 
products =
    ${buildout:directory}/products
    ${productdistros:location}
    ${plone:products}

Here, we reference the Zope 2 installation from the [zope2] part - if you specified a location yourself when creating the buildout, you would see that one here. Then, we specify the initial admin user and password, and the port that Zope will be bound to. We also turn on debug mode and verbose security. These options are used to generate an appropraite zope.conf file for this instance. See the recipe page in the Cheese Shop for more details on the options available.

Next, we specify which eggs that will be made available to Zope. This references the "global" eggs from the [buildout] section, as well as the eggs specified by Plone. You could add additional eggs here, though it is generally easier to specify these at the top of the file, so that they get included in the ${buildout:eggs} working set.

As explained previously, Zope 3 configure.zcml files are not loaded automatically for eggs or packages not the Products namespace. To load ZCML files for a regular package, we can make buildout create a ZCML slug by listing the package under the zcml option:

zcml =
    my.package
    my.package-overrides

This assumes that my.package was previously referenced in the buildout. This would load both the main configure.zcml and the overrides.zcml file from this package.

Finally, we list the various directories that contain Zope 2 style products - akin to the Products/ directory in a traditional instance. Notice how the products/ directory in the main buildout directory comes first, followed by the products downloaded with the [productdistros] part, followed by the products downloaded by the [plone] part. This means that even if Plone ships with a product, you could override it (e.g. with a newer product) by putting a product with the same name in the top-level products/ directory.

When the recipe is run, the Zope instance home will be parts/instance, and a control script is created in ./bin/instance.

The [zopepy] section

This final section creates a Python interpreter that has all the eggs and packages (but not Zope 2 style products) that Zope would have during startup. This can be useful for testing purposes.

[zopepy]
recipe = zc.recipe.egg
eggs = ${instance:eggs}
interpreter = zopepy
extra-paths = ${zope2:location}/lib/python
scripts = zopepy

Here, we copy the eggs from the [instance] section, and include in the pythonpath the Zope instance home.

When the recipe is run, the script will be created in ./bin/zopepy.

To set "global" options affecting all buildouts, create a directory .buildout (note leading dot) in your home directory, and add a file there called default.cfg. Any option set here will be applied to the corresponding section in any buildout.cfg that you run, unless it is overridden by a more specific option in the buildout.cfg file itself.

The most common options are:

executable

Specify a python interpreter other than the system default. This is useful if you have Python 2.5 installed, say, but you want your buildouts to use another installation of Python 2.4.

eggs-directory

Specify a directory where eggs will be downloaded. This allows multiple buildouts to share the same eggs, saving disk space and download time. Note that only those eggs explicitly required by a particular buildout will be activated. The eggs directory may contain many more eggs (or many different versions of the same package) than what is used at any one time.

download-directory

Specify a shared directory for downloaded archives. Again, this can save disk space and download time.

Here is an example ~/.buildout/default.cfg setting all three:

[buildout]
executable = /opt/python24/bin/python
eggs-directory = /home/username/.buildout/eggs
download-cache = /home/username/.buildout/downloads

This assumes Python 2.4 is installed in /opt/python2.4. For the last two options to work, you would need to create the directories eggs and downloads inside the ~/.buildout directory.

How to install a new third-party products will depend on whether it is packaged as an egg, or a traditional Zope 2 product.

Installing a traditional Zope 2 product

The easiest way to try out a traditional Zope 2 product is to extract it into the products/ folder inside the buildout. If you see documentation referring to the Products/ folder in a Zope instance, this is the same thing.

However, this approach makes it harder to redistribute your project and share it with other developers. It is often more predictable to let buildout download and install the package for you. You can do this with the [productdistros] section of buildout.cfg. For example, here is how you might install DocFinderTab and CacheFu for your project:

[productdistros]
recipe = plone.recipe.distros
urls =
    http://www.zope.org/Members/shh/DocFinderTab/1.0.2/DocFinderTab-1.0.2.tar.gz
    http://plone.org/products/cachefu/releases/1.1/CacheFu-1.1.tgz
nested-packages =
    CacheFu-1.1.tgz
version-suffix-packages =

Note that CacheFu is distributed as a single directory containing a number of products in sub-directories, so we list it under nested-packages.

As always, if you change buildout.cfg, you must re-run buildout:

$ ./bin/buildout

Installing an egg

So long as an egg has a release in the Cheese Shop or elsewhere, buildout can download and install it, including any explicitly specified dependencies. Simply list the egg, and optionally a version (otherwise, you get the latest available), in the eggs option.

[buildout]
...
eggs = 
    elementtree
    borg.project>=1.0b1,<2.0dev

If you want buildout to search an index other than the Cheese Shop's, you can add a URL to find-links that contains download links for the eggs. In fact, we have already seen an example of this: elementtree is found at http://effbot.org/downloads, not in the Cheese Shop directly. Thus, we have:

[buildout]
...

find-links =
    http://dist.plone.org
    http://download.zope.org/ppix/
    http://download.zope.org/distribution/
    http://effbot.org/downloads

eggs =
    elementtree

We have also listed some of the download locations for Zope and Plone eggs.

Again - re-run buildout for the changes to take effect:

$ ./bin/buildout

Development eggs

If there is not a release for your egg, or you want to track an egg in Subversion, check it out to the src/ directory. Make sure you get the full egg, including the top-level setup.py file. For example, to get the plone.portlets trunk development, egg do:

$ cd src
$ svn co https://svn.plone.org/svn/plone/plone.portlets/trunk plone.portlets

Then, add the following to buildout.cfg:

[buildout]
...
eggs =
    ...
    plone.portlets

develop =
    src/plone.portlets

Note that:

• The develop option contains a relative path to where the source egg is installed. Buildout will expect to find a suitable setup.py in this directory.
  
• Development eggs always take precedence over regular eggs.
• You still need to list the egg name  in the eggs option for it to be installed.
• If you are overriding an egg that ships with Plone, you may need to list it in the eggs section of the [plone] part instead:

[buildout]
...
develop =
    src/plone.portlets

...

[plone]
recipe = plone.recipe.plone
eggs = 
    plone.portlets

This is because plone.recipe.plone is very expilcit about which versions of its various eggs to use, to ensure Plone keeps running as it was released.

Buildout recipes (such as plone.recipe.plone) are distributed as eggs. You can use a development egg of a recipe by listing it under the develop option. There is no need to explicitly list it under the eggs option, since it is referenced by the recipe option of the relevant part.

Managing ZCML files

It is important to realize that Zope will not load configure.zcml files automatically for packages that are not in the Products.* namespace. Instead, you must explicitly reference the package. Buildout can create such a reference (known as a ZCML slug) with the zcml option under the [instance] part. Here is how to ensure that borg.project is available to Zope:

[buildout]
...
eggs =
    elementtree
    borg.project

...

[instance]
...
zcml = 
    borg.project

Should you need to load an overrides.zcml or a meta.zcml, you can use a syntax like:

zcml =
    some.package
    some.package-overrides
    some.package-meta

Policy products

Many developers prefer to create a single "policy product" (also known as a "deployment product") that orchestrates various dependencies. If you have such a product, you may want to include various dependencies directly from the policy product's configure.zcml file, with lines such as:

<configure xmlns="http://namespace.zope.org/zope">

    <include package="borg.project" />

</configure>

In this case, you may still need one slug (using the zcml option as above) for the policy product.

Adding a new custom package is not much different from installing a third-party one.

Creating a traditional Zope 2 product

To create a traditional Zope 2 product, put it in the top-level products/ directory and re-start Zope. Nothing more should be required. As explained previously, products placed here will be found automatically at start-up, and their configure.zcml files will be executed automatically.

Creating an egg

Of course, if you are using products, you cannot benefit from the additional features of eggs, including automatic dependency
management, distribution via the Cheese Shop and nested namespaces.

The easiest way to create a new egg is to use the paster command, which we already used to create the buildout. To create a new basic package, with a top-level namespace (e.g. your company name) and a specific name, go to the src/ directory and run:

$ cd src
$ paster create -t plone myorg.mypackage

You will be asked a series of questions. Make sure that the namespace package and package name correspond to the name of the egg. In this case, the namespace package is myorg and the package name is mypackage. In general, answer False to the question on whether your package if "zip safe". Enter other metadata as requested.

You will now have:

• A setup.py which contains the metadata you entered
• A package in myorg.mypackage/myorg/mypackage. Your source code goes here.
• A skeleton configure.zcml, tests.py and a few other useful starting points.
• Some generic documentation in myorg.mypackage/docs.

Of course, you must also add this package to the buildout. In buildout.cfg, you might have:

[buildout]
...
eggs =
    ...
    myorg.mypackage

develop =
    src/myorg.mypackage

Unless you plan to include this package from another one, you probably also need a ZCML slug:

[instance]
...
zcml =
    myorg.mypackage

Do not forget to re-run buildout after making the change:

$ ./bin/buildout

Specifying dependencies

If your new package has explicit dependencies, you can list them in setup.py. That way, buildout will be able to download and install these as well. Dependencies are listed in the install_requires argument to the setup() method, By default, setuptools is listed here, since we need this to support namespace packages. To add sqlalchemy 0.3 (but not 0.4), and the MySQL-Python driver, you could amend this to read:

install_requires=[
          'setuptools',
          'sqlalchemy>=0.3,<0.4dev',
          'MySQL-Python',
      ],

Uploading your egg to the Cheese Shop

If you want to share your packge with the rest of the Python community and make it easy to install using tools like buildout and easy_install, you can upload the package to the Cheese Shop.

Before doing so, you should:

• Commit your latest changes and tag the release in Subversion, if applicable.
  
• Remove (temporarily) the setup.cfg file: this makes your package a development release.
  
• Make sure the version number in setup.py is correct. This should use common conventions such as "1.0b2" for the second beta of version 1.0, or "2.1.3rc1" for the first release candidate of version 2.1.3.
• If you are using Mac OS X, run export COPY_EXTENDED_ATTRIBUTES_DISABLE=true on the shell first - otherwise, the egg will contain Mac OS X resource forks which cause problems if your egg is used on Windows.

When you are ready, run the following command from your package's directory (e.g. src/myorg.mypackage):

$ python setup.py egg_info -RDb "" sdist bdist_egg register upload

This will ask you to create a Cheese Shop account if you do not have one already. You can run this command as often as you'd like to release a new version (probably with a new version number).

Finally, let's take a look at a more advanced configuration, better suited for deployment. Save this file as deployment.cfg, at the root of the buildout next to the main buildout.cfg file:

[buildout]
extends =
    buildout.cfg

parts =
    plone
    zope2
    productdistros
    deploymentproducts
    zeoserver
    primary
    secondary
    varnish-build
    varnish-instance

[deploymentproducts]
recipe = plone.recipe.distros
urls =
    http://plone.org/products/cachefu/releases/1.1/CacheFu-1.1.tgz
nested-packages =
    CacheFu-1.1.tgz
version-suffix-packages = 

[zeoserver]
recipe = plone.recipe.zope2zeoserver
zope2-location = ${zope2:location}
zeo-address = 8100

[primary]
recipe = plone.recipe.zope2instance
zope2-location = ${zope2:location}
zeo-client = true
zeo-address = 8100
zodb-cache-size = 5000
zeo-client-cache-size = 300MB
user = admin:admin
http-address = 8080
debug-mode = off
verbose-security = off
eggs =
    ${plone:eggs}
    ${buildout:eggs}
zcml = 
    
products =
    ${buildout:directory}/products
    ${productdistros:location}
    ${deploymentproducts:location}
    ${plone:products}

[secondary]
recipe = plone.recipe.zope2instance
zope2-location = ${zope2:location}
zeo-client = true
zeo-address = 8100
zodb-cache-size = 5000
zeo-client-cache-size = 300MB
user = ${primary:user}
http-address = 8081
debug-mode = on
verbose-security = on
eggs = ${primary:eggs}
zcml = ${primary:zcml}
products = ${primary:products}
zope-conf-additional =
    zserver-threads 1
    
[varnish-build]
recipe = plone.recipe.varnish:build
url = http://puzzle.dl.sourceforge.net/sourceforge/varnish/varnish-1.0.4.tar.gz

[varnish-instance]
recipe = plone.recipe.varnish:instance
bind = 127.0.0.1:8082
backends = 127.0.0.1:8080
cache-size = 1G

Here, we are:

• Referencing the main buildout.cfg file, extending and overriding it with configuration more appropriate for deployment.
  
• Setting up a ZEO server with two client instances, primary and secondary (see plone.recipe.zope2zeoserver and plone.recipe.zope2instance for more details)
• Compiling the Varnish cache server (see plone.recipe.varnish for more details).

By combining buildout configuration files like this, you can create tailor-made configurations for different deployment scenarios. To learn more about the advanced features of buildout, see its documentation.

To build this environment, you must explicitly specify a configuration file:

$ ./bin/buildout -c deployment.cfg

To start Zope and Plone, you will need to start the ZEO server, the two clients (or at least, the primary one), and the Varnish server:

$ ./bin/zeoserver start
$ ./bin/primary start
$ ./bin/secondary start
$ ./bin/varnish-instance

The recipes will also create scripts to back up the ZODB (in ./bin/repozo) and to pack the database (in ./bin/zeopack).

Further options

zc.buildout is a very flexible system. It is relatively easy to create new recipes, and you can combine existing recipes in powerful ways. Search the Cheese Shop for "buildout" to find more recipes, or take a look at the source code for some of Plone's own recipes to understand how recipes are created.