Creating a buildout for your project

How to create a new buildout for a project, adding Plone and other third party products as dependencies

We are now ready to create a new buildout. The "buildout" is a directory containing all the parts that make up a project, including a Zope instance, the Plone sources, custom configuration options, and your our project's source code.

As of Plone 3.2, all of the Plone installers are buildout based.  You can get the latest installer and run it to have a working buildout without having to follow these steps. However, these steps are still valid if you want to create the buildout manually with ZopeSkel.

Create one like this:

$ paster create -t plone3_buildout myproject

This will ask a series of questions. If you want to use an existing installation of Zope rather than have buildout download and compile one for you, specify an absolute path as the zope2_install. Similarly, if you do not want buildout to download the core Plone products, you can point it to an existing directory containing all the products (it will still download Plone 3's eggs, but as we will see later, it is possible to share an eggs directory among multiple buildouts). You will need to enter a Zope administrator username and password, and you may want to turn debug mode and verbose security on during development.

Now, enter the newly created myproject directory, and run the buildout bootstrap script.  NOTE: Python 2.4 is currently required to Plone 3.x:

$ cd myproject
$ python2.4 bootstrap.py

This will create a number of directories and scripts and dowload the latest version of the zc.buildout egg. This step should be needed only once.

To get started straight away, run:

$ ./bin/buildout

This reads the generated buildout.cfg file and executes its various "parts", setting up Zope, creating a Zope instance, downloading and installing Plone. We will explain this file in more detail shortly.

You will need to run ./bin/buildout again each time you change buildout.cfg. If you do not want buildout to go online and look for updated versions of eggs or download other archives, you can run it in non-updating, offline mode, with;

$ ./bin/buildout -No

To start Zope, run:

$ ./bin/instance fg

The instance script is analogous to zopectl as found in a standard Zope instance. You can use ./bin/instance start to run Zope in daemon mode. It can also be used to run tests:

$ ./bin/instance test -s plone.portlets

Once your buildout installation is up and running, you will still need to install a Plone site.  Log in to the Zope Management Interface (ZMI) and from "select type to add..." choose Plone Site.  Fill in the required details and submit.  Now you have a Plone site at the ID that you specified.

Directories in the buildout

Before we dive into buildout.cfg, let us take a quick look at the directories that buildout has created for us:

bin/
Contains various executables, including the buildout command, and the instance Zope control script.
eggs/
Contains eggs that buildout has downloaded. These will be explicitly activated by the control scripts in the bin/ directory.
downloads/
Contains non-egg downloads, such as the Zope source code archive.
var/
Contains the log files (in var/log/) and the file storage ZODB data (in var/filestorage/Data.fs). Buildout will never overwrite these.
src/
Initially empty. You can place your own development eggs here and reference them in buildout.cfg. More on that later.
products/
This is analogous to a Zope instance's Products/ directory (note the difference in capitalisation). If you are developing any old-style Zope 2 products, place them here. We will see how buildout can automatically download and manage archives of products, but if you want to extract a product dependency manually, or check one out from Subversion, this is the place to do so.
parts/
Contains code and data managed by buildout. In our case, it will include the local Zope installation, a buildout-managed Zope instance, and Plone's source code. In general, you should not modify anything in this directory, as buildout may overwrite your changes.

You can check in a buildout directory to a source code repository to share it among developers. In this case, you should ignore the directories bin/, eggs/, downloads/, var/, and parts/. Each developer can run bootstrap.py to get these back, and will normally need local copies anyway. All your configuration should be in the buildout.cfg file, and all custom code in src/ or products/.

 

You *must* use Python 2.4

Posted by Mikko Ohtamaa at Nov 18, 2007 08:40 PM
If you execute buildout commands using your standard Python interpreter (/usr/bin/python) and it happens to be Python version 2.5, the buildout instance won't start.

This is very big GOTCH YA, since

1) There are no obvious warnings *in the process itself* (someone please add python version check to buildout!)

2) paster/buildout doesn't prompt you for python interpreter, but dummily assumes the default one, which does not work

You'll get this cryptic error messages when trying starting using python 2.5:
...

  File "/opt/zope2.10/lib/python/AccessControl/Implementation.py", line 51, in setImplementation
    from AccessControl import ImplC as impl
  File "/opt/zope2.10/lib/python/AccessControl/ImplC.py", line 18, in <module>
    from cAccessControl import rolesForPermissionOn, \
  File "/opt/zope2.10/lib/python/AccessControl/SimpleObjectPolicies.py", line 82, in <module>
    from DocumentTemplate.DT_Util import TemplateDict
  File "/opt/zope2.10/lib/python/DocumentTemplate/__init__.py", line 21, in <module>
    from DocumentTemplate import String, File, HTML, HTMLDefault, HTMLFile
  File "/opt/zope2.10/lib/python/DocumentTemplate/DocumentTemplate.py", line 112, in <module>
    from DT_String import String, File
  File "/opt/zope2.10/lib/python/DocumentTemplate/DT_String.py", line 19, in <module>
    from DT_Util import ParseError, InstanceDict, TemplateDict, render_blocks, str
  File "/opt/zope2.10/lib/python/DocumentTemplate/DT_Util.py", line 19, in <module>
    from html_quote import html_quote, ustr # for import by other modules, dont remove!
  File "/opt/zope2.10/lib/python/DocumentTemplate/html_quote.py", line 4, in <module>
    from ustr import ustr
  File "/opt/zope2.10/lib/python/DocumentTemplate/ustr.py", line 18, in <module>
    nasty_exception_str = Exception.__str__.im_func
AttributeError: 'wrapper_descriptor' object has no attribute 'im_func'

Install python 2.4 (apt-get install python2.4). Then execute all commands using Python 2.4. Instead of typing

./bootstrap.py

type

python2.4 bootstrap.py

and

./bin/buildout

type

python2.4 bin/buildout

and

./bin/instance

type

python2.4 bin/instance

Hopefully this helps (you Gutsy people)

python2.4-dev, too

Posted by Catherine Devlin at Dec 16, 2008 11:13 AM
Thanks, Mikko! On Ubuntu Hardy, I also needed to install python2.4-dev (apt-get install python2.4-dev); until I did, I got

... eggs/plone.recipe.zope2install-2.6-py2.4.egg/plone/recipe/zope2install/__init__.py", line 183, in install
    'build_ext', '-i',
AssertionError

Two things of note under Windows

Posted by Justin Bennett at Dec 31, 2007 06:13 PM
Two things I had to do under Windows to get this to work:

1. For buildout.exe, I had to change dir to c:\plone\myproject\bin\, and run buildout.exe like this: buildout -c c:\plone\myproject\buildout.cfg

2. After that, GCC was having trouble finding cc1.exe, until I added "c:\MinGW\libexec\gcc\mingw32\3.4.5" to my system PATH statement.

I did those two extra steps, and now its working like a charm. Windows makes everything more fun!

Buildout Fails on PAS

Posted by Chris Thompson at Jan 08, 2008 11:18 AM
I have successfully used the plone3 buildout on windows as described here and in the Book, however, for some reason it now fails on a permission error that seems to be caused by svn files in the distribution of PluggableAuthService. I get the error:

While:
  Installing plone.

An internal error occured due to a bug in either zc.buildout or in a
recipe being used:

OSError:
[Errno 13] Permission denied: 'c:\\docume~1\\thomps~1\\locals~1\\temp\\tmp5gaswqbuildout-plone\\PluggableAuthService\\.svn\\dir-prop-base'

Running buildout on Windows

Posted by Lynn Alford at Apr 13, 2008 10:58 PM
I've found two things to be true so far.

1) the path to the buildout should have no spaces. There is some Python utility called that chokes on any spaces in a Windows path
2) PythonWin is great for editing but don't depend on it to run the buildout. I've had much better success by getting a command line window instead.

I did end up downloading and installing a Zope instance instead of using buildout to do this job. Mostly the Zope requires a few more components that must be manually added before buildout can work and installing it by hand solved those issues.

Problem on Debian

Posted by David Broudy at Jul 24, 2008 12:08 AM
If you try to use the zope2.10 (from etch-backports) to as the zope2_install path in the buildout, the zope2instance recipe will fail trying to chown everything to zope:zope when run as a normal user.

It turns out that the mkzopeinstance.py script included with debian has been customized with an extra option --service-user, which defaults to the debian system user zope, since it's not specified by the recipe. Setting effective-user would negate the effect of this at runtime, but modified mkzopeinstance.py also tries to chown, which obviously fails.

The easiest solution is to just let the buildout get it's own zope install, even though it seems kind of redundant to me.

effective-user

Posted by Timothy Reaves at Aug 05, 2008 04:47 PM
This seems a pretty serious issue; following the instructions, I get errors relating to the chown too. But there seems to be no way to specify effective-user. I'm having this issue on a Mac.

effective-user

Posted by Timothy Reaves at Aug 05, 2008 04:50 PM
Teh exact message I get is:
ZConfig.ConfigurationError: A user was not specified to setuid to; fix this to start as root (change the effective-user directive in zope.conf)

edit directive

Posted by John DeStefano at Aug 06, 2008 01:08 PM
Did you try doing what the error message said (change the "effective-user directive in zope.conf")? What happens if you set this directive to 'plone'? What user owns the installed files?

Change the mybuildout folder owner

Posted by afewtips at Aug 25, 2008 03:30 PM
I was getting the same error and noticed that the owner of the folders for the buildout was root and not myself. So I changed the owner to myself and all works fine.

I think when the buildout is created, it uses the ownership of the folder mybuildout
(or whatever you called it when you called paster).

I don't know why it set the owner to root in the first place.

Don't execute commands with SUDO

Posted by afewtips at Aug 26, 2008 01:23 PM
I figured out why root was the owner of the folders and why it was asking to set effective-user to root.
Because I was running my commands with SUDO ...

That's why the "set effective-user" permission error was coming up.