#209: Add buildout to Unified Installer

Contents
  1. Definitions
  2. Motivation
  3. Assumptions
  4. Proposal
  5. Implementation
  6. Deliverables
  7. Risks
  8. Progress log
  9. Participants
by Steve McMahon last modified May 28, 2008 06:56 PM

The Unified Installer should provide a buildout-based framework to make it easier to manage product and egg additions.

Proposed by
Steve McMahon
Seconded by
Wichert Akkerman
Proposal type
Architecture
Assigned to release
Repository branch
trunk
State
completed

Definitions

 

Motivation

The big hurdle that the Unified Installer gets many users over is the building of a Python -- complete with PIL, libjpeg, and libz interfaces -- that will work with Zope and Python.

Users with Unified Installer builds face a problem, though, when it comes to adding egg-based products as they won't have the easy facilities for adding eggs and zcml provided by buildout.

Also, minor-version migration is painful for Unified Installer users. They must install the new update and copy over their data and custom packages. In a buildout environment, they'd be able to just run bin/buildout.

Assumptions

 

Proposal

Provide a version of the Unified Installer that builds the base Python, but then uses buildout for the final steps.

Implementation

I've been sketching this out in a Unified Installer branch and a new plone.recipe.unifiedinstaller, and have a working tarball in the Plone section of Launchpad (look for Unified Installer Plus Buildout).

A few key elements in this scheme:

  • The installer includes and installs a .buildout cache of eggs and downloads so that the installer may be run offline.
  • plone.recipe.unifiedinstaller does much of the finishing work previously done by the shell scripts.
  • plone.recipe.command is used to set ownership for the root install to match the effective user.

I've also taken the opportunity of the rewrite to add several POSIX style options to install.sh which will make it much easier to add new instances to an existing install.

 

Deliverables

Tarball that may be unpacked and executed.

 

Update of plone.org/documentation tutorial on Unified Installer.

Risks

Divergence from the binary installers, which can't depend on having a gcc/make environment, and will have to bundle a binary Zope.

Some (though not many) changes in layout from the old Unified Installer that will undoubtedly confuse some of the more recipe-bound users.

Users mucking up their production environments by running bin/buildout without backing up or planning a migration.

Progress log

An implementation of PLIP 209: "Unified Installer Plus Buildout" is

available for testing at:

 

https://launchpad.net/plone/3.0/3.0.5/+download/Plone-3.0.5-UnifiedInstallerBuildout-beta1.tar.gz

 

 

This is a full installer kit -- and a sizable download since it

includes all the components. If you'd like to just review the code,

the branch is at:

 

https://svn.plone.org/svn/plone/Installers/UnifiedInstaller/branches/UnifiedInstallerPlusBuildout/

 

 

I've also worked on two recipes that are used in the installer:

 

plone.recipe.precompiler compiles .pyc files for all the product .py

files, and makes it possible to tighten file ownerships a bit more

than if the Zope daemon write into those directories.

 

https://svn.plone.org/svn/collective/buildout/plone.recipe.precompiler/trunk/

 

 

plone.recipe.unifiedinstaller is only really useful for the Unified

Installer. It takes care of creating some scripts and texts similar to

those created by the old Unified Installer.

 

https://svn.plone.org/svn/collective/buildout/plone.recipe.unifiedinstaller/trunk/

 

Participants

Steve McMahon (smcmahon on plone.org, SteveM on #plone) 

Framework team vote

Posted by Martijn Pieters at Dec 13, 2007 10:31 PM
+1

Framework team vote

Posted by Andreas Zeidler at Dec 14, 2007 01:03 AM

Framework team vote

Posted by Raphael Ritz at Dec 17, 2007 12:20 PM
+1

Features to preserve

Posted by Claudio Battaglino at Dec 17, 2007 01:13 PM
The great benefits of using the unified installer are that it:
- comes already with the right python and the pil library
- installs all things under a unique directory
- is simple to use (./install.sh standalone)
I hope that these features will be preserved in the PloneUnifiedInstallerBuildout.

Definitely

Posted by Steve McMahon at Dec 17, 2007 04:36 PM
This is definitely the goal.

Also, the installer must be able to produce a production installation that will meet or exceed the standards of a security-minded system administrator.

Framework team vote

Posted by Tom Lazar at Dec 20, 2007 01:16 PM
+1

Framework team vote

Posted by Danny Bloemendaal at Dec 22, 2007 05:46 PM
+1

some useful buildout techniques in PloneBlobs account

Posted by ken manheimer at Jan 03, 2008 04:36 PM
i'm hoping whoever develops the unified installer buildout (SteveM?) will take a look at some of the techniques embodied in my PloneBlobs (http://myriadicity.net/Sundry/PloneBlobs) build.

in particular, i consolidate the settings which are shared between instances (which can be extensive, like the collections of eggs, src, zcml, etc) in a [common] virtual section, to then be used identically in the various types of instances - [instance], [client1], [client2] - and also, for some bits, in the [server]. see the buildout.cfg http://myriadicity.net/Sundry/PloneoutblobsBuildoutCfg and base.cfg http://myriadicity.net/Sundry/PloneoutblobsBaseCfg .

there's also some attention to mixing in elements of an independent third-party application's build recipe to the main one - described at http://myriadicity.net/Sund[…]lication-with-its-own-build . it would be nice if this kind of mixing would be considered in the overall design, to make it clearer to those developing builds for their own applications how to design it to fit together with the main build.