Multiple Plone sites per zope instance - using separate Data.fs files for each one.

Using Zope's mount point facilities, distinct ZODB files under the main zope root can be created to host Plone sites.

Introduction

In some cases, administrators with multiple Plone sites may want to isolate each site in a distinct database file.  By default there is just one "Data.fs" file containing the entire Zope site.  If each Plone site is in a distinct file, issues of backup, deletion, and relocation can (hopefully) be separable.  This is an alternative to running multiple Zope instances.  Discussion is welcome, since the author was attempting to optimize the problem of running multiple Plone sites corresponding to different user groups.  N.B. this discussion applies only to vanilla ZODB sites and not to ZEO sites. See the comment below regarding ZEO sites.

Steps to carry out

1) Edit Zope.conf

In zope.conf, the main Data.fs storage is defined by

<zodb_db main>
# Main FileStorage database
  <filestorage>
    path $INSTANCE/var/Data.fs
  </filestorage>
  mount-point /
</zodb_db>

Adding another section below this like

<zodb_db group1>
  # FileStorage for group1
  <filestorage>
    path $INSTANCE/var/Data_group1.fs
  </filestorage>
  mount-point /group1
</zodb_db>

Restarting zope will establish a new the file Data_group1.fs and make the mount point "/group1" available under the zope structure.

However, the new mount point "/group1", is not yet usable.

2) Use the zope management interface to enable the mount point.

The next step is to go to the Zope management page (not Plone's ZMI), and go to the Root Folder.  In the "Add" pull down menu near the bottom, find the "ZODB Mount Point" entry and choose it.  The "/group1" entry added above should be there, but not yet enabled.  Select the new mount point via the check box, and click on "Create Selected Mount Points"  After doing this, the new mount point should appear in the Root Folder as a subfolder.

3) Create the Plone site inside the new subfolder. 

Click on the new subfolder "/group1" and go inside.  Once there, use the pull down "Add" menu and choose the "Plone Site" entry.  Create a Plone site as usual and you are on your way.  The site will be contained in the distinct "Data_group1.fs" file. 

Issues

  • The path to the plone site is http://yoursite:8080/group1/<sitename> where <sitename> is the name given the Plone site on creation.  It makes sense to choose the names of "group1" and "sitename" in some sensible fashion to avoid a redundant name (easy to get this wrong the first time).

  • This technique is largely untested by the author of this How-to, hence tread carefully.  Feedback is welcome.

  • This discussion does NOT apply to ZEO based sites.  In particular: "the recipe only applies to Zope installs without ZEO. There is considerably more than needs to be done to get a mount point in ZEO. See: http://trizpug.org/Members/cbc/mkzeowmp.txt/file_view and http://thread.gmane.org/gmane.org.user-groups.zope.trizpug/528/focus=528"

 

 

 

 

Disintegrate multiple Plones

Posted by Juergen R. Plasser at Sep 21, 2006 07:58 PM
When you don't have more Plone instances in one single Zope instance, this method may be a simple way to extract each Plone instance into a separate data.fs. From then on you can use this files to mount them into other Zope instances. I have not tested this but it sounds reasonable.

-Jürgen

First think then write ... ;)

Posted by Juergen R. Plasser at Sep 21, 2006 08:03 PM
I meant "When you do have more Plone ..." (and you should read "these files" :) - sorry!

Your correct Sir!!!

Posted by bobb at Oct 10, 2006 02:07 AM
I do this all the time, and move instances between windows and *nix instances, also.
A best practice (for me) is that each site has it's own .fs, and therefore it's own directory, and therefore it's own plone site. Nothing else is included in the .fs.

converting existing plone instances to mounted

Posted by David Hostetler at Jun 24, 2007 06:10 PM
I'm sorry if this should be obvious, but I'm unsure of how to convert an existing plone instance to one that is mounted from its own .fs file. I have a single zope with a single plone instance (named 'plone') in the root folder, using the good ol' Data.fs. I want to add another plone instance, but I'd like both plones to act as mount points, as described in this HOWTO. Adding the 2nd plone seems trivial -- I can just follow the steps described. But how do I safely convert the existing plone to a mount point, i.e. get it out of Data.fs and into its own .fs? Thanks.

cut&paste Plone Site to new mount makes it disappear

Posted by Chris W. at Mar 22, 2008 09:09 AM
I thought the obvious way to transfer a Plone Site to a mounted Test1.fs database was to cut and paste it in the ZMI. But when looking at the Test1 database after pasting, the size does not change. If you then pack the databases your site disappers! I have found no way to undo this either. Opening the folder will result in an error (Error Type: POSKeyError) on Plone 3.0.6

I then tried to do the same thing with *copy* and paste and it worked. The Test1.fs database size increased by about 900kB, and after deleting the Plone Site from the main DB and packing the main DB everything was fie without errors.

Maybe this is obvious to those with more in-depth Zope knowledge, as it seems to me that cut and paste only transfers the reference to the Plone Site and not the site itself, whereas copy and paste (with subsequent deletion of the original site) actually creates a new copy of the Plone Site object in the newly mounted database as intended.

Avoid cross-mount references

Posted by Jean Jordaan at Jul 11, 2008 11:09 AM
liawagner -- you're quite right. If you cut'n'paste, only a reference is made in the mounted database, and the content still stays in the old one. See this thread:
  http://www.nabble.com/data-[…]te-and-pack-td15439751.html

I think copy'n'paste, and then delete the original, is safe.

If anyone with more ZODB knowledge than me would like to confirm, that would be greatly appreciated.

zeo and zodb mounts

Posted by Todd Mitchell at Oct 04, 2006 08:16 PM
After reading many articles on how hard it was to mount multiple databases using zeo I was amazed at how easy it actually is. With plone 2.5 install zeo server comes set up with two clients. You really only have to do the following to add additional zodb mounts:

in server/etc/zeo.conf add:

<filestorage 2>
 path $INSTANCE/var/mynew.fs
</filestorage>


then in the client1 and client2 zope.conf add the new mount point

<zodb_db mynew>
 <zeoclient>
   server servername:port
   storage 2
   name mynewdb
   var $INSTANCE/var
  </zeoclient>
    mount-point mynew
</zodb_db>

then kick 'er over, you will see the new mynew.fs created and it will appear in the databases under control panel. Just go and add the new ZODB mount as normal.

It appears to be working fine for me.

To your point

Posted by bobb at Oct 10, 2006 02:09 AM
This technique is largely untested by the author of this How-to, hence tread carefully. Feedback is welcome....

I do this all the time (make sure your zope and plone versions are the same!!!)

plone upgrades

Posted by David Hart at Dec 26, 2006 10:57 PM
I use this technique to manage multiple Plone sites; as others have pointed out, it's safe and easy.

It also makes Plone upgrades easier; simply install the desired new Zope in a unique location (e.g. /opt/zope/296), create a new Zope instance with the latest Plone products installed (e.g. /var/lib/zope/instances/migrate-target), *copy* the old Plone site's Data.fs (ZODB mount point), and run portal migration. This way, migration kinks can be worked out safely; if necessary, start over with a fresh copy of Data.fs. When migration is complete and tested, Apache rewrites can be changed for a near seamless migration.

Note, in the <zodb_db> section, I typically also add::
    #explicit declaration of object ZODB mount point object type
    container-class OFS.Folder.Folder
    #performance/memory tuning
    pool-size 8
    cache-size 2000

Previous Plone Version

Posted by Roberto Allende at Jan 22, 2007 02:31 PM
Hello

Sorry to point this bue this practice is allowed in previous plone versions, are there any particular reason to say that's a plone 2.5 one ?

kind regards
r.

Re: Previous Plone Version

Posted by David Convent at Jan 25, 2007 10:03 AM
This technique is a Zope feature, it's not restricted to Plone portals.

So yes, you can apply that technique to sites running older Plone versions :-)

Mounting an existing Plone site

Posted by Sergey Volobuev at May 10, 2007 04:45 AM
I just have spent some time trying to mount an existing Data.fs in order to access data there and would like to share my experience.

There seems to be a little trick which seems not to be documented very well. Zope actually doesn't mount the "root" of object tree, which would be a behaviour similiar to Unix mount. What it does is mounting an _object_ within that tree, which has the same name as the mount point. So if, for example, we want to access a folder called "mysites" located at the root of a ZODB we're trying to mount, we have to call the mount point "mysites" as well.

Also, in case we want to mount a Plone Site (or some other object different from Folder), there's a container-class parameter in <zodb_db> section.

Read this document in spanish

Posted by Roberto Allende at Jun 23, 2008 08:44 PM
Múltiples sitios plones en una instancia zope - usando un archivo Data.fs por cada sitio: http://plone.org/[…]/multiples-sitios-plones

seperate Data.fs files in buildout

Posted by Alexander Loechel at Aug 04, 2008 03:43 PM
Hi,

is there a proven is there a proven concept for having separated Data.fs files for each Plone instance in on Zope-ZEO Environments that works with buildout?

Or even to have for example 4 ZODB-Files on two ZEO-Servern and to use them in all ZEO-Clients based on buildout?

seperate Data.fs files in buildout

Posted by Jim Leek at Nov 27, 2008 10:30 AM
Yes.

In your [zeoserver] section add a "zeo-conf-additional" stanza:

[zeoserver]
recipe = plone.recipe.zope2zeoserver
zope2-location = ${zope2:location}
zeo-address = 127.0.0.1:8100
effective-user = plone
zeo-conf-additional =
    <filestorage 2>
      path ${buildout:zeo-filestorage}/site1.fs
    </filestorage>
    <filestorage 3>
      path ${buildout:zeo-filestorage}/site2.fs
    </filestorage>

Then you need to modify the [client1] section and add a "zope-conf-additional" stanza:

zope-conf-additional =
    <zodb_db site1>
      # FileStorage database for site1
      <zeoclient>
        server localhost:8100
        storage 2
        name site1
        var /opt/plone/Plone-3.1/zeocluster/parts/client1/var
      </zeoclient>
      mount-point /site1
      #explicit declaration of object ZODB mount point object type
      container-class OFS.Folder.Folder
    </zodb_db>
    <zodb_db site2>
      # FileStorage database for site2
      <zeoclient>
         server localhost:8100
         storage 3
         name site2
         var /opt/plone/Plone-3.1/zeocluster/parts/client1/var
      </zeoclient>
      mount-point /site2
      #explicit declaration of object ZODB mount point object type
      container-class OFS.Folder.Folder
    </zodb_db>

Unfortunately you cannot just copy this into the [client2] section with the following code:

zope-conf-additional = ${client1:zope-conf-additional}

This is because the above code refers to /opt/plone/Plone-3.1/zeocluster/parts/client1/var

Therefore you will need to write things out longhand for the [client2] section:

zope-conf-additional =
    <zodb_db site1>
      # FileStorage database for site1
      <zeoclient>
        server localhost:8100
        storage 2
        name site1
        var /opt/plone/Plone-3.1/zeocluster/parts/client2/var
      </zeoclient>
      mount-point /site1
      #explicit declaration of object ZODB mount point object type
      container-class OFS.Folder.Folder
    </zodb_db>
    <zodb_db site2>
      # FileStorage database for site2
      <zeoclient>
         server localhost:8100
         storage 3
         name site2
         var /opt/plone/Plone-3.1/zeocluster/parts/client2/var
      </zeoclient>
      mount-point /site2
      #explicit declaration of object ZODB mount point object type
      container-class OFS.Folder.Folder
    </zodb_db>

seperate Data.fs files in buildout

Posted by Jim Leek at Nov 27, 2008 10:41 AM
Remember to indent the zeo-conf-additional and zope-conf-additional sections with some white space!

seperate Data.fs files in buildout (ZEO)

Posted by Chris W. at Jun 01, 2009 04:26 PM
Thanks, Jim for this helpful ZEO buildout example!

Just one part did not work for me with Plone 3.2.2 installed with the 'Unified Installer' on Debian 5:

In the [zeoserver] section...
zeo-conf-additional =
    <filestorage 2>
      path ${buildout:zeo-filestorage}/site1.fs
    </filestorage>
    <filestorage 3>
      path ${buildout:zeo-filestorage}/site2.fs
    </filestorage>

...the option ${buildout:zeo-filestorage} will cause the following error during buildout: "Error: Referenced option does not exist: buildout zeo-filestorage"

Since this option is apparently not available any more, "${buildout:zeo-filestorage}" can be replaced by "${buildout:directory}/var/filestorage" and the buildout will complete successfully.

The example zeoserver section will then look like this:

[zeoserver]
recipe = plone.recipe.zope2zeoserver
zope2-location = ${zope2:location}
zeo-address = 127.0.0.1:8100
effective-user = plone

zeo-conf-additional =
    <filestorage 2>
      path ${buildout:directory}/var/filestorage/site1.fs
    </filestorage>
    <filestorage 3>
      path ${buildout:directory}/var/filestorage/site2.fs
    </filestorage>

seperate Data.fs files in buildout

Posted by Carl Brewer at Jan 29, 2009 03:52 AM
This is how I did it with a buildout.cfg file and just a zope server, not ZEO :

in buildout.cfg :

in the [instance] section
I added this :
zope-conf-additional =
    <zodb_db foo_sites>
        # FileStorage for foo_sites
        <filestorage>
            path /opt/Plone-3.1/zinstance/var/filestorage/Data_foo_sites.fs
        </filestorage>
        mount-point /foo_sites
    </zodb_db>

I think this will get squashed to the left margin, but it's all tabbed in. There's probably a better way to tell it the path.

virtual mount-points add flexibility

Posted by Dylan Jay at Jun 17, 2009 03:46 AM
By using virtual mount point syntax

mount-point /mount1:/plone

You can then easily move your plone sites between mount points. The plone site sits in /plone in every .fs.

Also collective.recipe.filestorage makes the zeo config very easy.