I recently had the pleasure of putting the world's foremost open source Content Management System into production on a nice shiny Xserve.[1] I thought I would share my experience here both to provide some pointers and to gather some feedback.[2]

Now, as we all know, Mac OS X is BSD with a thick candy coating, and is therefore reputed to have something for everyone. I usually host on FreeBSD, and I have to admit that I was skeptical coming into this project. However, I ended up having a great experience with OS X; it really is the best of both worlds. I especially recommend it if you will be handing things over to someone for whom piloting a web server is only a small part of their job.[3]

I am going to address four specific issues in this tutorial:

1. whether to install from the Plone for Mac installer or from source
2. how to start Zope automatically on bootup
3. whether to use Apple's Apache or install httpd from source
4. how to lay out the filesystem

First, however, let's look at the resources available for learning OS X.

Notes

[1] For the record, the box is a 1.0 GHz G4 with 1 GB of RAM running Mac OS X Server 10.3.7.

[2] Note that this doc is intended for those with an intermediate or greater knowledge of Plone hosting on BSD/Linux.

[3] Interestingly, I was concurrently installing Plone on Windows 2003 Server for another client. I also hope to write a document about that experience, which was much less positive.

Darwin does differ from the other BSDs, of course. Fortunately, it has a large community of users and some good documentation. Here's what I found:

#plone

OS X is fairly well-represented in the Plone community. Ask around.

#macosx

There is a well-stocked channel on freenode. I found them helpful when I asked questions, but they made too many pr0n jokes for my taste, so I didn't lurk.

www.apple.com

Apple has a good support website. I browsed and searched both the knowledge base and the discussion board, and found answers both ways. I did not resort to the mailing list.

GUI docs

On the other hand, I found the documentation within the OS X GUI itself to be less helpful. It was hard to move around the help system, and I kept getting dumped at search result pages that mostly linked to basic-level content.

man pages

I also didn't find the Darwin man pages particularly inspiring. For example, I hoped in vain that hier(7) would explain Apple's filesystem layout. Also, the prevailing web version -- http://www.hmug.org/man/ -- has a worse interface than the FreeBSD man interface, which unfortunately is a bit out of date at Darwin 7.0.1.

Apple Consultant Network

This turned out to be a solid base hit. Read on for details.

The first issue I faced was whether or not to use the pre-packaged Plone installer for Mac. The consensus on #plone seemed to be that all packaged versions of Plone — including the Windows and Mac installers of course, but also RPM and ports installs — are most useful for impressing new-comers to Plone, or perhaps for quick installs on development systems. However, folks doing serious Plone work seem to prefer installing manually from source, both for production and development installations.

I was glad for this feedback, since the Plone installer for Mac does a lot of crazy stuff. In fact, it seems to be the worst of both worlds: it lays things out on the filesystem in a very non-Unix way, but if I'm not mistaken it doesn't even give us a thick candy coating for it all. At least the Windows installer gives us a GUI.[4]

I wanted to keep the infrastructure as sane as possible, so I opted for source installs of Python and Zope (and Subversion, incidentally). There were no hitches with any of them. But while I went with a source install for Plone, I took a different tack with Apache.

Notes

[4] Update: Mark Murphy has replied with a different perspective regarding the Mac installer. Really it's just a cultural difference: I'm trying to use Mac OSX as a Unix system. (However, I admit that using the word "crazy" is a bit inflammatory; sorry. :^)

While I decided against a packaged install for Plone, I did in fact choose to integrate with the Apache instance that is bundled with OS X. I made this decision based on five considerations:

1. Unlike Plone, Apache is integrated with the operating system to begin with; I judged working with it to be less risky/costly than working against it, especially on my first time out on OS X.
2. The differences between a standard Apache install and OS X's are fairly well documented in the conf files and on the web.
3. There was very little I wanted to do that could not be done from the GUI.
4. The Apache GUI gives the client a chance to get their hands on their website in a tangible fashion. It is part of the Server Admin utility.[5]
5. I mean, c'mon, candy is yummy.

The one thing I couldn't do from Server Admin was RewriteRules. You can configure Alias and Redirect directives from there, but to implement RewriteRules (which are what I am used to) you have to edit the conf files by hand. Apache keeps a separate conf file for each virtual host that you configure through the GUI. This means that you don't have to touch httpd.conf, so your customizations are pretty well-isolated.

This is the point at which I decided to tap the Apple Consultant Network. I had researched enough to feel fairly comfortable that Server Admin was not going to overwrite my manual changes, but I didn't find anything directly applicable on the web, and I wanted a little more assurance for such a weak link in the chain.

I found a local company on consultants.apple.com and ended up hearing back from both principals within 5 minutes of calling their Help Desk, after hours even! I made a local business connection, and they answered my question for free! (They have an "under 15 minutes" policy.) In fact, for the next job I would seriously consider simply outsourcing sysadmin to these guys. Not sure how well this represents the rest of the Network, but these Mac guys do tend to be zealots, no?

One other note here: Server Admin complains if a website does not have a DocumentRoot (which it calls a "Web folder"). For this reason I created a dummy directory at /usr/local/zope/htdocs/. I put a README in there explaining its purpose, and left it at that.

Another slight advantage to using the bundled Apache install was that it is preconfigured to start automatically on boot-up. However, I had to manually configure Zope to do the same.

Notes

[5] You can find this program at Applications > Server > Server Admin

As expected, Apple uses a variant of the modern BSD startup system. However, instead of /usr/local/etc/rc.d/, user-defined startup scripts are located in /Library/StartupItems/. There are also other slight differences. Fortunately, I was not forced to learn these intricacies, because integrating Zope with it turned out to be fairly trivial.[6]

My strategy was to copy and paste the Apache startup folder and use that as an example:

  cp -r /System/Library/StartupItems/Apache /Library/StartupItems/Zope
  cd /Library/StartupItems/Zope
  rm -r Resources
  mv Apache Zope
  edit Zope

Edit the shell script, trimming unnecessary conditions and converting from apachectl to zopectl. I've posted mine in an appendix. Then edit StartupProperties.plist. PropertyList is an Apple-specific format that has both XML and non-XML implementations. If your system is like mine, the Apache plist will be in non-XML format. Somewhere along the line I ended up switching to XML format while trying to get things to work. In the end I don't think it matters, but my example is in XML.

The last thing you need to do is to edit /etc/hostconfig, and append the following to the end of it:

  ZOPESERVER=-YES-

I learned this critical tidbit from a random how-to on Apple.com (in the last paragraph). At first I thought the value of the Provides key in the plist file is what drives the service name in /etc/hostconfig. However, now I think that the hostconfig file provides context for the Zope startup script.[7] In any case, the files as described work for me.

After you edit these three files, bounce your server and see if Zope comes back up on reboot. If it does, then it is time to step back a bit and look at what we've done with the OS X filesystem.

Notes

[6] If you want to understand the system, as good a place to start as any is Start Me Up: Writing and Understanding OS X StartupItems from O'Reilly's Mac DevCenter. This tutorial is nearly great: he makes no mention of /etc/hostconfig, however.

[7] Incidentally, here's another place where the Darwin man pages let me down. Why no manual entry for hostconfig?

Apple does not adhere to any kind of standard or tradition in laying out the filesystem, so some work has to be done in order to make the box comfortable. In this particular box we had a drive for the system, and a RAID volume[8] for data at /Volumes/Zope/. I chose to link this from /usr/local/zope, and I did the Zope installation and created a Zope instance using this more conventional path.

I also made Apple's Apache setup a little more comfortable by adding a link from /usr/local/apache to /etc/httpd/ (even though Apache isn't actually local on this box—it's part of the base system). The latter is the default location, and is actually itself a link to /private/etc/httpd/.[9] However, I didn't use my link in any scripts, but purely as a convenience while logged in.

I decided to leave Apple's rc.d implementation alone. Adding our configuration there was really a one-time shot, and it came at the end of the project when things were rushed.

Therefore, the relevant parts of my final filesystem layout looked like this:

/Library/StartupItems/Zope
/usr/local/apache -> /etc/httpd/
/usr/local/zope -> /Volumes/Zope/
/usr/local/zope/2.7.3
/usr/local/zope/htdocs
/usr/local/zope/instance

The most paranoid among you will want to avoid using symlinks. However, I believe this is fairly standard practice. And in fact, upgrading is much easier if, for example, you link from zope/2.7.3 to zope/zope, and use that to make your instance. The same could/should be done with your python interpreter. This, however, is not an OS X-specific topic, and is better dealt with in detail elsewhere.

And on that note, let's wrap this up.

Notes

[8] We used Apple's built-in software RAID, which only supports RAIDs 0 and 1. Apple RAID is managed with the Disk Utility (Applications > Utilities > Disk Utility). The nav is a little goofy, but once I figured it out the actual config went fine.

[9] I learned a hard lesson here: in trying to remove this symlink I accidentally removed the httpd directory! If you include the trailing slash when referring to a link that points to a directory, then you are actually referring to the source directory itself. Whatever version of bash was installed gave me the trailing slash on the first tab-completion, and well, it was late, and I eventually added a -r to the rm! I spent the whole next morning on-site reinstalling Apache (it's on Disc 2, part of Server Essentials ;^) ). Of course, since I was just trying to rename the link I should have used mv in the first place. And that, boys and girls, is how we learn Unix!

So that's how I did it, folks. To be honest, there are some missing pieces. The two that I can think of are:

1. ZEO — for live debugging, maintenance, and scalability
2. caching — for performance and scalability

Both are technologies that I look forward to learning in the coming months. Regarding caching in particular, I am able to report that Apple has a "Web Performance Cache" which sits as a separate process in front of Apache. I was reluctant to use it without first understanding caching in general. And in reality, I don't anticipate that this client will be getting cache-worthy amounts of traffic.

Another issue to consider is the choice of filesystem. I used the default, Mac OS X Extended w/ Journaling (abbreviated HFS+). Apparently, older versions of HFS+ had performance issues versus the traditional UFS.[10] I have not noticed this problem.

With that said, I'd like to invite your feedback. Here are a few questions to prime the pumps, bearing in mind that we are talking about issues specific to production installations of Plone on OS X:

• Do you install software from source, from Darwin Ports, or from some other packaging system?
• In particular, do you use Apple's Apache or do you install that from source as well?
• Do you use Apple's Web Performance Cache, or a different mechanism?
• What public Plone sites do you have in production on OS X? Let's start a list! The one discussed here is: www.svsd.net.

Feedback on this doc should either be directed to the mailing list, or it should be posted here using the commenting feature on the appropriate page. I'll periodically factor these discussions back into the main document. Thanks!

Notes

[10] Thanks to Nate Aune for this link.

#!/bin/sh

##
# Zope Web Application Server
##

. /etc/rc.common

StartService ()
{
    if [ "${ZOPESERVER:=-NO-}" = "-YES-" ]; then
        ConsoleMessage "Starting Zope"
        /usr/local/zope/instance/bin/zopectl start
    fi
}

StopService ()
{
    ConsoleMessage "Stopping Zope"
    /usr/local/zope/instance/bin/zopectl stop
}

RestartService ()
{
    if [ "${ZOPESERVER:=-NO-}" = "-YES-" ]; then
        ConsoleMessage "Restarting Zope"
        /usr/local/zope/instance/bin/zopectl restart
    fi
}

RunService "$1"

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN"
"http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
     <key>Description</key>
     <string>Zope web application server</string>
     <key>OrderPreference</key>
     <string>None</string>
     <key>Provides</key>
     <array>
             <string>Zope Server</string>
     </array>
     <key>Requires</key>
     <array>
             <string>Web Server</string>
     </array>
</dict>
</plist>

Basically, the Mac OS X Installer was designed to appeal to the general Mac public. Mac users expect an application to install where they can see it and not spread out over the disk in various directories. Even Apple is careful not to put everything in a standard "unix" directory structure (though there is no such thing as a standard unix directory structure).

The other problem with putting stuff in unix dirs like /etc or /var is that the directories are hidden in the Finder. Also, during a Mac OS X migration from machine to machine, Apple's installer will not migrate those unix dirs. Most Mac users aren't going to understand why their Plone install breaks on their target machine after migration if we were to use the "unix" dirs.

We've debated on whether we should have a GUI controller or not. A GUI controller is a good eye candy piece, but if we really want to make an effective controller, we feel it should be a web based controller so that instances and configuration can be done remotely if needed. We'd also like something that could be utilized on all Plone installations whether it's Mac, Win, Unix, etc. However, we haven't gotten to the point of spending the time to develop such a thing.

We also put the Mac OS X Installer together in such a way that multiple instances could be supported in an easy, straight forward manner. In addition, the Installer structure supports new upgrades of Plone, Zope, and Python. All three are totally self contained within the install and not dependent on the version of python installed in Mac OS X. This sort of support is important to production level systems. If Mac OS X is upgraded, the installed Plone should continue to work and not break because of an OS upgrade.

All of these decisions were to make sure the Mac OS X Plone installation would feel like a retail level product out of the gate. It's also designed to what we feel is an "enterprise" level. The product should work out of the box, should handle multiple sites in an easy to configure way, should not break if an OS install occurs, should not hide in any murky areas of the OS, and it should all be visible in one place.

There are other things we'd like to add to the Installer, but we haven't done so yet.

The other advantage to using the standard installer is that it's been tested by thousands of users.