Plone behind Varnish using Pound for SSL

Purpose

While Plone is a fabulous CMS, it's not a particularly fast web server. This is by design: Plone is focused on serving dynamic content rather than static. For any medium- to large-scale site, Plone should be behind a caching HTTP accelerator, such as Varnish (or Squid or Apache). Varnish is more directly suited to this purpose than the alternatives because it was designed from the ground up to act as this kind of accelerator (whereas Squid and Apache only provide this functionality as extensions of their intended purposes). However,Varnish doesn't handle serving content over SSL. Pound, however, can be used in front of Varnish as an SSL wrapper.

Prerequisites

The configuration described in this document has been verified using the following software versions:

• Plone 3.2.3 (under Zope 2.10.7) with CacheSetup 1.2.1 installed, from http://plone.org, installed using the Unified Installer.
  
• Varnish 2.0.4 from http://varnish.projects.linpro.no/ (Note that the Varnish config file is modified from the one produced by the plone.recipe.varnish 1.0rc11 available on PyPi: http://pypi.python.org/pypi/plone.recipe.varnish
  
• Pound 2.2.7 from http://www.apsis.ch/pound/

This configuration has only been verified under Linux (Ubuntu 8.04 LTS), though it should work under any *nix platform.

This document does not cover installation of these products, only configuration. All of the software listed above should be installed already, including any process monitoring or init scripts needed to allow the configuration to survive a server reboot.

This document assumes that all three components (Plone, Varnish and Pound) will be running on a single server, which would be suitable for most low- to mid-range websites. It should be easy to modify this configuration to split the components out across servers, which can have both performance and security benefits.

This configuration has not been extensively tuned. Further enhancements are undoubtedly possible.

Extracts from various config files only show sections that differ from the "stock" files.

Step by step

The steps used to achieve this configuration are detailed below. Pay special attention to the comments in the configuration file listings: most of the meat of the how-to is there.

1. Verify that Plone is successfully installed and serving content on the two default ports configured by the Unified Installer (8080 and 8081). 
2. Edit the Varnish config file (varnish.vcl). By default, this file is in /etc/varnish/varnish.vcl.
   

#Configure Varnish to see both of the Plone ZEO clients for load balancing
backend backend_0 {
.host = "127.0.0.1";
.port = "8080";
}
backend backend_1 {
.host = "127.0.0.1";
.port = "8081";
}

# configure Varnish for to load balance between the two ZEO clients
director director_0 round-robin {
 {
 .backend = backend_0;
 }
 {
 .backend = backend_1;
 }
}

acl purge {
 "localhost";
}

# configure Varnish to look for HTTP headers indicating the protocol (HTTPS or HTTP) and respond accordingly
sub vcl_recv {
 set req.grace = 120s;
 set req.backend = director_0;
 if (req.http.X-Forwarded-Proto == "https" ) {
 set req.http.X-Forwarded-Port = "443";
 } else {
 set req.http.X-Forwarded-Port = "80";
 set req.http.X-Forwarded-Proto = "http";
 }

# Here is the magic: configure Varnish to rewrite incoming requests into Zope VirtualHostMonster URLs 
# where you see "(www\.|ipv6\.|dev\.)" each of "www" "ipv6" and "dev" are possible subdomains that 
# apply to your site. For instance, this allows people to access your site as both 
# http://mydomain.com/ AND as http://www.mydomain.com/ (or http://ipv6. . . . or http://dev. . . .)
# If your site has more than 2 or 3 DNS components (e.g., mysever.mydepartment.mycompany.com has 4 
# components, you'll need to adjust the Regex strings.
# note that the basis for this section was gracelessly lifted from the following site:
#  http://varnish.projects.linpro.no/ticket/536
if (req.http.host ~ "^(www\.|ipv6\.|dev\.)?([-0-9a-zA-Z]+)\.([a-zA-Z]+)$") {
 set req.http.host = regsub(req.http.host, "^(www\.|ipv6\.|dev\.)?([-0-9a-zA-Z]+)\.([a-zA-Z]+)$", "\1\2.\3");
 set req.url = "/VirtualHostBase/" req.http.X-Forwarded-Proto
 regsub(req.http.host, "^(www\.|ipv6\.|dev\.)?([-0-9a-zA-Z]+)\.([a-zA-Z]+)$", "/\1\2.\3:")
 req.http.X-Forwarded-Port
#You may want to UNcomment the line below (starting "regsub( . . ." and comment out the line starting 
# "/Plone/ . . .". The default Plone Unified Installer configuration creates your Plone site inside
# the Zope root as "/Plone". If you are serving multiple Plone sites, the commented-out line will allow 
# you to put them in the Zope root as "mysite.com" and "myothersite.com". If you're using the default,
# leave it as is.
#regsub(req.http.host, "^(www\.|ipv6\.|dev\.)?([-0-9a-zA-Z]+)\.([a-zA-Z]+)$", "/\1\2.\3/VirtualHostRoot")
 "/Plone/VirtualHostRoot"
 req.url;
 }


# Everything below here is "stock" from the plone.recipe.varnish package available on PyPi:
# http://pypi.python.org/pypi/plone.recipe.varnish

 if (req.request == "PURGE") {
 if (!client.ip ~ purge) {
 error 405 "Not allowed.";
 }
 lookup;
 }

 if (req.request != "GET" &&
 req.request != "HEAD" &&
 req.request != "PUT" &&
 req.request != "POST" &&
 req.request != "TRACE" &&
 req.request != "OPTIONS" &&
 req.request != "DELETE") {
 /* Non-RFC2616 or CONNECT which is weird. */
 pipe;
 }

 if (req.request != "GET" && req.request != "HEAD") {
 /* We only deal with GET and HEAD by default */
 pass;
 }

 if (req.http.If-None-Match) {
 pass;
 }

 if (req.url ~ "createObject") {
 pass;
 }

 remove req.http.Accept-Encoding;

 lookup;
}

sub vcl_pipe {
 # This is not necessary if you do not do any request rewriting.
 set req.http.connection = "close";
}

sub vcl_hit {
 if (req.request == "PURGE") {
 purge_url(req.url);
 error 200 "Purged";
 }

 if (!obj.cacheable) {
 pass;
 }
}

sub vcl_miss {
 if (req.request == "PURGE") {
 error 404 "Not in cache";
 }

}

sub vcl_fetch {
 set obj.grace = 120s;
 if (!obj.cacheable) {
 pass;
 }
 if (obj.http.Set-Cookie) {
 pass;
 }
 if (obj.http.Cache-Control ~ "(private|no-cache|no-store)") {
 pass;
 }
 if (req.http.Authorization && !obj.http.Cache-Control ~ "public") {
 pass;
 }

}

3. Make sure that whatever script is used to invoke Varnish tells it to listen to port 80 on BOTH the localhost and public IP addresses for your server (the argument to varnishd is "-a 127.0.0.1:80, 123.456.789.012:80" where the latter IP address is your server's public address).
4. Edit the Pound config file (pound.cfg). By default, this file is in /etc/pound/pound.cfg. Note that this  configures Pound to listen only for SSL requests, since Varnish can already handle the non-SSL requests handily. Also note the "Cert . . . " line: your SSL cert, in PEM format, must be at the path specified on that line.
   

ListenHTTPS
        Address 123.456.789.012 # put your server's public IP address here
        Port 443
        Cert "/etc/pound/server.pem"
        AddHeader "X-Forwarded-Proto: https"
        HeadRemove "X-Forwarded-Proto"
        Service
                BackEnd # this is the address and port for Varnish
                        Address 127.0.0.1
                        Port 80
                End
        End
End

5. Make sure that caching is turned on in your Plone site by visiting Site Setup->Cache Configuration Tool. In there, make sure the "Enable CacheFu" checkbox is checked, the "Active Cache Policy" is set to "With Caching Proxy", and the "Proxy Cache Purge Configuration" is set to "Simple Purge (squid/varnish in front)".
6. In the "Site Domains" blank, list BOTH the HTTP and HTTPS URLs to your website for any subdomain you're serving from that site, e.g.

http://www.mysite.com:80
https://www.mysite.com:443
http://mysite.com:80
https://mysite.com:443

7. Fire up both Varnish and Pound. Assuming your configuration files are correct, you should now be able to reach your site via both standard HTTP and SSL-encrypted HTTPS. Additionally, you site should serve content via both those protocols significantly faster than it would with Plone alone.