Simple Plone Clustering with Squid and Pound

A simple Squid/Pound/Zope setup. Now you can do it without rewrite rules or redirector scripts and take advantage of squid acceleration and the scalability of ZEO.

There are so many how-tos out there on how this is supposed to be setup, but each seem to be missing one part or adds in some extra complexities that aren't really needed.

  1. Pound is super simple to setup
  2. ZEO is a little more difficult
  3. Squid is the most complex

In this setup no redirector script is needed for squid since we can handle the mappings in the VirtualHostMonster mappings tab. We have to run pound and squid on the same port, but different interfaces so the url rewriting does the right thing. VirtualHostMonster itself won't rewrite the ports without some kind of rewriting done beforehand. This is ok since we can just bind Pound to the loopback interface and the squid to the external interface.

Here is a copy of my squid.conf. Just replace @@my.domain.com@@ with the host name that you are serving and @@my_external_ip_address@@ with the external IP of your server:

    hierarchy_stoplist cgi-bin ?
    acl QUERY urlpath_regex cgi-bin \?
    no_cache deny QUERY
    auth_param basic children 5
    auth_param basic realm Squid proxy-caching web server
    auth_param basic credentialsttl 2 hours
    auth_param basic casesensitive off
    refresh_pattern ^ftp:               1440    20%     10080
    refresh_pattern ^gopher:    1440    0%      1440
    refresh_pattern .           0       20%     4320
    # Basic ACLs
    acl all src 0.0.0.0/0.0.0.0
    acl localhost src 127.0.0.1/32
    acl ssl_ports port 443 563
    acl safe_ports port 80 443
    acl zope_servers src 127.0.0.2 127.0.0.1 
    acl manager proto cache_object
    acl connect method connect
    # deny requests to unknown ports
    http_access deny !safe_ports
    acl accelerated_protocols proto http https
    acl accelerated_domains dstdomain @@my.domain.com@@
    acl accelerated_ports myport 80 443 
    http_access allow accelerated_domains accelerated_ports accelerated_protocols
    # Purge access - zope servers can purge but nobody else
    acl purge method PURGE
    http_access allow zope_servers purge
    http_access deny purge
    # Reply access
    http_reply_access allow all
    # Cache manager setup - cache manager can only connect from localhost
    # only allow cache manager access from localhost
    http_access allow manager localhost
    http_access deny manager
    # deny connect to other than ssl ports
    http_access deny connect !ssl_ports
    # ICP access - anybody can access icp methods
    icp_access allow localhost
    # And finally deny all other access to this proxy
    http_access deny all
    coredump_dir /usr/local/squid/cache
    http_port @@my_external_ip_address@@:80
    httpd_accel_host 127.0.0.1
    httpd_accel_port 80
    httpd_accel_single_host on
    httpd_accel_with_proxy on
    httpd_accel_uses_host_header on 

Pound's 1.X config is simple also:

  ListenHTTP 127.0.0.1,80
  User zope
  Group zope
  LogLevel 1
  UrlGroup ".*"
  BackEnd 127.0.0.1,51003,1
  BackEnd 127.0.0.1,51004,1
  EndGroup

Here is the same thing for pound 2.X:

    User "zope"
    Group "zope"

    ListenHTTP
      Address 127.0.0.1
      Port 81
      xHTTP 2
    End

    Service
        BackEnd
            Address 127.0.0.1
            Port  51003
        End
        BackEnd
            Address 127.0.0.1
            Port  51004
        End
       Session
          Type    COOKIE
          ID      "__ac"
          TTL     300
       End
    End

All that is left is to configure 2 ZEO Clients and a ZEO Storage Server to answer the requests. In one of the ZEO Clients go into the root VirtualHostMonster and on the mappings tab add one entry per domain name and site you want to serve.

!!!!! WARNING !!!!

Posted by Will T at May 26, 2006 10:37 AM
The suggested changes are totally insecure !!

If anyone changes their proxy settings in their browser to www.your-plone-server.com they will then be able to surf the net via your server thereby masking their own identity and also avoiding their own company firewall chewing though your bandwidth in the process

What you should do is chane the http_access deny to_localhost to allow if your running pound on the same box as this will then only allow reqests to be proxied only of their final destination is your own server

Security Fix

Posted by Calvin Hendryx-Parker at May 30, 2006 07:06 PM
We actually just caught this a couple days before hand, but I had forgotten about this howto. I have updated the configuration with one that is secure. Thanks for the notification.

Pound for the uninitiated

Posted by Estienne Swart at Jun 08, 2006 08:20 PM
Might be worth mentioning what Pound (a reverse-proxy and load-balancer) and where it can be obtained.

Pound is found here:

http://www.apsis.ch/pound

I managed to get things working with Pound 1.10 (was feeling too lazy to figure out differences in the config file format for Pound 2).

This was certainly much easier to get running than a Squid + Rewrite rules setup (after considerable troubleshooting, I eventually gave up on the semi-automated approach provided by CacheFu). Has anyone benchmarked and assessed robustness of these two approaches?

(also added comment to http://plone.org/[…]/squid-integration-pointers mentioning this how-to)

Pound 1.X required for the example

Posted by Fred van Dijk at Jan 30, 2007 09:50 PM
The example Pound config requires Pound 1.x . Pound 2.x has a new configuration file syntax. If an expert could post it here... I still have to figure it out.

Re: Pound 1.X required for the example

Posted by Calvin Hendryx-Parker at Jan 31, 2007 07:21 PM
I just added the 2.X version of the config for those that are interested.

Squid 2.6+

Posted by Andy Ferguson at Nov 03, 2007 02:50 PM
Newer versions of Squid no longer accept the httpd_accel_* directives.
Instead you need something like this:

http_port @@my_external_ip_address@@:80 accel defaultsite=@@my.domain.com@@
cache_peer 127.0.0.1 parent 81 0 no-query originserver

or, for virtual hosts,

acl accelerated_domains dstdomain @@my.domain_1.com@@
  ...
acl accelerated_domains dstdomain @@my.domain_n.com@@

http_port @@my_external_ip_address@@:80 accel vhost
cache_peer 127.0.0.1 parent 81 0 no-query originserver


warning: pound rewrites redirect URLs

Posted by David Glick at Oct 02, 2008 09:54 PM
If you're having trouble with Pound rewriting redirect URLs incorrectly, add the following line to pound.cfg:
RewriteRedirect 0

(The option is called RewriteLocation in earlier versions of Pound.)