Plone 3 - Apache2/SSL - Squid on FreeBSD 7 with PF

« Return to page index

How to configure a Plone 3 production server with Squid and Apache 2 + SSL on a FreeBSD 7 server with PF, the packet filter. (updated for Plone 3.3.5)

Introduction (READ THIS FIRST)

Tutorial Overview

     This tutorial will guide you through the configuration of an Apache 2.2 SSL installation in front of a Plone 3.2.2 instance with Squid running in between Apache and Zope on FreeBSD 7.x.  SSH tunnelling will be used for remote management of the Zope/Plone instance (i.e. ZMI).   All of this will reside behind a PF firewall.  Configuration and installation of the individual components will be covered in each section.

     Apache 2.2 will accept http and https connections from the public Internet.  Apache will then rewrite the URL and proxy the HTTP or HTTPS request to the Squid caching server.  Squid will then forward the request to the ZEO (Zope Enterprise Object) client.  The ZEO client (part of Zope)  will send the request to the ZEO server (part of Zope). The ZEO client keeps a cache of recently used database objects.  These objects are retrieved from the ZEO Server when not in the cache.

Apache
a.b.c.d:80/443
=> Squid Cache
127.0.0.1:8902
=> Zope Client
127.0.0.1:8901
=> Zope Server
127.0.0.1:8900

NOTE:  a.b.c.d denotes the public static IP address that is mapped to your domain name.

     By configuring Plone 3 in this manner, ZEO clients can easily be added to the buildout configuration  Each of the ZEO clients can share cached objects thus increasing concurrency and reducing ZODB (Zope Object Database) access frequency.  Apache2 provides the full power of mod_rewrite for URL customization.  HTTP over SSL will provide secure access to the Zope Management Interface (ZMI).  Last of all, PF, OpenBSD's packet filter will lock down access to the server.

 

Prepare FreeBSD

Kernel modules and tunable parameters

Enable IP Forwarding in the Kernel

If you have not already done so, enable IP forwarding in the kernel:

# sysctl net.inet.ip.forwarding=1
# sysctl net.inet.ip.fastforwarding=1
# sysctl net.inet6.ip6.forwarding=1

Make sure that the following lines exist in /etc/sysctl.conf so that the next time you reboot, IP forwarding is enabled by default:

# /etc/sysctl.conf
net.inet.ip.forwarding=1
net.inet.ip.fastforwarding=1
net.inet6.ip6.forwarding=1

Enable HTTP Accept Filter

Next, make sure that the HTTP Accept filter is loaded into the kernel.  You can check this by running the following command:

# kldstat
2    1 0xc0b12000 2464     accf_http.ko

If the filter is not loaded, edit /boot/loader.conf and add the following line so that when you reboot, the HTTP Accept filter kernel module is loaded.

# /boot/loader.conf
accf_http_load="YES"

Last of all, to load the module immediately, run the following command:

# kldload accf_http

System V Shared Memory and Semaphore Parameters

Modify System V shared memory and semaphore parameters

# sysctl kern.ipc.shmall=32768
# sysctl kern.ipc.shmmax=134217728
# sysctl kern.ipc.semmap=256

Once again, make these changes permanent by adding the following to /etc/sysctl.conf

# /etc/sysctl.conf
kern.ipc.shmall=32768
kern.ipc.shmmax=134217728
kern.ipc.semmap=256
net.inet.ip.forwarding=1
net.inet.ip.fastforwarding=1
net.inet6.ip6.forwarding=1

System V "Read-Only" Semaphore Parameters

Modify the System V "Read-Only" Semaphore Parameters by adding the following to /boot/loader.conf

NOTE: You must reboot for the new values of these parameters to take effect

# /boot/loader.conf
kern.ipc.semmni=256
kern.ipc.semmns=512
kern.ipc.semmnu=256
accf_http_load="YES"

Setup Server Environment

Create privileged users & install Python 2.4, PIL, Apache2, LibXML2, ZopeSkel & Setuptools

NOTE: # denotes that the command is typed by the root user

Python 2.4

# python2.4
-su: python2.4 command not found
# which python2.4
# ls /usr/local/bin/python2.4
ls: /usr/local/bin/python2.4: No such file or directory
# cd /usr/ports/lang/python24
# make install clean
# export PATH=$PATH:/usr/local/bin
# echo "export PATH=\$PATH:/usr/local/bin" >> /root/.bash_profile


Python Imaging Library

# cd /usr/ports/graphics/py-imaging
# make install clean


LibXSLT

# cd /usr/ports/textproc/libxslt
# make install clean

 

LibXML2

# cd /usr/ports/textproc/libxml2
# make install clean
# cd ../py-libxml2
# make install clean
# cd ../py-xml
# make install clean


Apache 2.2

Add any additional modules that you may need

# cd /usr/ports/www/apache22
# make config
--accept the defaults and make sure the following are checked--
X ALIAS
X Threads
X PROXY
X PROXY_CONNECT
X PROXY_FTP
X PROXY_HTTP
X PROXY_AJP
X PROXY_BALANCER
X SSL
X SUEXEC
X VHOST_ALIAS
X REWRITE
X MIME
X MIME_MAGIC
X INCLUDE
X EXPIRES
X HEADERS
X SSL
# make install clean


Setuptools and ZopeSkel

# mkdir /usr/local/plone
# cd /usr/local/plone
# wget http://peak.telecommunity.com/dist/ez_setup.py
# python2.4 ez_setup.py
# easy_install -U ZopeSkel

 

Create group and users for Web services to run under

# groupadd www
# pw useradd www -c "Apache Server" -d /dev/null -g www -s /sbin/nologin
# pw useradd zadmin "Zope Admin" -d /dev/null -g www -s /sbin/nologin

Setup Plone & configure Buildout

Create & Configure Plone instance via zc.buildout (updated for Plone 3.3.5)

Plone 3 Buildout

# cd /usr/local/plone
# paster create -t plone3_buildout mydomain.com
# cd mydomain.com
# python2.4 bootstrap.py
# mkdir /usr/local/plone/mydomain-buildout-cache
# mkdir /usr/local/plone/mydomain-buildout-cache/eggs
# mkdir /usr/local/plone/mydomain-buildout-cache/downloads
# chown -R zadmin:www

buildout.cfg

[buildout]
eggs-directory=/usr/local/plone/mydomain-buildout-cache/eggs
download-cache=/usr/local/plone/mydomain-buildout-cache/downloads

parts =
    zope2
    productdistros
    client1
    zeoserver
    zopepy
    squid-build
    squid-instance

# Change the number here to change the version of Plone being used
extends = 
    http://dist.plone.org/release/3.3.5/versions.cfg
versions = versions

# Add additional egg download sources here. dist.plone.org contains archives
# of Plone packages.
find-links =
    http://dist.plone.org/release/3.3.5
    http://dist.plone.org/thirdparty
    http://dist.plone.org
    http://download.zope.org/ppix/
    http://download.zope.org/distribution/
    http://effbot.org/downloads

# Add additional eggs here
eggs = 
    Products.DocFinderTab
    Products.CacheSetup

# Reference any eggs you are developing here, one per line
# e.g.: develop = src/my.package
develop =

[zope2]
# For more information on this step and configuration options see:
# http://pypi.python.org/pypi/plone.recipe.zope2install
recipe = plone.recipe.zope2install
fake-zope-eggs = true
url = ${versions:zope2-url}

additional-fake-eggs = 
    ZConfig
    pytz

# Use this section to download additional old-style products.
# List any number of URLs for product tarballs under URLs (separate
# with whitespace, or break over several lines, with subsequent lines
# indented). If any archives contain several products inside a top-level
# directory, list the archive file name (i.e. the last part of the URL,
# normally with a .tar.gz suffix or similar) under 'nested-packages'.
# If any archives extract to a product directory with a version suffix, list
# the archive name under 'version-suffix-packages'.
[productdistros]
# For more information on this step and configuration options see:
# http://pypi.python.org/pypi/plone.recipe.distros
recipe = plone.recipe.distros
urls =
nested-packages =
version-suffix-packages =

[zeoserver]
recipe = plone.recipe.zope2zeoserver
zope2-location = ${zope2:location}
zeo-address = 127.0.0.1:8900
effective-user = zadmin

[client1]
# For more information on this step and configuration options see:
# http://pypi.python.org/pypi/plone.recipe.zope2instance
recipe = plone.recipe.zope2instance
zope2-location = ${zope2:location}
zeo-client = true
zeo-address = ${zeoserver:zeo-address}
zodb-cache-size = 5000
zeo-client-cache-size = 300MB
user = admin:mysecretpassphrase
effective-user = zadmin
http-address = 127.0.0.1:8901
debug-mode = off
verbose-security = off

# If you want Zope to know about any additional eggs, list them here.
# This should include any development eggs you listed in develop-eggs above,
# e.g. eggs = Plone my.package
eggs =
    Plone
    ${buildout:eggs}

# If you want to register ZCML slugs for any packages, list them here.
# e.g. zcml = my.package my.other.package
zcml =

products =
    ${buildout:directory}/products
    ${productdistros:location}

[zopepy]
# For more information on this step and configuration options see:
# http://pypi.python.org/pypi/zc.recipe.egg
recipe = zc.recipe.egg
eggs = ${client1:eggs}
interpreter = zopepy
extra-paths = ${zope2:location}/lib/python
scripts = zopepy

[squid-build]
recipe = plone.recipe.squid:build
url = http://www.squid-cache.org/Versions/v2/2.7/squid-2.7.STABLE9.tar.gz

[squid-instance]
recipe = plone.recipe.squid:instance
bind = 127.0.0.1:8902
backends = 127.0.0.1:8901
cache-size = 1G
user = zadmin
group = www
# ./bin/buildout
# chown -R zadmin:www ../mydomain.com*
# ./bin/zeoserver start
# ./bin/client1 start

 

If your server is remote, then perform the following from the client (desktop computer):

ssh -f user@mydomain.com -L 8901:localhost:8901 -N

Now open a browser window and go to http://localhost:8901/manage

  1. login as admin with mysecretpassphrase (the passphrase you put in buildout.cfg) as the password
  2. go to the top right of the screen and click on the drop down menu.
  3. select add and then choose plone site
  4. set the ID to Plone
  5. click on  portal_quickinstaller which will be on the left under your new plone site
  6. install CacheSetup
  7. install any other Products that you may need
  8. you can close the ssh connection now

Configure Apache

Configure Apache 2.2 VirtualHosts for HTTP/HTTPS, Generate SSL key


Copy Default Apache 2.2 Configuration Files

# cd /usr/local/etc/apache22
# cp extra/httpd-ssl.conf Includes/
# cp extra/httpd-vhosts.conf Includes/
# cp extra/httpd-default.conf Includes/
# cp extra/httpd-mpm.conf Includes/


Create OpenSSL Key

# openssl req -new -x509 -nodes -out mydomain.com.crt -keyout mydomain.com.key
# mkdir ssl.crt
# mkdir ssl.key
# mv mydomain.com.crt ssl.crt
# mv mydomain.com.key ssl.key
# chmod -R 400 ssl.key
# chmod -R 400 ssl.crt
# chown -R www:www ssl.key
# chown -R www:www ssl.crt

 

The relevant sections that I have added to each of the Apache configuration files are as follows:

/usr/local/etc/apache22/httpd.conf

Line:33  Listen a.b.c.e:80   # default
Line:34  Listen a.b.c.e:443  # default

Line:36  Listen a.b.c.d:80   # virtual host (will connect to squid)
Line:37  Listen a.b.c.d:443  # virtual host (will connect to squid)

Line:77  LoadModule cache_module libexec/apache22/mod_cache.so
Line:78  LoadModule disk_cache_module libexec/apache22/mod_disk_cache.so

Line:83  LoadModule deflate_module libexec/apache22/mod_deflate.so
Line:84  LoadModule log_config_module libexec/apache22/mod_log_config.so
Line:88  LoadModule expires_module libexec/apache22/mod_expires.so
Line:89  LoadModule headers_module libexec/apache22/mod_headers.so
Line:95  LoadModule ssl_module libexec/apache22/mod_ssl.so
Line:96  LoadModule mime_module libexec/apache22/mod_mime.so
Line:112 LoadModule rewrite_module libexec/apache22/mod_rewrite.so
Line:119 LoadModule proxy_module libexec/apache22/mod_proxy.so
Line:120 LoadModule proxy_connect_module libexec/apache22/mod_proxy_connect.so
Line:121 LoadModule proxy_ftp_module libexec/apache22/mod_proxy_ftp.so
Line:122 LoadModule proxy_http_module libexec/apache22/mod_proxy_http.so

Line:178 User www
Line:179 Group www
Line:210 ServerName internalhostname.mydomain.com # default
Line:217 DocumentRoot "/usr/local/www/apache22/data"


Line:500 #Include etc/apache22/extra/httpd-mpm.conf
Line:518 #Include etc/apache22/extra/httpd-vhosts.conf
Line:530 #Include etc/apache22/extra/httpd-ssl.conf


Line:541 Include etc/apache22/Includes/*.conf

 

/usr/local/etc/apache22/Includes/httpd-vhosts.conf

NameVirtualHost a.b.c.d:80


<VirtualHost a.b.c.d:80>

   ServerName mydomain.com
   ServerAdmin user@mydomain.com
   ServerSignature On

   ErrorLog "/var/log/mydomain-error_log"
   CustomLog "/var/log/mydomain-access_log" common
   LogLevel warn

   <IfModule mod_rewrite.c>
      RewriteEngine On
      RewriteLogLevel 2
      RewriteRule ^/(.*) \
         http://127.0.0.1:8902/VirtualHostBase/http/%{SERVER_NAME}:80/Plone/VirtualHostRoot/$1 [P]
      <IfModule mod_proxy.c>
         ProxyVia On
         <ProxyMatch http://127.0.0.1:*/.* >
            Order deny,allow
            Deny from all
            Allow from mydomain.com
            Allow from internalhostname.mydomain.com
            Allow from 127.0.0.1
         </ProxyMatch>
      <Directory proxy:*>
          Order deny,allow
          Deny from all
          Allow from mydomain.com
          Allow from internalhostname.mydomain.com
          Allow from 127.0.0.1      
      </Directory>
      <LocationMatch "^[^/]">
         Deny from all
      </LocationMatch>
    </IfModule>
  </IfModule>
</VirtualHost>

 

/usr/local/etc/apache22/Includes/httpd-ssl.conf

NameVirtualHost a.b.c.d:443

<VirtualHost a.b.c.d:443>

DocumentRoot "/usr/local/www/apache22/data"
ServerName mydomain.com
ServerAdmin user@mydomain.com
ErrorLog "/var/log/mydomain-ssl-error_log"
TransferLog "/var/log/mydomain-ssl_access.log"

   ServerSignature On
   SSLEngine on

   SSLCipherSuite ALL:!ADH:!EXPORT56:RC4+RSA:+HIGH:+MEDIUM:+LOW:+SSLv2:+SSLv3:+EXP:+eNULL
   SSLCertificateFile "/usr/local/etc/apache22/ssl.crt/mydomain.com.crt"
   SSLCertificateKeyFile "/usr/local/etc/apache22/ssl.key/mydomain.com.key"

   <FilesMatch "\.(cgi|shtml|phtml|php)$">
      SSLOptions +StdEnvVars
   </FilesMatch>
   <Directory "/usr/local/www/apache22/cgi-bin">
      SSLOptions +StdEnvVars
   </Directory>
   BrowserMatch ".*MSIE.*" \
      nokeepalive ssl-unclean-shutdown \
      downgrade-1.0 force-response-1.0

   CustomLog "/var/log/mydomain-ssl_request.log" \
      "%t %h %{SSL_PROTOCOL}x %{SSL_CIPHER}x \"%r\" %b"

 <IfModule mod_rewrite.c>
      RewriteEngine On
      RewriteLogLevel 2
      RewriteRule ^/(.*) \
      http://127.0.0.1:8902/VirtualHostBase/https/%{SERVER_NAME}:443/Plone/VirtualHostRoot/$1 [P]

    </IfModule>
    <IfModule mod_proxy.c>
       ProxyVia On
       <ProxyMatch http://127.0.0.1:*/.* >
          Order deny,allow
          Deny from all
          Allow from mydomain.com
          Allow from internalhostname.mydomain.com
          Allow from localhost
          Allow from 127.0.0.1
       </ProxyMatch>
       <Directory proxy:*>
          Order deny,allow
          Deny from all
          Allow from mydomain.com
          Allow from internalhostname.mydomain.com
          Allow from localhost
          Allow from 127.0.0.1
       </Directory>
       <LocationMatch "^[^/]">
          Deny from all
       </LocationMatch>
    </IfModule>
</VirtualHost>

Fix permissions on Apache directories

# chown -R www:www /usr/local/www

Configure Squid

Configure Squid

Edit squid.conf as follows:

/usr/local/plone/mydomain.com/parts/squid-instance/squid.conf

# This configuration file requires squid 2.6+.  It is untested with squid 3.x.

visible_hostname mydomain.com
cache_effective_user zadmin
cache_effective_group www
http_port 127.0.0.1:8902 vhost defaultsite=mydomain.com
pid_filename /usr/local/plone/mydomain.com/var/squid.pid
icp_port 3131

## Log files (http://wiki.squid-cache.org/SquidFaq/SquidLogs)
cache_access_log /usr/local/plone/mydomain.com/var/log/squid-access.log
cache_log /usr/local/plone/mydomain.com/var/log/squid-cache.log
cache_store_log none

# Cache storage
cache_dir ufs /usr/local/plone/mydomain.com/var/squidstorage 800 16 256
cache_mem 64 MB
maximum_object_size 10 MB
maximum_object_size_in_memory 1 MB

# Purge access - zope servers can purge but nobody else
# (works best if Zope and Apache are on different IPs)
acl zope_servers src 127.0.0.1
acl purge method PURGE
http_access allow zope_servers purge
http_access deny purge

# Deny caching of POST requests
acl post_requests method POST
cache deny post_requests

# Cache Peers
cache_peer 127.0.0.1 parent 8901 0 no-query originserver login=PASS name=server_0


# Cache Peer Access
acl all src 0.0.0.0/0.0.0.0

cache_peer_access server_0 allow all


Start Squid

# cd /usr/local/plone/mydomain.com
# chown -R zadmin:www ../mydomain*
# ./bin/squid -z -f /usr/local/plone/mydomain.com/parts/squid-instance/squid.conf
# ./bin/squid-instance

Configure PF

Configure PF Firewall

Configure PF Firewall

## MACROS-----
ext_if="bge0"
set loginterface $ext_if
internet_ports = "{80, 443}"

# Table Setup
# /etc/iface_addresses contains the following
# a.b.c.d
# a.b.c.e
table <iface_addresses> persist file "/etc/iface_addresses"
table <bruteforce> persist

# set Block Policy option
set block-policy return

# set Skip Filtering option on localhost
set skip on lo0

scrub in all
antispoof quick for $ext_if inet

# block ip addresses contained in bruteforce table
block in log (all, to pflog0) quick on $ext_if from <bruteforce> to any

# block and then log outgoing packets that don't have our address as source
block out log (all, to pflog0) quick on $ext_if from ! <iface_addresses> to any

# block nmap scans
block in log (all, to pflog0) quick on $ext_if inet proto { tcp, udp } from any to any flags FUP/FUP

# block everything by default
block in on $ext_if all

# pass in icmp and keep state
pass in quick on $ext_if inet proto icmp all keep state

# pass in traffic from localhost
pass in quick on $ext_if proto tcp from 127.0.0.1 to <iface_addresses>

# pass in traffic on internet ports
pass in on $ext_if proto { tcp, udp } from any to <iface_addresses> port $internet_ports flags S/SA keep state

# pass in throttle ssh connection attempts and block their ip if a bruteforce attempt is detected
pass in quick on $ext_if proto tcp from any to any port ssh \
     flags S/SA keep state \
     (max-src-conn 15, max-src-conn-rate 5/3, \
      overload <bruteforce> flush global)

# allow planet admin ip addresses
pass in on $ext_if proto { tcp, udp } from $planet_admin to $ext_if

# keep state on outbound connections made from one of the ip addresses on interface
# prevent sequence number attacks
pass out on $ext_if proto { tcp, udp } all modulate state

# keep state on remaining outbound connections
pass out on $ext_if all keep state

Start All Services

Configure CacheSetup - Start Apache, Squid, Plone

  1. open a browser window and go to http://mydomain.com/manage
  2. login as admin:mysecretpassphrase
  3. go to /portal_squid
  4. select the Proxy Urls tab and input the following in the corresponding sections:

 Cache URLs

http://127.0.0.1:8902
https://127.0.0.1:8902

 Purge URLs

Python:object.getUrlsToPurge(setup='squid_behind_apache')

 

Now go to http://mydomain.com/portal_cache_settings

Enable CacheFu

Default Cache Policy (v1.2)

   Proxy Cache Purge Configuration

        Purge with VHM URLs (squid/varnish behind apache, VHM virtual hosting)

Under Site Domains place the following

http://mydomain.com:80

http://www.mydomain.com:80

https://mydomain.com:443

https://www.mydomain.com:443

Under Proxy Cache Domains place the following

http://127.0.0.1:8902

https://127.0.0.1:8902

Now click on Save

Restart PF

# /etc/rc.d/pf restart

 

Before starting all services, you will need to make sure that /etc/hosts is properly configured:

Here is an example:

::1     localhost
127.0.0.1     localhost
a.b.c.e   internalhostname internalhostname.mydomain.com
a.b.c.d   mydomain.com www.mydomain.com

Now let's check all permissions and restart Apache, Squid, and Plone/Zope

# apachectl stop
# kill -9 `cat /usr/local/plone/mydomain.com/var/squid.pid` \
&& rm /usr/local/plone/mydomain.com/var/squid.pid
# /usr/local/plone/mydomain.com/bin/client1 stop
# /usr/local/plone/mydomain.com/bin/zeoserver stop

# chown -R zadmin:www /usr/local/plone/mydomain*
# chown -R www:www /usr/local/www/data
# cd /usr/local/plone/mydomain.com
# ./bin/zeoserver start
# ./bin/client1 start
# ./bin/squid-instance
# apachectl start