SOAP support for Plone

Why?

The use case is plain and simple: you'd like your Plone (or rather Zope) service to support SOAP. Zope natively supports XML-RPC, which is a good start, but there is certainly some demand for SOAP out there. Documentation on this topic is scarce (almost in the same way as the Dodo is scarce) and hardly relevant for the current state of the art. This document tries to fill the documentation gap, and provide some relevant information on how to get your Plone doing webservices using SOAP.

The steps describe in this document should enable a developer to get SOAP support going in 20 minutes. This is the time it took me, and I'm by no means a SOAP expert. Finding all the relevant documentation, browsing through lot's of source code, etc., took me more than a week though...

Prerequisites

In this howto we make some basic assumptions, that are true for most modern Plone development. If you do things differently, well, there is some extra work involved.

SOAP support has been available for ages for Zope, in the guise of the SOAPSupport package. Sadly, there has been no new release since 2007, and there's no sign of life there. Also, the solution works, but is a typical case of ancient technology for most contemporary developers. So we have based this howto on a new(-ish) kid on the block: z3c.soap, and some older technologies: soaplib and ZSI, the Zope SOAP Infrastructure.

Also, we assume the use of buildout. Please note: you do not really need soaplib unless you'd like your WSDL to be automagically generated. And who wouldn't..?

z3c.soap

What z3c.soap does, is add a namespace to ZCML, that enables you to set up a BrowserView as a SOAP view. It really is that simple. Isn't that just what you've always wanted? Under the hood, z3c.soap patches the publisher, just to let you know... This may or may not be a problem for your setup.

soaplib

soaplib is a python SOAP library, that provides (undocumented) WSDL generation and gives you a way to define complex in- and output parameters. The project seems to be stalled, but the WSDL generation works, and is very useful.

Sadly, for python2.4 the soaplib that is distributed contains a syntax error in the file xml.py. The diff is given here:

36c36
<         ns = self.nsmap[key] if key in self.nsmap else ''
---
>         ns = self.nsmap.get(key, '')
64c64,66
<     namespace_map = { None: default_ns } if default_ns is not None else {}
---
>     namespace_map = {}
>     if default_ns is not None:
>         namespace_map = {None: default_ns }

so you'll need to patch those two lines...

ZSI

The ZSI project is currently the de-facto project for SOAP support for Zope. The package provides some tools to generate SOAP stubs for client/server and also, like soaplib, provides an infrastructure to define complex in- and output. Sadly, when you want it all (SOAP support and WSDL generation, you'll need both ways of defining in- and output)

The code/setup described below has been developed using Python 2.4, soaplib 0.5.1, z3c.soap 0.8.1 and ZSI 2.0 rc3.

Getting ready

Configure your buildout, using the eggs section like so:

eggs =
  ...
  z3c.soap
  soaplib
  ZSI
  ...

always assuming you'd like the ZSI and z3c.soap as part of your buildout, not of your global python...

This should add basic SOAP support to your Plone site.

The SOAP service

Now either you create a new egg, or you include the SOAP service(s) you want into an existing egg. Whatever you do, this is outside the scope of this document. We assume you are familiar with setting up egg infrastructure. Let's add the SOAP service to an existing egg/product...

In your base configure.zcml, add an entry:

<include package=".soap" />

and create a directory called soap in you package base. Add an __init__.py file in the directory, so as to make it into a module. The file can be left empty, but of course will need to be valid Python. I always use this content:

""" Being there... """

Now we'll create the actual service. We need a service that sets or gets the title of your portal folders. This can be achieved using the following code (in a file called soapfolderview.py):

import ZSI
from Products.Five import BrowserView
class SOAPFolderView(BrowserView):
  def get_title(self):
    return self.context.Title()
  def set_title(self, title):
    try:
      self.context.setTitle(title)
      self.context.reindexObject()
    except:
      raise ZSI.Fault(ZSI.Fault.Client, "Title not set")

It is indeed a very simple service, but as soon as you have the infrastructure in place, you should be able to figure out more interesting services... Check out the testing directory of z3c.soap or the README.txt in SVN for other examples.

Now, the one thing remaining is to configure the view as a SOAP view in the configure.zcml within your soap directory. This should be something like:

<configure
  xmlns="http://namespaces.zope.org/zope"
  xmlns:soap="http://namespaces.zope.org/soap"
>
  <!-- include the soap namespace for the soap:view directive -->
  <include package="z3c.soap" file="meta.zcml" />
  <soap:view
    for="OFS.interfaces.IFolder"
    methods="set_title"
    class=".soapfolderview.SOAPFolderView"
    permission="cmf.ModifyPortalContent"
  />
  <soap:view
    for="OFS.interfaces.IFolder"
    methods="get_title"
    class=".soapfolderview.SOAPFolderView"
    permission="zope2.View"
  />
</configure>

Please note that the permission for the setter is cmf.ModifyPortalContent, so you'd need your client to authenticate for setting the title.

Your egg basedir should now contain a directory called soap, that holds: a file called __init__.py, a file called configure.zcml and a file called soapfolderview.py.

That's all folks! really? Really! From here it's only a matter of starting your Plone (Zope) service, and you should have your first SOAP service.

Of course you may wish to test it... so I'll kindly give you the source code for a client:

from SOAPpy import SOAPProxy, URLopener
# define your plone URL here, and point it to some folder
url = http://localhost:8080/plone/news/
# If you need authorization, use this url
# url = http://username:password@localhost:8080/plone/news/
# define the namespace 
namespace = http://www.evilempire.org/
server = SOAPProxy(url) 
# if you don't want to see the SOAP message exchanged,
# comment the two following lines
#
server.config.dumpSOAPOut = 1
server.config.dumpSOAPIn = 1
print server._ns(namespace).get_title()
print server._ns(namespace).set_title("New title")
print server._ns(namespace).get_title()

Note that you'll have to edit the script to use the proper username and password...

Run the client code, and you should see some XML in, some XML out, and finally the echo of your call...

Adding WSDL

To make your webservice into something that other people can use, you'd have to add a description of the service. This is the WSD(L), an XML description of what you have to offer in terms of public operations. You can either write a WSDL for your service by hand, but a much nicer idea is to use introspection and declarations on your existing service class, to generate the WSDL. soaplib can do this for you, although documentation on this topic is hard to find. Mainly because it is close to non-existent...

soaplib exploits decorators to enable introspection of your service, and generate the WSDL. The class SoapServiceBase gives you a wsdl method, that will create the WSD (XML) for your service.

Let's make the created folder view generate it's own WSDL:

import ZSI
from Products.Five import BrowserView
from soaplib.service import SoapServiceBase, soapmethod
from soaplib.serializers.primitive import String
class SOAPFolderView(BrowserView, SoapServiceBase):
  def __init__(self, context, request):
      BrowserView.__init__(self, context, request)
      SoapServiceBase.__init__(self)
  def wsdl(self):
      res = self.request.RESPONSE
      res.setHeader(Content-Type, text/xml; charset="utf-8")
      return SoapServiceBase.wsdl(self, self.context.absolute_url())
  @soapmethod(_returns=String)
  def get_title(self):
      return self.context.Title()
  @soapmethod(String)
  def set_title(self, title):
      try:
          self.context.setTitle(title)
          self.context.reindexObject()
      except:
          raise ZSI.Fault(ZSI.Fault.Client, "Title not set")

This is it for the service class, now let's configure wsdl as page view for the service in the configure.zcml in your soap directory:

  <browser:page
    for="OFS.interfaces.IFolder"
    name="wsdl"
    class=".folderview.SOAPFolderView"
    attribute="wsdl"
    permission="zope2.View"
  />

Do not forget to add the browser namespace to the configure element (xmlns:browser="http://namespaces.zope.org/browser").

Now rev up your Plone, and browse to http://localhost:8080/plone/somefolder/wsdl (always assuming your plone runs on localhost, port 8080, and you have a folder called somefolder).

You should now get the WSDL for this service! Now don't bother trying to call the actual service again at this stage, since the method that is called now, is the decorator, which greatly confuses z3.soap... Read on for a solution.

@soapmethod

This decorator is used by soaplib to describe your SOAP methods on the service. The decorator should at least specify in and out paramaters (if any). For more complex types, soaplib provides serializers that you can use to describe your parameters. Let's say you want to return a complex type: an array of objects that have a url, and an array of keywords. In your service, define this complex type like so:

class ContentItem(ClassSerializer):
    class types:
        url = String
        keywords = Array(String)

and decorate your method like:

@soapmethod(_returns=Array(ContentItem))

Serializers like String, Array and ClassSerializer are all imported from soaplib.serializers. This way you can easily create complex in- and outputparameters.

Throwing in complex types

The service described above is nice, but in general you'd require something a bit more interesting for your SOAP services. Support for more complex in- and output is provided by ZSI  for z3c.soap. It would be really nice if z3c.soap would support the same stuff as soaplib, but this is sadly not the case. So when you want to have complex in- and output, and WSDL generation, you'll need to describe your in- and output in two ways: using the soaplib serializer framework, and using the ZSI typecode concept. The typecode will be interpreted by the z3c.soap handler to parse and return the proper XML structures.

Assume you'd like to return an array of objects that hold one string attribute, and one attribute that is an array of strings, you'll need the following definitions:

import ZSI
from soaplib.serializers.clazz import ClassSerializer
from soaplib.serializers.primitive import String, Array
# The soaplib part
class SomeClass(ClassSerializer):
    class types:
        title = String
        keywords = Array(String)
  
    def __init__(self, name, title=None, keywords=[]):
        self.name = name
        self.title = title
        self.keywords = keywords
    def __str__(self):
        return str((self.name, self.title))
# Do the ZSI part
SomeClass.typecode = ZSI.TC.Struct(SomeClass,
                                       (ZSI.TC.String(title),
                                        ZSI.TC.Array(string, ZSI.TC.String(keyword), 
                                                     keywords)
                                        ),
                                       SomeClass)
@soapmethod(_returns=Array(SomeClass))
def yourMethod(self):
  ...

Further primitive and complex type support for both libraries can be found in the source...

If you find declaring for both libraries tedious, you may wish to experiment with the code below, that implements a rudimentary soaplib2zsi bridge... your mileage may vary...

from soaplib.serializers.clazz import ClassSerializer
from soaplib.serializers.primitive import String, Any, Array, Integer
from soaplib.serializers.primitive import Double, Float, Boolean
import ZSI
simpleTypes = {
    String: ZSI.TC.String,
    Integer: ZSI.TC.Integer,
    Any: ZSI.TC.Any,
    Boolean: ZSI.TC.Boolean,
    Float: ZSI.TC.Decimal,
    }
class ClassProperty(property):
    def __get__(self, cls, owner):
        return self.fget.__get__(None, owner)()
class SoaplibZSIBridge(ClassSerializer):
    @classmethod
    def getZSIType(cls, name, soaplibType):
        if simpleTypes.haskey(soaplibType):
            zsiType = simpleTypes.get(soaplibType)
            return zsiType(name)
        elif soaplibType.__class__.__name_ == "Array":
            sub = cls.getZSIType(soaplibType.serializer.get_datatype(),
                                 soaplibType.serializer)
            return ZSI.TC.Array(name, sub, pname=name)
    @classmethod
    def typecode(cls):
        struct = []
        for k, v in cls.soapmembers.items():
            struct.append(cls.getZSIType(k, v))
        return ZSI.TC.Struct(cls, struct, cls.__name)
    """ Make typecode accessible as class property """
    typecode = ClassProperty(_typecode)

N.B. your classes used for return values should now extend SoaplibZSIBridge instead of ClassSerializer.

Sadly, when trying to make a full round trip (using soapUI to create the client request based upon the generated WSDL), it appeared that the z3c.soap part cannot handle empty input... So you actually need to make sure that the request body contains an actual message. Based on the WSDL that is generated for operations that take no parameters, soapUI creates a request with an empty body...

Also, when using the soaplib decorator, the call to your SOAP method is actually a call to the decorator method... So the request fails since the introspection of ZSI introspects the decorator method instead of the actual method.

The first problem is solved by creating another serializer for the input, that acts as placeholder:

class Nill(ClassSerializer):
    class types:
        pass
    def __init__(self, name):
        self.name = name
        
    def __str__(self):
        return str(self.name)

and change the decorator into this:

@soapmethod(Nill, _returns=Array(SomeClass)

The second problem is solved by using a marker for the actual service, that is used for introspection:

class MarkerService(SoapServiceBase):
    @soapmethod(Nill, _returns=Array(SomeClass))
    def getUrls(self, nill):
        pass
class ActualService(BrowserView):
    def wsdl(self):
        """ Generate WSDL for this service """
        res = self.request.RESPONSE
        res.setHeader(Content-Type, text/xml; charset="utf-8")
        return SoapServiceBase.wsdl(MarkerService(), self.context.absolute_url())
    def getUrls(self, nill):
        ....

By now you should have a SOAP service that generates it's own WSDL, and actually works!