Embrace and Extend Existing Products: The Zope 3 Way

« Return to page index

With Zope 3 techniques (also available in Zope 2) you can cleanly extend an existing product without changing any of its code directly. In this tutorial we add annotations to a content type based on keywords. The use case we have is: extending Quills. The code is in the keywordadapter product in the collective.

Introduction

Use case: extending Quills

I have a weblog_. At the time of writing this is a hand-made Zope
blog, consisting entirely of page templates and scripts, which
sit in the Zope Database. See the status from early 2006 on the
`wayback machine`_. It works, it is fast, but it is kludgey. I
have to do too many things before I can publish a new weblog entry.
So there is room for improvement. Plus several other weblogs are
available in Plone. Quills seems to be good, and I know a few persons
who also use it, for instance my brother_, so I choose to use that
too.

.. _weblog: http://maurits.vanrees.org/weblog
.. _`wayback machine`: http://web.archive.org/web/20060425175543/http://maurits.vanrees.org/weblog/
.. _brother: http://vanrees.org/weblog

Actually, I have two weblogs at my site. One is a general blog where
I post personal things or entries relating to programming. The other
is a completely `Dutch blog`_ for my church_. Look at the
`old version`_. Actually it is more of a podcast. It contains small
entries with a link to an external mp3 file containing a complete
service of my church, and bigger entries with a link to a local mp3
file with just the sermon in case our pastor Erik was leading this
service. So one service could actually have two entries. Again, it
works, it is fast, but it is kludgey.

.. _`Dutch blog`: http://maurits.vanrees.org/preken
.. _church: http://www.herbergonderweg.nl
.. _`old version`: http://web.archive.org/web/20060425175532/http://maurits.vanrees.org/preken/

For this second blog, Quills seems like a reasonable choice as well.
But to really make it useful, I would want two extra fields for the
audio blog, that can contain links to those mp3 files. For the sermon
it could be even nicer to be able to upload this file via Quills and
have the link correctly made. But that may be added later. For now,
I am happy with just two extra links in a Quills WeblogEntry.

Extend with Zope 2

We could extend Quills in the Zope 2 way, but is that the best option here?

Brave New Content Type
----------------------

Alright, let's create a totally new content type, shall we? With
ArchGenXML_ we can easily create a basic new content type, let's call
it AudioEntry, that has the exact attributes that we need. Then
again, other people have already thought a lot about what a weblog
entry should contain and what it should do. We would introduce yet
another weblog application, which Plone does not really need and that
is currently only maintained by us. There are certainly occasions
where this is a splendid option, but not here.

.. _ArchGenXML: http://plone.org/products/archgenxml

The olden days of Zope 2
------------------------

Our second option is to extend Quills in the Zope 2 style: we create a
class AudioEntry that inherits from the Quills WeblogEntry. If our
class really offers something new, this is a fine choice. A good
example is RichDocument. This extends the Schema of ATDocument to add
a few attributes. The class inherits from ATDocument and so it uses
the functions already defined for that basic ATDocument and adds a few
functions that make sense for RichDocument. This is very well
documented in `Extending ATContentTypes`_ by Martin Aspeli.

.. _`Extending ATContentTypes`: http://plone.org/documentation/tutorial/richdocument/extending-atct

What would that mean for our use case of Extending Quills? AudioEntry
would inherit from WeblogEntry and extend it with two attributes for
the links. But it would not stop at that. For starters, a Quills
Weblog can currently only contain Folders and WeblogEntries. Adding
our AudioEntry there is not possible out of the box. Of course this
is changeable, but then we might also have to override the Weblog
class with our own AudioWeblog class. This can easily get out of hand
and result in a cascade of overrides. Also, when someone else has a
similar idea and introduces a VideoWeblogEntry, these two extension
products would be incompatible.

There are certainly use cases like RichDocument where this Zope 2 way
makes sense. But for our use case we simply want to add two links.
Is there no easier and less intrusive way to do this?

Zope 3: A New Hope

One general strategy and one Quills specific strategy

When we extend a product in Zope 3 style, we adapt an existing content
type without making a new content type. In our case this means that
we only need to adapt WeblogEntry, without the need for changing the
Weblog class. Also, we can arrange that our adaptations work for the
VideoWeblogEntry class as well, as long as that class plays nice and
implements the IWeblogEntry interface that Quills defines.

So already we can see that the Zope 3 way indeed brings new hope, in
that our changes can be much less intrusive and much more compatible
with other changes.

We will be using and introducing the following Zope 3 technologies:

* events

* utilities

* adapters

* annotations

General Strategy
----------------

We are going to make two products: one general product for extending
an object when it fulfils some condition, and one specific product for
extending a Quills WeblogEntry when it has audio content. The
strategy for the general product is this:

* An **event** is handled by an event handler, which uses a...

* **utility** to decide if an...

* **adapter** is necessary, in which case...

* **annotations** are added to the original object.

So: when an event takes place, we ask a utility if this event warrants
an adaptation of an object. The adaptation is implemented by adding
annotations to that object.

This is implemented by the **keywordannotator** product. And this
strategy probably has most people blinking their eyes and rereading
that list. :) Don't panic. You probably *do* want to reread it, but
if you do not fully understand it, you can just go to the next
section, which will translate this general idea into a specific
strategy for our use case. It should be much clearer then.

Quills Strategy
---------------

Do you remember the four Zope 3 technologies that we planned on using?
Events, utilities, adapters and annotations. This is how our specific
strategy uses those:

event
Someone adds a keyword `audio` to a WeblogEntry. When
this happens, Zope fires a so-called IObjectModified event. We will
register an event handler that springs into action when a
WeblogEntry (or actually any object that claims to implement the
IWeblogEntry interface) is modified or added.

utility
The code that handles this event, calls a utility. This utility
looks at the WeblogEntry and decides that the addition of this
keyword means that this entry now also implements the marker
interface IMaudio that we will define. If an object implements or
provides a 'normal' interface, then this object has all the
functions and attributes that are defined by that interface. A
marker interface has no functions or attributes. So it is basically
just a label: the object is marked as being IMaudio.

We also make sure that we can later add annotations to this entry by
letting it provide the standard IAttributeAnnotatable interface
defined by Zope 3.

adapter
We can now actually adapt (extend) this object that
implements IMaudio and add annotations to it. Adapting basically
does the same that inheritance (the Zope 2 way) does, but with a
different technique. You temporarily put a `wrapper` around
an object, do something to it (in our case add annotations) and then
remove the wrapper or just forget about it. Any other code (for
example the code in Quills itself) does not know or care that the
WeblogEntry object has been adapted or wrapped and now has something
extra. To that code, the WeblogEntry is still a normal WeblogEntry,
even though it now has annotations.

annotations
Annotations can be added in several ways, but most
used is the method associated with the IAttributeAnnotatable
interface that I mentioned above. In our case, two links are added
in a new hidden attribute of that WeblogEntry.

This is implemented by the **quadapter** (Quills Adapter) product,
which uses the keywordannotator product. In fact, this quadapter
product is added in the examples directory of keywordannotator. By the
way, both products have tests, so you can just run them and see for
yourself that all this actually works.

This section is rather important. If you understand what our goal and
our strategy is, you stand a good chance of understanding the next
sections which present the actual code. On the other hand, if you
prefer a *bottom up* approach, then reading the next sections may help
you in understanding this strategy.

Each of the next few sections first present the code from the general
keywordannotator product and then the code from the specific quadapter
product. Sometimes I find it hard myself to follow what
keywordannotator is actually doing, but reading the corresponding
quadapter code usually helps. :)

Event handler

zmcl and code for the event handlers

zcml registration

Okay, for the general keywordannotator product we want an event handler that gets activated whenever an object implementing the IAttributeAnnotatable interface gets modified or added. We register that in the configure.zcml file like this:

<subscriber
 for="zope.app.annotation.interfaces.IAttributeAnnotatable
      zope.app.event.interfaces.IObjectModifiedEvent"
 handler=".events.annotationEventHandler" />

That takes care of modified objects. Not shown here is the subscriber for added objects. It looks the same except that we replace 'IObjectModifiedEvent' with 'IObjectCreatedEvent'.

In the case of quadapter we want an event handler that gets activated only for object implementing the IWeblogEntry interface:

<subscriber
 for="Products.Quills.interfaces.IWeblogEntry
      zope.app.event.interfaces.IObjectModifiedEvent"
 handler="Products.keywordannotator.events.annotationEventHandler" />

Event handler code

In keywordannotator the code is this:

def annotationEventHandler(ob, event):
    from zope.component import getUtility
    decider = getUtility(IAnnotationDecider, context=ob)
    if decider.matchesKeywords(ob):
        decider.provideInterfaces(ob)

So this event handler looks for a utility. That utility tells us whether the object the event was fired for has a keyword that has been marked as special in our code or not. If this condition has been met, we instruct the utility to make sure that the object now provides a few more interfaces.

Note: quadapter simply uses this event handler from keywordannotator without overwriting it.

Now let's look at what that utility looks like.

Utility

zcml and code for the utilities

zcml registration
^^^^^^^^^^^^^^^^^

In keywordannotator it looks like this::

<utility
provides=".interfaces.IAnnotationDecider"
factory=".events.DefaultAnnotationDecider" />

This means that when some code wants to have a utility that provides
the IAnnotationDecider interface (the event handler code wants this,
see above), such a utility can be created by calling the
DefaultAnnotationDecider class in the events.py file in the
keywordannotator product.

The quadapter overrides this utility in overrides.zcml::

<utility
provides="Products.keywordannotator.interfaces.IAnnotationDecider"
factory="Products.quadapter.events.AudioDecider" />

Since this is an override, this means that the only known way to
create a utility that provides the IAnnotationDecider interface, is
now calling the AudioDecider class in the events.py file in the
quadapter product.

Utility code
^^^^^^^^^^^^

So what does that code look like? In keywordannotator it is this::

class DefaultAnnotationDecider(object):
implements(IAnnotationDecider)
keywords = KEYWORDS
ifaces = (IKeywordMatch,)
def matchesKeywords(self, object):
...
def provideInterfaces(self, object):
for iface in self.ifaces:
if not iface.providedBy(object):
alsoProvides(object, iface)

The implementation of the function 'matchesKeywords' is not
interesting here. It simply checks whether the keywords of the object
match one of the special keywords. By default, only the literal word
'special' is considered special.

The function provideInterfaces is more interesting. It makes sure
that a certain object provides all wanted interfaces. By default this
is the IKeywordMatch marker interface. In the next section we will
register an adapter for objects implementing that interface.

In quadapter the utility code is just four lines::

class AudioDecider(DefaultAnnotationDecider):
implements(IAnnotationDecider)
keywords = KEYWORDS
ifaces = PROVIDE_INTERFACES

It uses a different list of keywords that are special, and different
interfaces that need to be provided to objects that match one of the
keywords. These values are specified in the config.py file::

KEYWORDS = ['audio', 'preken']
PROVIDE_INTERFACES = (IMaudio, IAttributeAnnotatable,)

Note: 'preken' is the Dutch word for 'sermons'.

So, with just a few lines, the quadapter product changes the default
adapter so it reacts to different keywords and provides different
interfaces.

Note that a Quills WeblogEntry does not by default provide the
IAttributeAnnotatable interface, so we must instruct the utility to
make sure that WeblogEntries with one of the special keywords now
implement that interface as well. If wanted, we could add code to
quadapter that makes sure that *all* WeblogEntries also provide the
IAttributeAnnotatable interface. If you want that, look at the
utils.py file of keywordannotator.

At this point you may want to look back at Event handler code to
see how this utility is used by the event handler.

Current Status

We take a step back and look where we are.

We have covered a lot of ground already. So before we continue with
adapters and annotations, let's take a break and see what we have
accomplished so far.

If you are only using the keywordannotator product, the situation is
this:

* An object implementing the IAttributeAnnotatable interface,

* given the special keyword ('special'),

* now also implements the IKeywordMatch interface.

If next to keywordannotator you are also using Quills and the
quadapter product, then your situation is this:

* An object implementing the IWeblogEntry interface,

* given one of the special keywords ('audio' or 'preken'),

* now also implements the IMaudio interface

* and the IAttributeAnnotatable interface.

If this is not clear yet, then it is better to reread some of the
previous sections. If you *do* understand the current status, then it
is time for adapters and annotations.

Adapter

zcml and code for the adapters

zcml registration
^^^^^^^^^^^^^^^^^

Remember what we want to do: we want to adapt objects implementing a
certain interface by giving them annotations. In keywordannotator we
register an adapter for objects implementing the IKeywordMatch
interface. This adapter should provide such an object the new
IKeywordBasedAnnotations interface, which is just a marker interface
(or label). This adapter can be created by calling the
KeywordBasedAnnotations class in events.py. We register that adapter
in the configure.zcml file::

<adapter
for=".interfaces.IKeywordMatch"
provides=".interfaces.IKeywordBasedAnnotations"
factory=".events.KeywordBasedAnnotations"
/>

In quadapter we do something similar::

<adapter
for=".interfaces.IMaudio"
provides=".interfaces.IAudioAnnotations"
factory=".events.AudioAnnotations"
/>

This means that the adapter can be created by calling the
AudioAnnotations class in the events.py file of quadapter. This
adapter provides the IAudioAnnotations interface for objects that
already implement the IMaudio interface.

IAudioAnnotations is not a marker interface but a 'normal' interface,
if you want to make that distinction. In the interfaces.py file of
quadapter we state that any object that claims to implement the
IAudioAnnotations interface should have the attributes completeURL and
partURL::

class IAudioAnnotations(Interface):
"""Provide access to the audio annotations of an IMaudio object.
"""

completeURL = schema.URI(title=u'URL to complete audio content')
partURL = schema.URI(title=u'URL to a part of the audio content')

At this point it may be good to say that Zope 3 tends to use a lot of
interfaces. But you have probably already noticed that. :)

Adapter Code
^^^^^^^^^^^^

We have registered two adapters. Now how does the code look? In
keywordannotator::

class KeywordBasedAnnotations(object):
...
def __init__(self, context):
self.context = context
annotations = IAnnotations(self.context)
self._metadata = annotations.get(self._anno_key, None)
if self._metadata is None:
self._metadata = PersistentDict()
annotations[self._anno_key] = self._metadata

The __init__ function is the factory; in other words, it is the
function that creates a KeywordBasedAnnotations object based on
another object that is passed in via the 'context' parameter. This is
the spot where we first really see annotations. It is this line::

annotations = IAnnotations(self.context)

Here the context object is adapted to the IAnnotations interface and
the resulting wrapped or adapted object is stored in the annotations
variable. That variable is now basically a python dictionary which at
this point probably does not contain any values. The other lines make
sure that a basic structure for storing annotations is now available
in that object.

Now on to the adapter code in quadapter::

class AudioAnnotations(KeywordBasedAnnotations):
...
def __get_completeURL(self):
return self._metadata.get(COMP_ANNO)
def __set_completeURL(self, url):
self._metadata[COMP_ANNO] = url
completeURL = property(__get_completeURL, __set_completeURL)

This class subclasses the KeywordBasedAnnotations class from
keywordannotator, so it inherits the init function we saw above. But
the lines in this code do the stuff that we actually started this
product for: they add the completeURL property, where we can store the
link to the complete service of my church. This link is stored in the
self._metadata property, which is an annotation to this object.

The same is done (in some code that is not shown here as it is
basically the same) for the partURL property, where we can now store a
link to the partial service or more sane terms: the sermon.

Annotations

Adding and showing annotations

In the previous section we have actually already seen how annotations are added to an object and that they are basically just a python dictionary. But before this is possible, we need to add one line to the configure.zcml of keywordannotator:

<include package="zope.app.annotation" />

This directive loads another zcml directive which is in the zope software directory, in the file lib/python/zope/app/annotation/configure.zcml:

<adapter
    for=".interfaces.IAttributeAnnotatable"
    provides=".interfaces.IAnnotations"
    factory=".attribute.AttributeAnnotations"
    />

This actually concludes our strategy. All the code and configuration is now in place to add those annotations to a WeblogEntry by adapting it if it implements or provides the IMaudio interface, which is added to it by a utility if if decides that the condition is met after an event takes places that is handled by an event handler. If you understood what I just wrote, then you are very smart. :)

Adding Annotations

At this point, if you look at the state of things on the Plone level, we can add a normal WeblogEntry and give it a keyword 'audio'. Then our code makes sure that it provides the IMaudio interface. And then it stops! There is not yet a way to actually put something in the annotations. But we can arrange that. First we install a new product: CMFonFive.

That product may get assimilated into the core of CMF itself some day (Plone is based on CMF, the Content Management Framework). But for now this extra product is needed for the following lines that we add in the configure.zcml of quadapter:

<browser:menu
    id="object_tabs"
    title="Object tabs" />

<browser:menuItem
    for="Products.quadapter.interfaces.IMaudio"
    menu="object_tabs"
    title="Audio urls"
    action="maudio_edit"
    description="Edit form for audio urls"
    permission="zope2.ManageProperties"
    />

This adds a menu item in the Plone site to the object tabs of any object that implements the IMaudio interface. When you add this code, restart your Zope instance, and look at a WeblogEntry that has one of the special words and thus provides the IMaudio interface, you will see a tab that links to the action maudio_edit.

At that point we are almost done. We now need to make an edit form for that tab, that calls the code that sets the Annotations for this object. If you know how to make an edit form in html, then you should be able to make this yourself, possibly with the use of the Archetypes product.

Viewing Annotations

When you have added those links in the annotation of a WeblogEntry, you also want to be able to see them when you actually look at a WeblogEntry in your browser. We can use one more Zope 3 technique for this: the BrowserView. Essentially this also is an adapter. We need to define a BrowserView in quadapter:

<browser:page
    name="audio_entry_view"
    for="Products.quadapter.interfaces.IMaudio"
    permission="zope2.View"
    allowed_interface="Products.quadapter.interfaces.IAudioWeblogView"
    class=".browser.AudioWeblogView"
    />

This basically means that for an object providing the IMaudio interface we have a python class that gives us some functions to call in an html page template. In fact, we can now copy the file entry_macros.pt from Quills and add these lines at the right spot:

<tal:imaudio tal:define="view entry/@@audio_entry_view|nothing;"
           tal:condition="view">
  <metal:block use-macro="here/maudio_macros/macros/extratext" />
</tal:imaudio>

This uses a maudio_macros.pt file that gets the links from the object. Now when you view a normal WeblogEntry you just see the normal page that you would otherwise see. But when you view an entry with the IMaudio interface, for which you have added links with the edit form in the previous section, you will now see some extra text containing those links. The specifics are left as an exercise to the reader as they are just standard page template techniques, which should be familiar and which are not too interesting for this tutorial. If you do not get it working, contact me. Well, okay, I have just added them in the skins folder of quadapter.

Conclusions

Zope 3 is the way to go!

Using Zope 3 like this is:

* clean:

* Only one page template from Quills is changed and no code is
overwritten.

* If we added a metal:macro with a fill-slot to Quills, even that
single overwrite would not be necessary.

* compatible:

* We only *add* features, we do not change existing behaviour of
WeblogEntries.

* Our audio enhanced WeblogEntries are still first class
WeblogEntries. Existing code that expects a normal WeblogEntry
can handle our audio entries just fine.

* Our changes work on *any* content type that implements the
IWeblogEntry interface. So if someone really does create a
VideoWeblogEntry implementing IWeblogEntry, our code would work
for such an object as well.

In other words: Zope 3 is the place to be!