It is often useful to calculate a default value for a field. This value will be used on the add form, before the field is set.

To continue with our conference example, let's set the default values for the start and end dates to one week in the future and ten days in the future, respectively. We can do this by adding the following to program.py:

@form.default_value(field=IProgram['start'])
def startDefaultValue(data):
    # To get hold of the folder, do: context = data.context
    return datetime.datetime.today() + datetime.timedelta(7)


@form.default_value(field=IProgram['end'])
def endDefaultValue(data):
    # To get hold of the folder, do: context = data.context
    return datetime.datetime.today() + datetime.timedelta(10)

We also need to import datetime at the top of the file, of course.

Notice how the functions specify a particular schema field that they provide the default value for. The decorator will actually register these as "value adapters" for z3c.form, but you probably don't need to worry about that.

The data argument is an object that contains an attribute for each field in the schema. On the add form, most of these are likely to be None, but on a different form, the values may be populated from the context. The data object also has a context attribute that you can use to get the form's context. For add forms, that's the containing folder; for other forms, it is normally a content object being edited or displayed. If you need to look up tools (getToolByName) or acquire a value from a parent object, use data.context as the starting point, e.g.:

from Products.CMFCore.utils import getToolByName
...
catalog = getToolByName(data.context, 'portal_catalog')

The value returned by the method should be a value that's allowable for the field. In the case of Datetime fields, that's a Python datetime object.

It is possible to provide different default values depending on the type of context, a request layer, the type of form, or the type of widget used. See the plone.directives.form documentation for more details.

For example, if you wanted to have a differently calculated default for a particular form, you could use a decorator like:

@form.default_value(field=IProgram['start'], form=FormClass)

We'll cover creating custom forms later in this manual.

Many applications require some form of data entry validation. The simplest form of validation you get for free – the z3c.form library ensures that all data entered on Dexterity add and edit forms is valid for the field type.

It is also possible to set certain properties on the fields to add further validation (or even create your own fields with custom validation logic, although that is a lot less common). These properties are set as parameters to the field constructor when the schema interface is created. You should see the zope.schema package for details, but the most common constraints are:

• required=True/False, to make a field required or optional
• min and max, used for Int, Float, Datetime, Date, and Timedelta fields, specify the minimum and maximum (inclusive) allowed values of the given type
• min_length and max_length, used for collection fields (Tuple, List, Set, Frozenset, Dict) and text fields (Bytes, BytesLine, ASCII, ASCIILine, Text, TextLine), set the minimum and maximum (inclusive) length of a field

Constraints

If this does not suffice, you can pass your own constraint function to a field. The constraint function should take a single argument: the value that is to be validated. This will be of the field's type. The function should return a boolean True or False.

def checkForMagic(value):
    return 'magic' in value

Hint: The constraint function does not have access to the context, but if you need to acquire a tool, you can use the zope.app.component.hooks.getSite() method to obtain the site root

To use the constraint, pass the function as the constraint argument to the field constructor, e.g.:

    my_field = schema.TextLine(title=_(u"My field"), constraint=checkForMagic)

Constraints are easy to write, but do not necessarily produce very friendly error messages. It is however possible to customise these error messages using z3c.form error view snippets. See the z3c.form documentation for more details.

Invariants

You'll also notice that constraints only check a single field value. If you need to write a validator that compares multiple values, you can use an invariant. Invariants use exceptions to signal errors, which are displayed at the top of the form rather than next to a particular field.

To illustrate an invariant, let's make sure that the start date of a Program is before the end date. In program.py, we add the following. Code not relevant to this example is snipped with an ellipsis (...):

...

from zope.interface import invariant, Invalid

class StartBeforeEnd(Invalid):
    __doc__ = _(u"The start or end date is invalid")

class IProgram(form.Schema):
    
    ...
    
    start = schema.Datetime(
            title=_(u"Start date"),
            required=False,
        )

    end = schema.Datetime(
            title=_(u"End date"),
            required=False,
        )
    
    ...
    
    @invariant
    def validateStartEnd(data):
        if data.start is not None and data.end is not None:
            if data.start > data.end:
                raise StartBeforeEnd(_(u"The start date must be before the end date."))

...

Form validators

Finally, you can write more powerful validators by using the z3c.form widget validators. See the z3c.form documentation for details.

Vocabularies are normally used in conjunction with selection fields, and are supported by the zope.schema package, with widgets provided by z3c.form.

Selection fields use the Choice field type. To allow the user to select a single value, use a Choice field directly:

class IMySchema(form.Schema):
    myChoice = schema.Choice(...)

For a multi-select field, use a List, Tuple, Set or Frozenset with a Choice as the value_type:

class IMySchema(form.Schema):
    myList = schema.List(..., value_type=schema.Choice(...))

The choice field must be passed one of the following arguments:

• values can be used to give a list of static values
• source can be used to refer to an IContextSourceBinder or ISource instance
• vocabulary can be used to refer to an IVocabulary instance or (more commonly) a string giving the name of an IVocabularyFactory named utility.

In the remainder of this section, we will show the various techniques for defining vocabularies through several iterations of a new field added to the Program type allowing the user to pick the organiser responsible for the program.

Static vocabularies

Our first attempt uses a static list of organisers. We use the message factory to allow the labels (term titles) to be translated. The values stored in the organizer field will be a unicode string representing the chosen label, or None if no value is selected.

from zope.schema.vocabulary import SimpleVocabulary, SimpleTerm

organizers = SimpleVocabulary(
    [SimpleTerm(value=u'Bill', title=_(u'Bill')),
     SimpleTerm(value=u'Bob', title=_(u'Bob')),
     SimpleTerm(value=u'Jim', title=_(u'Jim'))]
    )

organizer = schema.Choice(
            title=_(u"Organiser"),
            vocabulary=organizers,
            required=False,
        )

Since required is False, there will be a "no value" option in the drop-down list.

Dynamic sources

The static vocabulary is obviously a bit limited. Not only is it hard-coded in Python, it also does not allow separation of the stored values and the labels shown in the selection widget.

We can make a one-off dynamic vocabulary using a context source binder. This is simply a callable (usually a function or an object with a __call__ method) that provides the IContextSourceBinder interface and takes a context parameter. The context argument is the context of the form (i.e. the folder on an add form, and the content object on an edit form). The callable should return a vocabulary, which is most easily achieved by using the SimpleVocabulary class from zope.schema. 

Here is an example using a function to return all users in a particular group.

from zope.schema.interfaces import IContextSourceBinder
from zope.schema.vocabulary import SimpleVocabulary
from Products.CMFCore.utils import getToolByName

@grok.provider(IContextSourceBinder)
def possibleOrganizers(context):
    acl_users = getToolByName(context, 'acl_users')
    group = acl_users.getGroupById('organizers')
    terms = []
    
    if group is not None:
        for member_id in group.getMemberIds():
            user = acl_users.getUserById(member_id)
            if user is not None:
                member_name = user.getProperty('fullname') or member_id
                terms.append(SimpleVocabulary.createTerm(member_id, str(member_id), member_name))
            
    return SimpleVocabulary(terms)

We use the PAS API to get the group and its members, building a list, which we then turn into a vocabulary.

When working with vocabularies, you'll come across some terminology that is worth explaining:

• A term is an entry in the vocabulary. The term has a value. Most terms are tokenised terms which also have a token, and some terms are titled, meaning they have a title that is different to the token.
• The token must be an ASCII string. It is the value passed with the request when the form is submitted. A token must uniquely identify a term.
• The value is the actual value stored on the object. This is not passed to the browser or used in the form. The value is often a unicode string, but can be any type of object.
• The title is a unicode string or translatable message. It is used in the form.

The SimpleVocabulary class contains two class methods that can be used to create vocabularies from lists:

• fromValues() takes a simple list of values and returns a tokenised vocabulary where the values are the items in the list, and the tokens are created by calling str() on the values.
• fromItems() takes a list of (token, value) tuples and creates a tokenised vocabulary with the token and value specified.

You can also instantiate a SimpleVocabulary yourself and pass a list of terms in the initialiser. The createTerm() class method can be used to create a term from a value, token and title. Only the value is required.

In the example above, we have chosen to create a SimpleVocabulary from terms with the user id used as value and token, and the user's full name as a title.

To use this context source binder, we use the source argument to the Choice constructor:

    organizer = schema.Choice(
            title=_(u"Organiser"),
            source=possibleOrganizers,
            required=False,
        )

Parameterised sources

We can improve this example by moving the group name out of the function, allowing it to be set on a per-field basis. To do so, we turn our IContextSourceBinder into a class that is initialised with the group name.

class GroupMembers(object):
    """Context source binder to provide a vocabulary of users in a given
    group.
    """
    
    grok.implements(IContextSourceBinder)
    
    def __init__(self, group_name):
        self.group_name = group_name
    
    def __call__(self, context):
        acl_users = getToolByName(context, 'acl_users')
        group = acl_users.getGroupById(self.group_name)
        terms = []
    
        if group is not None:
            for member_id in group.getMemberIds():
                user = acl_users.getUserById(member_id)
                if user is not None:
                    member_name = user.getProperty('fullname') or member_id
                    terms.append(SimpleVocabulary.createTerm(member_id, str(member_id), member_name))
            
        return SimpleVocabulary(terms)

Again, the source is set using the source argument to the Choice constructor:

    organizer = schema.Choice(
            title=_(u"Organiser"),
            source=GroupMembers('organizers'),
            required=False,
        )

When the schema is initialised on startup, the a GroupMembers object is instantiated, storing the desired group name. Each time the vocabulary is needed, this object will be called (i.e. the __call__() method is invoked) with the context as an argument, expected to return an appropriate vocabulary.

Named vocabularies

Context source binders are great for simple dynamic vocabularies. They are also re-usable, since you can import the source from a single location and use it in multiple instances.

Sometimes, however, we want to provide an additional level of decoupling, by using named vocabularies. These are similar to context source binders, but are components registered as named utilities, referenced in the schema by name only. This allows local overrides of the vocabulary via the Component Architecture, and makes it easier to distribute vocabularies in third party packages.

Note: Named vocabularies cannot be parameterised in the way as we did with the GroupMembers context source binder, since they are looked up by name only.

We can turn our first "members in the organizers group" vocabulary into a named vocabulary by creating a named utility providing IVocabularyFactory, like so:

from zope.schema.interfaces import IVocabularyFactory

...

class OrganizersVocabulary(object):
    grok.implements(IVocabularyFactory)
    
    def __call__(self, context):
        acl_users = getToolByName(context, 'acl_users')
        group = acl_users.getGroupById('organizers')
        terms = []
    
        if group is not None:
            for member_id in group.getMemberIds():
                user = acl_users.getUserById(member_id)
                if user is not None:
                    member_name = user.getProperty('fullname') or member_id
                    terms.append(SimpleVocabulary.createTerm(member_id, str(member_id), member_name))
            
        return SimpleVocabulary(terms)

grok.global_utility(OrganizersVocabulary, name=u"example.conference.Organizers")

By convention, the vocabulary name is prefixed with the package name, to ensure uniqueness.

We can make use of this vocabulary in any schema by passing its name to the vocabulary argument of the Choice field constructor:

    organizer = schema.Choice(
            title=_(u"Organiser"),
            vocabulary=u"example.conference.Organizers",
            required=False,
        )

Some common vocabularies

As you might expect, there are a number of standard vocabularies that come with Plone. These are found in the plone.app.vocabularies package. Some of the more useful ones include

• plone.app.vocabularies.AvailableContentLanguages, a list of all available content languages
• plone.app.vocabularies.SupportedContentLanguages, a list of currently supported content languages
• plone.app.vocabularies.Roles, the user roles available in the site
• plone.app.vocabularies.PortalTypes, a list of types installed in portal_types
• plone.app.vocabularies.ReallyUserFriendlyTypes, a list of those types that are likely to mean something to users
• plone.app.vocabularies.Workflows, a list of workflows
• plone.app.vocabularies.WorkflowStates, a list of all states from all workflows
• plone.app.vocabularies.WorkflowTransitions, a list of all transitions from all workflows

In addition, the package plone.principalsource provides several vocabularies that are useful for selecting users and groups in a Dexterity context:

• plone.principalsource.Users provides users
• plone.principalsource.Groups provides groups
• plone.principalsource.Principals provides security principals (users or groups)

Importantly, these sources are not iterable, which means that you cannot use them to provide a list of all users in the site. This is intentional: calculating this list can be extremely expensive if you have a large site with many users, especially if you are connecting to LDAP or Active Directory. Instead, you should use a search-based source such as one of these.

We will use one of these together with an auto-complete widget to finalise our organizer field. To do so, we need to add plone.principalsource as a dependency of example.conference. In setup.py, we add:

    install_requires=[
          ...
          'plone.principalsource',
      ],

Since we use an <includeDependencies /> line in configure.zcml, we do not need a separate <include /> line in configure.zcml for this new dependency.

The organizer field now looks like:

    organizer = schema.Choice(
            title=_(u"Organiser"),
            vocabulary=u"plone.principalsource.Users",
            required=False,
        )

The autocomplete selection widget

The organizer field now has a query-based source. The standard selection widget (a drop-down list) is not capable of rendering such a source. Instead, we need to use a more powerful widget. For a basic widget, see z3c.formwidget.query, but in a Plone context, you will more likely want to use plone.formwidget.autocomplete, which extends z3c.formwidget.query to provide friendlier user interface.

The widget is provided with plone.app.dexterity, so we do not need to configure it ourselves. We only need to tell Dexterity to use this widget instead of the default, using a form widget hint as shown earlier. At the top of program.py, we add the following import:

from plone.formwidget.autocomplete import AutocompleteFieldWidget

If we were using a multi-valued field, such as a List with a Choice value_type, we would use the AutocompleteMultiFieldWidget instead

In the IProgram schema (which, recall, derives from form.Schema and is therefore processed for form hints at startup), we then add the following:

    form.widget(organizer=AutocompleteFieldWidget)
    organizer = schema.Choice(
            title=_(u"Organiser"),
            vocabulary=u"plone.principalsource.Users",
            required=False,
        )

You should now see a dynamic auto-complete widget on the form, so long as you have JavaScript enabled. Start typing a user name and see what happens. The widget also has fall-back for non-JavaScript capable browsers.

References are a way to maintain links between content that remain valid even if one or both content items are moved or renamed.

Under the hood, Dexterity's reference system uses five.intid, a Zope 2 integration layer for zope.intid, to give each content item a unique integer id. These are the basis for relationships maintained with the zc.relationship package, which in turn is accessed via an API provided by z3c.relationfield, integrated into Zope 2 with plone.app.relationfield. For most purposes, you need only to worry about the z3c.relationfield API, which provides methods for finding source and target objects for references and searching the relationship catalog.

References are most commonly used in form fields with a selection or content browser widget. Dexterity comes with a standard widget in plone.formwidget.contenttree configured for the RelationList and RelationChoice fields from z3c.relationfield.

To illustrate the use of references, we will allow the user to create a link between a Session and its Presenter. Since Dexterity already ships with and installs plone.formwidget.contenttree and z3c.relationfield, we do not need to add any further setup code, and we can use the field directly in session.py:

...

from z3c.relationfield.schema import RelationChoice
from plone.formwidget.contenttree import ObjPathSourceBinder

...

from example.conference.presenter import IPresenter

class ISession(form.Schema):
    """A conference session. Sessions are managed inside Programs.
    """
    
    ...
        
    presenter = RelationChoice(
            title=_(u"Presenter"),
            source=ObjPathSourceBinder(object_provides=IPresenter.__identifier__),
            required=False,
        )

To allow multiple items to be selected, we could have used a RelationList like:

    relatedItems = RelationList(
            title=u"Related Items",
            default=[],
            value_type=RelationChoice(title=_(u"Related"),
                                      source=ObjPathSourceBinder()),
            required=False,
        )

The ObjPathSourceBinder class is an IContextSourceBinder that returns a vocabulary with content objects as values, object titles as term titles and object paths as tokens.

You can pass keyword arguments to the constructor for ObjPathSourceBinder() to restrict the selectable objects. Here, we demand that the object must provide the IPresenter interface. The syntax is the same as that used in a catalog search, except that only simple values and lists are allowed (e.g. you can't use a dict to specify a range or values for a field index).

If you want to restrict the folders and other content shown in the content browser, you can pass a dictionary with catalog search parameters (and here, any valid catalog query will do) as the first non-keyword argument (navigation_tree_query) to the ObjPathSourceBinder() constructor.

If you want to use a different widget, you can use the same source (or a custom source that has content objects as values) with something like the autocomplete widget. The following line added to the interface will make the presenter selection similar to the organizer selection widget we showed in the previous section:

    form.widget(presenter=AutocompleteFieldWidget)

Once the user has created some relationships, the value stored in the relation field is a RelationValue object. This provides various attributes, including:

• from_object, the object from which the relationship is made
• to_object, the object to which the relationship is made
• from_id and to_id, the integer ids of the source and target
• from_path and to_path, the path of the source and target

The isBroken() method can be used to determine if the relationship is broken. This normally happens if the target object is deleted.

To display the relationship on our form, we can either use a display widget on a DisplayForm, or use this API to find the object and display it. We'll do the latter in session_templates/view.pt:

        <div tal:condition="context/presenter">
            <label i18n:translate="presenter">Presenter:</label>
            <span tal:content="context/presenter/to_object/Title | nothing" />
        </div>

Many content items need to allow users to provide rich text in some kind of markup, be that HTML (perhaps entered using a WYSIWYG editor), reStructuredText, Markdown or some other format. This markup typically needs to be transformed into HTML for the view template, but we also want to keep track of the original "raw" markup so that it can be edited again. Even when the input format is HTML, there is often a need for a transformation to tidy up the HTML and strip out tags that are not permitted.

It is possible to store HTML in a standard Text field. You can even get a WYSIWYG widget, by using a schema such as this:

from plone.directives import form
from zope import schema
from plone.app.z3cform.wysiwyg import WysiwygFieldWidget

class ITestSchema(form.Schema):

    form.widget(body=WysiwygFieldWidget)
    body = schema.Text(title=u"Body text")

However, this approach does not allow for alternative markups or any form of content filtering. For that, we need to use a more powerful field: RichText from the plone.app.textfield package.

from plone.directives import form
from plone.app.textfield import RichText

class ITestSchema(form.Schema):

    body = RichText(title=u"Body text")

The RichText field constructor can take the following arguments in addition to the usual arguments for a Text field:

• default_mime_type, a string representing the default MIME type of the input markup. This defaults to text/html.
  
• output_mime_type, a string representing the default output MIME type. This defaults to text/x-html-safe, which is a Plone-specific MIME type that disallows certain tags. Use the HTML Filtering control panel in Plone to control the tags.
• allowed_mime_types, a tuple of strings giving a vocabulary of allowed input MIME types. If this is None (the default), the allowable types will be restricted to those set in Plone's Markup control panel.

Also note: The default field can be set to either a unicode string (in which case it will be assumed to be a string of the default MIME type) or a RichTextValue object (see below).

Below is an example of a field allow StructuredText and reStructuredText, transformed to HTML by default.

from plone.directives import form
from plone.app.textfield import RichText

defaultBody = """\
Background
==========

Please fill this in

Details
=======

And this
"""

class ITestSchema(form.Schema):

    body = RichText(
            title=u"Body text",
            default_mime_type='text/x-rst',
            output_mime_type='text/x-html',
            allowed_mime_types=('text/x-rst', 'text/structured',),
            default=defaultBody,
        )

The RichTextValue

The RichText field does not store a string. Instead, it stores a RichTextValue object. This is an immutable object that has the following properties:

• raw, a unicode string with the original input markup
• mimeType, the MIME type of the original markup, e.g. text/html or text/structured.
  
• encoding, the default character encoding used when transforming the input markup. Most likely, this will be utf-8
• raw_encoded, the raw input encoded in the given encoding
• outputMimeType, the MIME type of the default output, taken from the field at the time of instantiation
• output, a unicode string representing the transformed output. If possible, this is cached persistently until the RichTextValue is replaced with a new one (as happens when an edit form is saved, for example).
  

The storage of the RichTextValue object is optimised for the case where the transformed output will be read frequently (i.e. on the view screen of the content object) and the raw value will be read infrequently (i.e. on the edit screen). Because the output value is cached indefinitely, you will need to replace the RichTextValue object with a new one if any of the transformation parameters change. However, as we will see below, it is possible to apply a different transformation on demand should you need to.

The code snippet belows shows how a RichTextValue object can be constructed in code. In this case, we have a raw input string of type text/plain that will be transformed to a default output of text/html. (Note that we would normally look up the default output type from the field instance.)

from plone.app.textfield.value import RichTextValue

...

context.body = RichTextValue(u"Some input text", 'text/plain', 'text/html')

Of course, the standard widget used for a RichText field will correctly store this type of object for you, so it is rarely necessary to create one yourself.

Using rich text fields in templates

What about using the text field in a template? If you are using a DisplayForm, the display widget for the RichText field will render the transformed output markup automatically. If you are writing TAL manually, you may try something like this:

<div tal:content="structure context/body" />

This, however, will render a string like:

RichTextValue object. (Did you mean <attribute>.raw or <attribute>.output?)

The correct syntax is:

<div tal:content="structure context/body/output" />

This will rendred the cached, transformed output. This operation is approximately as efficient as rendering a simple Text field, since the transformation is only applied once, when the value is first saved.

Alternative transformations

Sometimes, you may want to invoke alternative transformations. Under the hood, the default implementation uses the portal_transforms tool to calculate a transform chain from the raw value's input MIME type to the desired output MIME type. (Should you need to write your own transforms, take a look at this tutorial.) This is abstracted behind an ITransformer adapter to allow alternative implementations.

To invoke a transformation in code, you can use the following syntax:

from plone.app.textfield.interfaces import ITransformer

transformer = ITransformer(context)
transformedValue = transformer(context.body, 'text/plain')

The __call__() method of the ITransformer adapter takes a RichTextValue object and an output MIME type as parameters.

If you are writing a page template, there is an even more convenient syntax:

<div tal:content="structure context/@@text-transform/body/text/plain" />

The first traversal name gives the name of the field on the context (body in this case). The second and third give the output MIME type. If the MIME type is omitted, the default output MIME type will be used.

Note: Unlike the output property, the value is not cached, and so will be calculated each time the page is rendered.

Plone has dedicated File and Image types, and it is often preferable to use these for managing files and images. However, it is sometimes useful to treat binary data as fields on an object. When working with Dexterity, you can accomplish this by using plone.namedfile and plone.formwidget.namedfile.

The plone.namedfile package includes four field types, all found in the plone.namedfile.field module:

• NamedFile stores non-BLOB files. This is useful for small files when you don't want to configure BLOB storage.
• NamedImage stores non-BLOB images.
• NamedBlobFile stores BLOB files (see note below). It is otherwise identical to NamedFile.
• NamedBlobImage stores BLOB images (see note below). It is otherwise identical to NamedImage.

Note that NamedBlobFile and NamedBlobImage depends on z3c.blobfile. This dependency is specified via the [blobs] extra.

If you do not have z3c.blobfile in your buildout (most likely because you did not depend on the [blobs] extra for plone.namedfile), the NamedBlobFile and NamedBlobImage field and value types will not be importable. They are only defined if BLOB support is detected.

In use, the four field types are all pretty similar. They actually store persistent objects of type plone.namedfile.NamedFile, plone.namedfile.NamedImage, plone.namedfile.NamedBlobFile (if available) and plone.namedfile.NamedBlobImage (if available), respectively. Note the different module! These objects have attributes like data, to access the raw binary data, contentType, to get a MIME type, and filename, to get the original filename. The image values also support _height and _width to get image dimensions.

To use the non-BLOB image and file fields, it is sufficient to depend on plone.formwidget.namedfile, since this includes plone.namefile as a dependency. We prefer to be explicit in setup.py, however, since we will actually import directly from plone.namedfile:

      install_requires=[
          ...
          'plone.namedfile',
          'plone.formwidget.namedfile',
      ],

To use the [blobs] extra, we would need the following line instead of the plone.namedfile line instead:

          'plone.namedfile[blobs]',

Again, we do not need separate <include /> lines in configure.zcml for these new dependencies, because we use <includeDependencies />.

For the sake of illustration, we will add a (non-BLOB) image of the speaker to the Presenter type. In presenter.py, we add:

from plone.namedfile.field import NamedImage

class IPresenter(form.Schema):
    ...
    
    picture = NamedImage(
            title=_(u"Please upload an image"),
            required=False,
        )

To use this in a view, we can either use a display widget via a DisplayForm, or construct a download URL manually. Since we don't have a DisplayForm for the Presenter type, we'll do the latter (of course, we could easily turn the view into a display form as well).

In presenter_templates/view.pt, we add this block of TAL:

        <div tal:define="picture nocall:context/picture"
             tal:condition="nocall:picture">
            <img tal:attributes="src string:${context/absolute_url}/@@download/picture/${picture/filename};
                                 height picture/_height | nothing;
                                 width picture/_width | nothing;"
                />
        </div>

This constructs an image URL using the @@download view from plone.namedfile. This view takes the name of the field containing the file or image on the traversal subpath (/picture), and optionally a filename on a further sub-path. The filename is used mainly so that the URL ends in the correct extension, which can help ensure web browsers display the picture correctly. We also define the height and width of the image based on the values set on the object.

For file fields, you can construct a download URL in a similar way, using an <a /> tag, e.g.:

<a tal:attributes="href string:${context/absolute_url}/@@download/some_field/${context/some_field/filename}" />

Earlier in this manual, we have seen how to create views, and how to use file and image fields. These are all dynamic, however, and often we just want to ship with a static image/icon, CSS or JavaScript file. For this, we need to register static resources.

Registering a static resource directory

The easiest way to manage static resources is to make use of the static resource directory feature in five.grok. Simply add a directory called static in the package and make sure that the <grok:grok package="." /> line appears in configure.zcml.

If a static resource directory in the example.conference package contains a file called conference.css, it will be accessible on a URL like http://<server>/site/++resource++example.conference/conference.css. The resource name is the same as the package name wherein the static directory appears.

If you need to register additional directories, you can do so using the <browser:resourceDirectory /> ZCML directive. This requires two attributes: name is the name that appears after the ++resource++ namespace; directory is a relative path to the directory containing resources.

Importing CSS and JavaScript files in templates

One common use of static resources is to add a static CSS or JavaScript file to a specific template. We can do this by filling the style_slot or javascript_slot in Plone's main_template in our own view template and using an appropriate resource link.

For example, we could add the following near the top of presenter_templates/view.pt:

<head>
    <metal:block fill-slot="style_slot">
        <link rel="stylesheet" type="text/css" 
            tal:define="navroot context/@@plone_portal_state/navigation_root_url"
            tal:attributes="href string:${navroot}/++resource++example.conference/conference.css"
            />
    </metal:block>
</head>

Always create the resource URL relative to the navigation root as shown here, so that the URL is the same for all content objects using this view. This allows for efficient resource caching.

Registering resources with Plone's resource registries

Sometimes it is more appropriate to register a stylesheet with Plone's portal_css registry (or a JavaScript file with portal_javascripts), rather than add the registration on a per-template basis. This ensures that the resource is available site-wide.

It may seem wasteful to include a resource that is not be used on all pages in the global registry. Remember, however, that portal_css and portal_javascripts will merge and compress resources, and set caching headers such that browsers and caching proxies can cache resources well. It is often more effective to have one slightly larger file that caches well, than to have a variable number of files that may need to be loaded at different times.

To add a static resource file, you can use the GenericSetup cssregistry.xml or jsregistry.xml import steps in the profiles/default directory. For example, an import step to add the conference.css file site-wide may involve a cssregistry.xml file that looks like this:

<?xml version="1.0"?>
<object name="portal_css">
 <stylesheet id="++resource++example.conference/conference.css"
    title="" cacheable="True" compression="safe" cookable="True"
    enabled="1" expression="" media="screen" rel="stylesheet" rendering="import"
    />
</object>

Similarly, a JavaScript resource could be imported with a jsregistry.xml like:

<?xml version="1.0"?>
<object name="portal_javascripts">
 <javascript cacheable="True" compression="none" cookable="True"
    enabled="False" expression=""
    id="++resource++example.conference/conference.js" inline="False"/>
</object>

Image resources

Images can be added to resource directories just like any other type of resource. To use the image in a view, you can construct an <img /> tag like this:

        <img style="float: left; margin-right: 2px; margin-top: 2px"
             tal:define="navroot context/@@plone_portal_state/navigation_root_url"
             tal:attributes="src string:${navroot}/++resource++example.conference/program.gif"
             />

Content type icons

Finally, to use an image resource as the icon for a content type, simply list it in the FTI under the content_icon property. For example, in profiles/default/types/example.conference.presenter.xml, we can use the following line, presuming we have a presenter.gif in the static directory:

 <property name="content_icon">++resource++example.conference/presenter.gif</property>

Dexterity introduces the concept of behaviors – re-usable bundles of functionality and/or form fields which can be turned on or off on a per-type basis.

Each behavior has a unique interface. When a behavior is enabled on a type, you will be able to adapt that type to the behavior's interface. If the behavior is disabled, the adaptation will fail. The behavior interface can also be marked as an IFormFieldsProvider, in which case it will add fields to the standard add and edit forms. Finally, a behavior may imply a sub-type: a marker interface which will be dynamically provided by instances of the type for which the behavior is enabled.

We will not cover writing new behaviors in this manual, but we will show how to enable behaviors on a type. In fact, we've already seen one standard behavior applied to our example types, registered in the FTI and imported using GenericSetup:

 <property name="behaviors">
     <element value="plone.app.content.interfaces.INameFromTitle" />
 </property>

Other behaviors are added in the same way, by listing additional behavior interfaces as elements of the behaviors property.

Behaviors are normally registered with the <plone:behavior /> ZCML directive. When registered, a behavior will create a global utility providing IBehavior, which is used to provide some metadata, such as a title and description for the behavior. 

You can find and apply behaviors via the Dexterity Content Types control panel that is installed with plone.app.dexterity. For a list of standard behaviors that ship with Dexterity, see the reference at the end of this manual.

So far, we have mainly been concerned with content types' schemata and forms created from these. However, we often want to add more dynamic functionality, reacting when something happens to objects of our type. In Zope, that usually means writing event subscribers.

Zope's event model is synchronous. When an event is broadcast (via the notify() function from the zope.event package), for example from the save action of an add form, all registered event handlers will be called. There is no guarantee of which order the event handlers will be called in, however.

Each event is described by an interface, and will typically carry some information about the event. Some events are known as object events, and provide zope.component.interfaces.IObjectEvent. These have an object attribute giving access to the (content) object that the event relates to. Object events allow event handlers to be registered for a specific type of object as well as a specific type of event.

Some of the most commonly used event types in Plone are shown below. They are all object events.

• zope.lifecycleevent.interfaces.IObjectCreatedEvent, fired by the standard add form just after an object has been created, but before it has been added on the container. Note that it is often easier to write a handler for IObjectAddedEvent (see below), because at this point the object has a proper acquisition context.
• zope.lifecycleevent.interfaces.IObjectModifiedEvent, fired by the standard edit form when an object has been modified
• zope.app.container.interfaces.IObjectAddedEvent, fired when an object has been added to its container. The container is available as the newParent attribute, and the name the new item holds in the container is available as newName.
• zope.app.container.interfaces.IObjectRemovedEvent, fired when an object has been removed from its container. The container is available as the oldParent attribute, and the name the item held in the container is available as oldName.
• zope.app.container.interfaces.IObjectMovedEvent, fired when an object is added to, removed from, renamed in, or moved between containers. This event is a super-type of IObjectAddedEvent and IObjectRemovedEvent, shown above, so an event handler registered for this interface will be invoked for the 'added' and 'removed' cases as well. When an object is moved or renamed, all of oldParent, newParent, oldName and newName will be set.
• Products.CMFCore.interfaces.IActionSucceededEvent, fired when a workflow event has completed. The workflow attribute holds the workflow instance involved, and the action attribute holds the action (transition) invoked.

Event handlers can be registered using ZCML with the <subscriber /> directive, but when working with Dexterity types, we'll more commonly use the grok.subscriber() in Python code.

As an example, let's add an event handler to the Presenter type that tries to find users with matching names matching the presenter id, and send these users an email.

First, we require a few additional imports at the top of presenter.py:

from zope.app.container.interfaces import IObjectAddedEvent
from Products.CMFCore.utils import getToolByName

Then, we'll add the following event subscriber after the schema definition:

@grok.subscribe(IPresenter, IObjectAddedEvent)
def notifyUser(presenter, event):
    acl_users = getToolByName(presenter, 'acl_users')
    mail_host = getToolByName(presenter, 'MailHost')
    portal_url = getToolByName(presenter, 'portal_url')
    
    portal = portal_url.getPortalObject()
    sender = portal.getProperty('email_from_address')
    
    if not sender:
        return
    
    subject = "Is this you?"
    message = "A presenter called %s was added here %s" % (presenter.title, presenter.absolute_url(),)
    
    matching_users = acl_users.searchUsers(fullname=presenter.title)
    for user_info in matching_users:
        email = user_info.get('email', None)
        if email is not None:
            mail_host.secureSend(message, email, sender, subject)

There are many ways to improve this rather simplistic event handler, but it illustrates how events can be used. The first argument to grok.subscribe() is an interface describing the object type. For non-object events, this is omitted. The second arugment is the event type. The arguments to the function reflects these two, so the first argument is the IPresenter instance and the second is an IObjectAddedEvent instance.

Plone's security system is based the concept of permissions protecting operations (like accessing a view, viewing a field, modifying a field, or adding a type of content) that are granted to roles, which in turn are granted to users and/or groups. In the context of developing content types, permissions are typically used in three different ways:

• A content type or group of related content types often has a custom add permission which controls who can add this type of content.
• Views (including forms) are sometimes protected by custom permissions.
• Individual fields are sometimes protected by permissions, so that some users can view and edit fields that others can't see.

It is easy to create new permissions. However, be aware that it is considered good practice to use the standard permissions wherever possible and use workflow to control which roles are granted these permissions on a per-instance basis. We'll cover workflow later in this manual.

Standard permissions

The standard permissions can be found in Product.Five's permissions.zcml (parts/omelette/Products/Five/permissions.zcml). Here, you will find a short id (also known as the Zope 3 permission id) and a longer title (also known as the Zope 2 permission title). For historical reasons, some areas in Plone use the id, whilst others use the title. As a rule of thumb:

• Browser views defined in ZCML or protected via a grok.require() directive use the Zope 3 permission id
• Security checks using zope.security.checkPermission() use the Zope 3 permission id
• Dexterity's add_permission FTI variable uses the Zope 3 permission id.
• The rolemap.xml GenericSetup handler and workflows use the Zope 2 permission title.
• Security checks using AccessControl's getSecurityManager().checkPermission(), including the methods on the portal_membership tool, use the Zope 2 permission title.

The most commonly used permission are shown below. The Zope 2 permission title is shown in parentheses.

• zope2.View (View) is used to control access to the standard view of a content item
• zope2.DeleteObjects (Delete objects) is used to control the ability to delete child objects in a container
• cmf.ModifyPortalContent (Modify portal content) is used to control write access to content items
• cmf.ManagePortal (Manage portal) is used to control access to management screens
• cmf.AddPortalContent (Add portal content) is the standard add permission required to add content to a folder
• cmf.SetOwnProperties (Set own properties) is used to allow users to set their own member properties
• cmf.RequestReview (Request Review) is typically used as a workflow transition guard to allow users to submit content for review
• cmf.ReviewPortalContent (Review portal content) is usually granted to the Reviewer role, controlling the ability to publish or reject content

Standard roles

As with permissions, it is easy to create custom roles (use the rolemap.xml GenericSetup import step - see CMFPlone's version of this file for an example), although you should use the standard roles where possible.

The standard roles in Plone are:

• Anonymous, a pseudo-role that represents non-logged in users.

Note that if a permission is granted to Anonymous, it is effectively granted to everyone. It is not possible to grant permissions to non-logged in users without also granting them to logged in ones.

• Authenticated, a pseudo-role that represents logged-in users.
• Owner, which is automatically granted to the creator of an object.
• Manager, which represents super-users/administrators. Almost all permissions that are not granted to Anonymous are granted to Manager.
• Reviewer, which represents content reviewers separately from site administrators. It is possible to grant the Reviewer role locally on the Sharing tab, where it is shown as Can review.
• Member, representing "standard" Plone users

In addition, there are three roles that are intended to be used as local roles only. These are granted to specific users or groups via the Sharing tab, where they appear under more user friendly pseudonyms.

• Reader, aka Can view, confers the right to view content. As a role of thumb, the Reader role should have the View and Access contents information permissions if the Owner roles does.
• Editor, aka Can edit, confers the right to edit content. As a role of thumb, the Editor role should have the Modify portal content permission if the Owner roles does.
• Contributor, aka Can add, confers the right to add new content. As a role of thumb, the Contributor role should have the Add portal content permission and any type-specific add permissions globally (i.e. granted in rolemap.xml), although these permissions are sometimes managed in workflow as well.

Performing permission checks in code

It is sometimes necessary to check permissions explicitly in code, for example in a view. A permission check always checks a permission on a context object, since permissions can change with workflow.

Never make security dependent on users' roles directly. Always check for a permission, and assign the permission to the appropriate role or roles.

As an example, let's display a message on the view of a Session type if the user has the cmf.RequestReview permission. In session.py, we update the View class with the following:

from zope.security import checkPermission

class View(dexterity.DisplayForm):
    grok.context(ISession)
    grok.require('zope2.View')
    
    def canRequestReview(self):
        return checkPermission('cmf.RequestReview', self.context)

And in the session_templates/view.pt template, we add:

        <div class="discreet" tal:condition="view/canRequestReview" i18n:translate="suggest_review">
            Please submit this for review.
        </div>

Creating custom permissions

Although the standard permissions should be used to control basic operations (view, modify, delete, review), it is sometimes useful to create new permissions. Combined with custom workflows, custom permissions can be used to create highly tailored content review cycles and data entry applications. They are also an important way to control who can add what content.

The easiest way to create a custom permission is with the help of the collective.autopermission package, which allows permissions to be defined using the <permission /> ZCML statement.

collective.autopermission is obsolete in Zope 2.12, where its functionality has been merged into Zope itself

As an example, let's create some custom permissions for use with the Session type. We'll create a new add permission, so that we can let any member submit a session to a program, and a permission which we will later use to let reviewers edit some specific fields on the Session type.

First, we need to depend on collective.autopermission. In setup.py:

      install_requires=[
          ...
          'collective.autopermission',
      ],

Make sure collective.autopermission's configuration is included before any custom permissions are defined.  In our case, the <includeDependencies /> line takes care of this.

Next, we'll create a file called permissions.zcml to hold the permissions (we could also place them directly into configure.zcml). We need to include this in configure.zcml, just after the <includeDependencies /> line.

    <include file="permissions.zcml" />

Note: All permissions need to be defined before the <grok:grok package="." /> line in configure.zcml. Otherwise, you may get errors trying to use the permission with a grok.require() directive.

The permissions.zcml file looks like this:

<configure
    xmlns="http://namespaces.zope.org/zope"
    i18n_domain="example.conference">

    <permission
        id="example.conference.AddSession"
        title="example.conference: Add session"
        />

    <permission
        id="example.conference.ModifyTrack"
        title="example.conference: Modify track"
        />
        
</configure>

New permissions are granted to the Manager role only by default. To set a different default, we can use the rolemap.xml GenericSetup import step, which maps permissions to roles at the site root.

In profiles/default/rolemap.xml, we have the following:

<?xml version="1.0"?>
<rolemap>
  <permissions>
    <permission name="example.conference: Add session" acquire="True">
      <role name="Owner"/>
      <role name="Manager"/>
      <role name="Member"/>
      <role name="Contributor"/>
    </permission>
    <permission name="example.conference: Modify track" acquire="True">
      <role name="Manager"/>
      <role name="Reviewer"/>
    </permission>
  </permissions>
</rolemap>

This file uses the Zope 2 permission title instead of the shorter Zope 3 permission id

Content type add permissions 

Dexterity content types' add permissions are set in the FTI, using the add_permission property. This can be changed through the web or in the GenericSetup import step for the content type.

To make the Session type use our new permission, we modify the add_permission line in profiles/default/example.conference.session.xml:

 <property name="add_permission">example.conference.AddSession</property>

Protecting views and forms

Access to views and other browser resources (like viewlets or portlets) can be protected by permissions, either using the permission attribute on ZCML statements like <browser:page /> or using the grok.require() directive.

We have already seen this directive on our views:

class View(grok.View):
    grok.context(IPresenter)
    grok.require('zope2.View')

We could use a custom permission name as the argument to grok.require(). We could also use the special zope.Public permission name to make the view accessible to anyone.

Protecting form fields

Individual fields in a schema may be associated with a read permission and a write permission. The read permission is used to control access to the field's value via protected code (e.g. scripts or templates created through the web) and URL traversal, and can be used to control the appearance of fields when using display forms (if you use custom views that access the attribute directly, you'll need to perform your own checks). Write permissions can be used to control whether or not a given field appears on a type's add and edit forms.

In both cases, read and write permissions are annotated onto the schema using directives similar to those we've already seen for form widget hints. The read_permission() and write_permission() directives are found in the plone.directives.dexterity package.

As an example, let's add a field for Session reviewers to record the track for a session. We'll store the vocabulary of available tracks on the parent Program object in a text field, so that the creator of the Program can choose the available tracks.

First, we add this to the IProgram schema in program.py:

    form.widget(tracks=TextLinesFieldWidget)
    tracks = schema.List(
            title=_(u"Tracks"),
            required=True,
            default=[],
            value_type=schema.TextLine(),
        )

The TextLinesFieldWidget is used to edit a list of text lines in a text area. It is imported as:

from plone.z3cform.textlines.textlines import TextLinesFieldWidget

Next, we'll add a vocabulary for this to session.py:

from Acquisition import aq_inner, aq_parent
from zope.schema.interfaces import IContextSourceBinder
from zope.schema.vocabulary import SimpleVocabulary

...


@grok.provider(IContextSourceBinder)
def possibleTracks(context):
    
    # we put the import here to avoid a circular import
    from example.conference.program import IProgram
    while context is not None and not IProgram.providedBy(context):
        context = aq_parent(aq_inner(context))
    
    values = []
    if context is not None and context.tracks:
        values = context.tracks
    
    return SimpleVocabulary.fromValues(values)

This vocabulary finds the closest IProgram (in the add form, the context will be the Program, but on the edit form, it will be the Session, so we need to check the parent) and uses its tracks variable as the vocabulary.

Next, we add a field to the ISession interface in the same file and protect it with the relevant write permission:

    dexterity.write_permission(track='example.conference.ModifyTrack')
    track = schema.Choice(
            title=_(u"Track"),
            source=possibleTracks,
            required=False,
        )

The dexterity module is the root of the plone.directives.dexterity package, imported as:

from plone.directives import dexterity

With this in place, users with the example.conference: Modify track permission should be able to edit tracks for a session. For everyone else, the field will be hidden in the edit form.

Workflow is used in Plone for three distinct, but overlapping purposes:

• To keep track of metadata, chiefly an object's state
• To create content review cycles and model other types of processes
• To manage object security

When writing content types, we will often create custom workflows to go with them. In this section, we will explain at a high level how Plone's workflow system works (pardon the pun), and then show an example of a simple workflow to go with our example types. An exhaustive manual on using workflows is beyond the scope of this manual, but hopefully this will cover the basics.

There is nothing Dexterity-specific about this section. Everything here applies equally well to content objects created with Archetypes or using CMF directly.

A DCWorkflow refresher

What follows is a fairly detailed description of DCWorkflow, originally posted here. You may find some of this a little detailed on first reading, so feel free to skip to the specifics later on. However, is useful to be familiar with the high level concepts. You're unlikely to need multi-workflow chains in your first few attempts at workflow, for instance, but it's useful to know what it is if you come across the term.

Plone's workflow system is known as DCWorkflow. It is a states-and-transitions system, which means that your workflow starts in a particular state (the initial state) and then moves to other states via transitions (also called actions in CMF).

When an object enters a particular state (including the initial state), the workflow is given a chance to update permissions on the object. A workflow manages a number of permissions - typically the "core" CMF permissions like View, Modify portal content and so on - and will set those on the object at each state change. Note that this is event-driven, rather than a real-time security check: only by changing the state is the security information updated. This is why you need to click Update security settings at the bottom of the portal_workflow screen in the ZMI when you change your workflows' security settings and want to update existing objects.

A state can also assign local roles to groups. This is akin to assigning roles to groups on Plone's Sharing tab, but the mapping of roles to groups happens on each state change, much like the mapping of roles to permissions. Thus, you can say that in the pending_secondary state, members of the Secondary reviewers group has the Reviewer local role. This is powerful stuff when combined with the more usual role-to-permission mapping, although it is not very commonly used.

State changes result in a number of variables being recorded, such as the actor (the user that invoked the transition), the action (the name of the transition), the date and time and so on. The list of variables is dynamic, so each workflow can define any number of variables linked to TALES expressions that are invoked to calculate the current value at the point of transition. The workflow also keeps track of the current state of each object. The state is exposed as a special type of workflow variable called the state variable. Most workflows in Plone uses the name review_state as the state variable.

Workflow variables are recorded for each state change in the workflow history. This allows you to see when a transition occurred, who effected it, and what state the object was in before or after. In fact, the "current state" of the workflow is internally looked up as the most recent entry in the workflow history.

Workflow variables are also the basis for worklists. These are basically pre-defined catalog queries run against the current set of workflow variables. Plone's review portlet shows all current worklists from all installed workflows. This can be a bit slow, but it does meant that you can use a single portlet to display an amalgamated list of all items on all worklists that apply to the current user. Most Plone workflows have a single worklist that matches on the review_state variable, e.g. showing all items in the pending state.

If states are the static entities in the workflow system, transitions (actions) are provide the dynamic parts. Each state defines zero or more possible exit transitions, and each transition defines exactly one target state, though it is possible to mark a transition as "stay in current state". This can be useful if you want to do something in reaction to a transition and record that the transition happened in the workflow history, but not change the state (or security) of the object.

Transitions are controlled by one or more guards. These can be permissions (the preferred approach), roles (mostly useful for the Owner role - in other cases it is normally better to use permissions) or TALES expressions. A transition is available if all its guard conditions are true. A transition with no guard conditions is available to everyone (including anonymous!).

Transitions are user-triggered by default, but may be automatic. An automatic transition triggers immediately following another transition provided its guard conditions pass. It will not necessarily trigger as soon as the guard condition becomes true, as that would involve continually re-evaluating guards for all active workflows on all objects!

When a transition is triggered, the IBeforeTransitionEvent and IAfterTransitionEvent events are triggered. These are low-level events from Products.DCWorkflow that can tell you a lot about the previous and current states. There is a higher level IActionSucceededEvent in Products.CMFCore that is more commonly used to react after a workflow action has completed.

In addition to the events, you can configure workflow scripts. These are either created through-the-web or (more commonly) as External Methods, and may be set to execute before a transition is complete (i.e. before the object enters the target state) or just after it has been completed (the object is in the new state). Note that if you are using event handlers, you'll need to check the event object to find out which transition was invoked, since the events are fired on all transitions. The per-transition scripts are only called for the specific transitions for which they were configured.

Multi-chain workflows

Workflows are mapped to types via the portal_workflow tool. There is a default workflow, indicated by the string (Default). Some types have no workflow, which means that they hold no state information and typically inherit permissions from their parent. It is also possible for types to have multiple workflows. You can list multiple workflows by separating their names by commas. This is called a workflow chain.

Note that in Plone, the workflow chain of an object is looked up by multi-adapting the object and the workflow to the IWorkflowChain interface. The adapter factory should return a tuple of string workflow names (IWorkflowChain is a specialisation of IReadSequence, i.e. a tuple). The default obviously looks at the mappings in the portal_workflow tool, but it is possible to override the mapping, e.g. by using a custom adapter registered for some marker interface, which in turn could be provided by a type-specific behavior.

Multiple workflows applied in a single chain co-exist in time. Typically, you need each workflow in the chain to have a different state variable name. The standard portal_workflow API (in particular, doActionFor(), which is used to change the state of an object) also asumes the transition ids are unique. If you have two workflows in the chain and both currently have a submit action available, only the first workflow will be transitioned if you do portal_workflow.doActionFor(context, 'submit'). Plone will show all available transitions from all workflows in the current object's chain in the State drop-down, so you do not need to create any custom UI for this. However, Plone always assumes the state variable is called review_state (which is also the variable indexed in portal_catalog). Therefore, the state of a secondary workflow won't show up unless you build some custom UI.

In terms of security, remember that the role-to-permission (and group-to-local-role) mappings are event-driven and are set after each transition. If you have two concurrent workflows that manage the same permissions, the settings from the last transition invoked will apply. If they manage different permissions (or there is a partial overlap) then only the permissions managed by the most-recently-invoked workflow will change, leaving the settings for other permissions untouched.

Multiple workflows can be very useful in case you have concurrent processes. For example, an object may be published, but require translation. You can track the review state in the main workflow and the translation state in another. If you index the state variable for the second workflow in the catalog (the state variable is always available on the indexable object wrapper so you only need to add an index with the appropriate name to portal_catalog) you can search for all objects pending translation, for example using a Collection.

Creating a new workflow

With the theory out of the way, let's show how to create a new workflow.

Workflows are managed in the portal_workflow tool. You can use the ZMI to create new workflows and assign them to types. However, it is usually preferable to create an installable workflow configuration using GenericSetup. By default, each workflow as well as the workflow assignments are imported and exported using an XML syntax. This syntax is comprehensive, but rather verbose if you are writing it manually.

For the purposes of this manual, we will show an alternative configuration syntax based on spreadsheets (in CSV format). This is provided by the collective.wtf package. You can read more about the details of the syntax in its documentation. Here, we will only show how to use it to create a simple workflow for the Session type, allowing members to submit sessions for review.

To use collective.wtf, we need to depend on it. In setup.py, we have:

      install_requires=[
          ...
          'collective.wtf',
      ],

As before, the <includeDependencies /> line in configure.zcml takes care of configuring the package for us.

A workflow definition using collective.wtf consists of a CSV file in the profiles/default/workflow_csv directory, which we will create, and a workflows.xml file in profiles/default which maps types to workflows.

The workflow mapping in profiles/default/workflows.xml looks like this:

<?xml version="1.0"?>
<object name="portal_workflow">
    <bindings>
        <type type_id="example.conference.session">
            <bound-workflow workflow_id="example.conference.session_workflow"/>
        </type>
    </bindings>
</object>

The CSV file itself is found in profiles/default/workflow_csv/example.conference.session_workflow.csv. It contains the following, which was exported to CSV from an OpenOffice spreadsheet. You can find the original spreadsheet with the example.conference source code. This applies some useful formatting, which is obviously lost in the CSV version.

For your own workflows, you may want to use this template as a starting point.

"[Workflow]"
"Id:","example.conference.session_workflow"
"Title:","Conference session workflow"
"Description:","Allows members to submit session proposals for review"
"Initial state:","draft"

"[State]"
"Id:","draft"
"Title:","Draft"
"Description:","The proposal is being drafted."
"Transitions","submit"
"Permissions","Acquire","Anonymous","Authenticated","Member","Manager","Owner","Editor","Reader","Contributor","Reviewer"
"View","N",,,,"X","X","X","X",,
"Access contents information","N",,,,"X","X","X","X",,
"Modify portal content","N",,,,"X","X","X",,,


"[State]"
"Id:","pending"
"Title:","Pending"
"Description:","The proposal is pending review"
"Worklist:","Pending review"
"Worklist label:","Conference sessions pending review"
"Worklist guard permission:","Review portal content"
"Transitions:","reject, publish"
"Permissions","Acquire","Anonymous","Authenticated","Member","Manager","Owner","Editor","Reader","Contributor","Reviewer"
"View","N",,,,"X","X","X","X",,"X"
"Access contents information","N",,,,"X","X","X","X",,"X"
"Modify portal content","N",,,,"X","X","X",,,"X"

"[State]"
"Id:","published"
"Title:","Published"
"Description:","The proposal has been accepted"
"Transitions:","reject"
"Permissions","Acquire","Anonymous","Authenticated","Member","Manager","Owner","Editor","Reader","Contributor","Reviewer"
"View","Y","X",,,,,,,,
"Access contents information","Y","X",,,,,,,,
"Modify portal content","Y",,,,"X","X","X",,,

"[Transition]"
"Id:","submit"
"Title:","Submit"
"Description:","Submit the session for review"
"Target state:","pending"
"Guard permission:","Request review"

"[Transition]"
"Id:","reject"
"Title:","Reject"
"Description:","Reject the session from the program"
"Target state:","draft"
"Guard permission:","Review portal content"

"[Transition]"
"Id:","publish"
"Title:","Publish"
"Description:","Accept and publish the session proposal"
"Target state:","published"
"Guard permission:","Review portal content"

Here, you can see several states and transitions. Each state contains a role/permission map, and a list of the possible exit transitions. Each transition contains a target state and other meta-data such as a title and a description, as well as guard permissions.

Note that like most other GenericSetup import steps, the workflow uses the Zope 2 permission title when referring to permissions.

When the package is (re-)installed, this workflow should be available under portal_workflow and mapped to the Session type.

If you have existing instances, don't forget to go to portal_workflow in the ZMI and click Update security settings at the bottom of the page. This ensures that existing objects reflect the most recent security settings in the workflow.

A note about add permissions

This workflow assumes that regular members can add Session proposals to Programs, which are then reviewed. Previously, we granted the example.conference: Add session permission to the Member role. This is necessary, but not sufficient to allow memers to add sessions to programs. The user will also need the generic Add portal content permission in the Program folder.

There are two ways to achieve this:

• Build a workflow for the Program type that manages this permission
• Use the Sharing tab to grant Can add to the Authenticated Users group. This grants the Contributor local role to members. By default, this role is granted the Add portal content permission.

The ZODB is a hierarchical object store where objects of different schemata and sizes can live side by side. This is great for managing individual content items, but not optimal for searching across the content repository. A naive search would need to walk the entire object graph, loading each object into memory and comparing object metadata with search criteria. On a large site, this would quickly become prohibitive.

Luckily, Zope comes with a technology called the ZCatalog, which is basically a table structure optimised for searching. In Plone, there's a ZCatalog instance called portal_catalog. Standard event handlers will index content in the catalog when it is created or modified, and unindex when the content is removed.

The catalog manages indexes, which can be searched, and metadata (also known as columns), which are object attributes for which the value is copied into the catalog. When we perform a search, the result is a lazily loaded list of objects known as catalog brains. Catalog brains contain the value of metadata columns (but not indexes) as attributes. The functions getURL(), getPath() and getObject() can be used to get the URL and path of the indexed content item, and to load the full item into memory.

Dexterity objects are more lightweight than Archetypes objects. This means that loading objects into memory is not quite as undesirable as is sometimes assumed. If you're working with references, parent objects, or a small number of child objects, it is usually OK to load objects directly to work with them. However, if you are working with a large or unknown-but-potentially-large number of objects, you should consider using catalog searches to find them and use catalog metadata to store frequently used values. There is an important trade-off to be made between limiting object access and bloating the catalog with unneeded indexes and metadata, though. In particular, large strings (such as the body text of a document) or binary data (such as the contents of image or file fields) should not be stored as catalog metadata.

Plone comes with a number of standard indexes and metadata columns. These correspond to much of the Dublin Core set of metadata as well as several Plone-specific attributes. You can view the indexes, columns and the contents of the catalog through the ZMI pages of the portal_catalog tool. If you've never done this, it is probably instructive to have a look, both to understand how the indexes and columns may apply to your own content types, and to learn what searches are already possible.

Indexes come in various types. The most common ones are:

• FieldIndex, the most common type, used to index a single value.
• KeywordIndex, used to index lists of values where you want to be able to search for a subset of the values. As the name implies, commonly used for keyword fields, such as the Subject Dublin Core metadata field.
• DateIndex, used to index Zope 2 DateTime objects. Note that if your type uses a Python datetime object, you'll need to convert it to a Zope 2 DateTime using a custom indexer!
• DateRangeIndex, used mainly for the effective date range.
• ZCTextIndex, used mainly for the SearchableText index. This is the index used for full-text search.
• ExtendedPathIndex, a variant of PathIndex, which is used for the path index. This is used to search for content by path and optionally depth.

Adding new indexes and metadata columns

When an object is indexed, the catalog will by default attempt to find attributes and methods that match index and column names on the object. Methods will be called (with no arguments) in an attempt to get a value. If a value is found, it is indexed.

Objects are normally acquisition-wrapped when they are indexed, which means that an indexed value may be acquired from a parent. This can be confusing, especially if you are building container types and creating new indexes for them. If child objects don't have attributes/methods with the same name, the parent object's value will be indexed for all children as well.

Catalog indexes and metadata can be installed with the catalog.xml GenericSetup import step. It is useful to look at the one in Plone (parts/omelette/Products/CMFPlone/profiles/default/catalog.xml).

As an example, let's index the track property of a Session in the catalog, and add a metadata column for this property as well. In profiles/default/catalog.xml, we have:

<?xml version="1.0"?>
<object name="portal_catalog">
    <index name="track" meta_type="FieldIndex">
        <indexed_attr value="track"/>
    </index>
    <column value="track"/>
</object>

Notice how we specify both the index name and the indexed attribute. It is possible to use an index name (the key you use when searching) that is different to the indexed attribute, although they are usually the same. The metadata column is just the name of an attribute. 

Creating custom indexers

Indexing based on attributes can sometimes be limiting. First of all, the catalog is indiscriminate in that it attempts to index every attribute that's listed against an index or metadata column for every object. Secondly, it is not always feasible to add a method or attribute to a class just to calculate an indexed value.

Plone 3.3 and later ships with a package called plone.indexer to help make it easier to write custom indexers: components that are invoked to calculate the value the catalog sees when it tries to index a given attribute. Indexers can be used to index a different value to the one stored on the object, or to allow indexing of a "virtual" attribute that does not actually exist on the object is question. Indexers are usually registered on a per-type basis, so you can have different implementations for different types of content.

To illustrate indexers, we will add three indexers to program.py: Two will provide values for the start and end indexes, normally used by Plone's Event type. We actually have attributes with the correct name for these already, but they use Python datetime objects whereas the DateIndex requires a Zope 2 DateTime.DateTime object. (Python didn't have a datetime module when this part of Zope was created!) The third indexer will be used to provide a value for the Subject index that takes its value from the tracks list.

from DateTime import DateTime
from plone.indexer import indexer

...

@indexer(IProgram)
def startIndexer(obj):
    if obj.start is None:
        return None
    return DateTime(obj.start.isoformat())
grok.global_adapter(startIndexer, name="start")

@indexer(IProgram)
def endIndexer(obj):
    if obj.end is None:
        return None
    return DateTime(obj.end.isoformat())
grok.global_adapter(endIndexer, name="end")

@indexer(IProgram)
def tracksIndexer(obj):
    return obj.tracks
grok.global_adapter(tracksIndexer, name="Subject")

Here, we use the @indexer decorator to create an indexer. This doesn't register the indexer component, though, so we need to use grok.global_adapter() to finalise the registration. Crucially, this is where the indexer's name is defined. This is the name of the indexed attribute for which the indexer is providing a value.

Since all of these indexes are part of a standard Plone installation, we won't register them in catalog.xml. If you are creating custom indexers and need to add new catalog indexes or columns for them, remember that the "indexed attribute" name (and the column name) must match the name of the indexer as set in its adapter registration.

Searching using your indexes

Once we have registered our indexers and re-installed our product (to ensure that the catalog.xml import step is allowed to install new indexes in the catalog), we can use our new indexes just like we would any of the default indexes. The pattern is always the same:

from Products.CMFCore.utils import getToolByName
# get the tool
catalog = getToolByName(context, 'portal_catalog')
# execute a search
results = catalog(track='Track 1')
# examine the results
for brain in results:
    start = brain.start
    url = brain.getURL()
    obj = brain.getObject() # Performance hit!

This shows a simple search using the portal_catalog tool, which we look up from some context object. We call the tool to perform a search, passing search criteria as keyword arguments, where the left hand side refers to an installed index and the right hand side is the search term.

Some of the more commonly used indexes are:

• Title, the object's title.
• Description, the object's description.
• path, the object's path. The argument is a string like '/foo/bar'. To get the path of an object (e.g. a parent folder), do '/'.join(folder.getPhysicalPath()). Searching for an object's path will return the object and any children. To depth-limit the search, e.g. to get only those 1 level deep, use a compound query, e.g. path={'query': '/'.join(folder.getPhysicalPath()), 'depth': 1}. If a depth is specified, the object at the given path is not returned (but any children within the depth limit are).
• object_provides, used to match interfaces provided by the object. The argument is an interface name or list of interface names (of which any one may match). To get the name of a given interface, you can call ISomeInterface.__identifier__.
• portal_type, used to match the portal type. Note that users can rename portal types, so it is often better not to hardcode these. Often, using an object_provides search for a type-specific interface will be better. Conversely, if you are asking the user to select a particular type to search for, then they should be choosing from the currently installed portal_types.
• SearchableText, used for full-text searches. This supports operands like AND and OR in the search string.
• Creator, the username of the creator of a content item
• Subject, a KeywordIndex of object keywords
• review_state, an object's workflow state

In addition, the search results can be sorted based on any FieldIndex, KeywordIndex or DateIndex using the following keyword arguments:

• Use sort_on='<index name>' to sort on a particular index. For example, sort_on='sortable_title' will produce a sensible title-based sort. sort_on='Date' will sort on the publication date, or the creation date if this is not set.
• Add sort_order='reverse' to sort in reverse. The default is sort_order='ascending'. 'descending' can be used as an alias for 'reverse'.
• Add sort_limit=10 to limit to approximately 10 search results. Note that it is possible to get more results due to index optimisations. Use a list slice on the catalog search results to be absolutely sure that you have got the maximum number of results, e.g. results = catalog(..., sort_limit=10)[:10]. Also note that the use of sort_limit requires a sort_on as well.

Some of the more commonly used metadata columns are:

• Creator, the user who created the content object
• Date, the publication date or creation date, whichever is later
• Title, the object's title
• Description, the object's description
• getId, the object's id (note that this is an attribute, not a function)
• review_state, the object's workflow state
• portal_type, the object's portal type

For more information about catalog indexes and searching, see the ZCatalog chapter in the Zope 2 book.

Until now, we have used Dexterity's default content add and edit forms, supplying form hints in our schemata to influence how the forms are built. For most types, that is all that's ever needed. In some cases, however, we want to build custom forms, or supply additional forms.

Dexterity uses the z3c.form library to build its forms, via the plone.z3cform integration package.

Note that the plone.z3cform package requires that standard z3c.form forms are used via a form wrapper view. In Dexterity, this wrapper is normally applied automatically by the form grokkers in plone.directives.form and plone.directives.dexterity.

Dexterity also relies on plone.autoform, in particular its AutoExtensibleForm base class, which is responsible for processing form hints and setting up z3c.form widgets and groups (fieldsets). A custom form, therefore, is simply a view that uses these libraries, although Dexterity provides some helpful base classes that make it easier to construct forms based on the schema and behaviors of a Dexterity type.

If you want to build standalone forms not related to content objects, see the z3c.form documentation. For convenience, you may want to use the base classes and schema support in plone.directives.form.

Edit forms

An edit form is just a form that is registered for a particular type of content and knows how to register its fields. If the form is named edit, it will replace the default edit form, which is registered with that name for the more general IDexterityContent interface.

Dexterity provides a standard edit form base class that provides sensible defaults for buttons, labels and so on. This should be registered for a type schema (not a class). To create an edit form that is identical to the default, we could do:

class EditForm(dexterity.EditForm):
    grok.context(IFSPage)

The dexterity module is plone.directives.dexterity and the grok module is five.grok.

The default name for the form is edit, but we could supply a different name using grok.name(). The default permission is cmf.ModifyPortalContent, but we could require a different permission with grok.require(). We could also register the form for a particular browser layer, using grok.layer().

This form is of course not terribly interesting, since it is identical to the default. However, we can now start changing fields and values. For example, we could:

• Override the schema property to tell plone.autoform to use a different schema interface (with different form hints) than the content type schema
• Override the additional_schemata property to tell plone.autoform to use different supplemental schema interfaces. The default is to use all behavior interfaces that provide the IFormFieldProvider marker from plone.directives.form.
• Override the label and description properties to provide different a different title and description for the form.
• Set the z3c.form fields and groups attributes directly.
• Override the updateWidgets() method to modify widget properties, or one of the other update*() methods, to perform additional processing on the fields. In most cases, these require us to call the super version at the beginning. See the plone.autoform and z3c.form documentation to learn more about the sequence of calls that eminate from the form update() method in the z3c.form.form.BaseForm class.

Content add sequence

Add forms are similar to edit forms in that they are built from a type's schema and the schemata of its behaviors. However, for an add form to be able to construct a content object, it needs to know the portal_type to use.

You should realise that the FTIs in the portal_types tool can be modified through the web. It is even possible to create new types through the web that re-use existing classes and factories.

For this reason, add forms are looked up via a namespace traversal adapter alled ++add++. You may have noticed this in the URLs to add forms already. What actually happens is this:

• Plone renders the add menu.
• To do so, it looks, among other places, for actions in the folder/add category. This category is provided by the portal_types tool.
• The folder/add action category is constructed by looking up the add_view_expr property on the FTIs of all addable types. This is a TALES expression telling the add menu which URL to use.
• The default add_view_expr in Dexterity (and CMF 2.2) is string:${folder_url}/++add++${fti/getId}. That is, it uses the ++add++ traversal namespace with an argument containing the FTI name.
• A user clicks on an entry in the menu as is taken to a URL like /path/to/folder/++add++my.type.
• The ++add++ namespace adapter looks up the FTI with the given name, and gets its factory property.
• The factory property of an FTI gives the name of a particular zope.component.interfaces.IFactory utility, which is used later to construct an instance of the content object. Dexterity automatically registers a factory instance for each type, with a name that matches the type name, although it is possible to use an existing factory name in a new type. This allows administrators to create new "logical" types that are functionally identical to an existing type.
• The ++add++ namespace adapter looks up the actual form to render as a multi-adapter from (context, request, fti) to Interface with a name matching the factory property. Recall that a standard view is a multi-adapter from (context, request) to Interface with a name matching the URL segment for which the view is looked up. As such, add forms are not standard views, because they get the additional fti parameter when constructed.
• If this fails, there is no custom add form for this factory (as is normally the case). The fallback is an unnamed adapter from (context, request, fti). The default Dexterity add form is registered as such an adapter, specific to the IDexterityFTI interface.
• The form is rendered like any other z3c.form form instance, and is subject to validation, which may cause it to be loaded several times.
• Eventually, the is successfully submitted. At this point: 
• The standard AddForm base class will look up the factory from the FTI reference it holds and call it to create an instance.
• The default Dexterity factory looks at the klass attribute of the FTI (class is a reserved word in Python...) to determine the actual content class to use, creates an object and initialises it.
• The portal_type attribute of the newly created instance is set to the name of the FTI. Thus, if the FTI is a "logical type" created through the web, but using an existing factory, the new instance's portal_type will be set to the "logical type".
• The object is initialised with the values submitted in the form
• An IObjectCreatedEvent is fired
• The object is added to its container
• The user is redirected to the view specified in the immediate_view property of the FTI

This sequence is pretty long, but thankfully we rarely have to worry about it. In most cases, we can use the default add form, and when we can't, creating a custom add form is no more difficult than creating a custom edit form. The add form grokker take care of registering the add view appropriately.

Custom add forms

As with edit forms, Dexterity provides a sensible base class for add forms that knows how to deal with the Dexterity FTI and factory.

A custom form replicating the default would look like this:

class AddForm(dexterity.AddForm):
    grok.name('example.fspage')

The name here should match the factory name. By default, Dexterity types have a factory called the same as the FTI name. If no such factory exists (i.e. you have not registered a custom IFactory utility), a local factory utility will be created and managed by Dexterity when the FTI is installed.

Also note that we do not specify a context here. Add forms are always registered for any IFolderish context. We can specify a layer with grok.layer() and a permission other than the default cmf.AddPortalContent with grok.require().

If the permission used for the add form is different to the add_permission set in the FTI, the user needs to have both permissions to be able to see the form and add content. For this reason, most add forms will use the generic cmf.AddPortalContent permission. The add menu will not render links to types where the user does not have the add permission stated in the FTI, even if this is different to cmf.AddPortalContent.

As with edit forms, we can customise this form by overriding z3c.form and plone.autoform properties and methods. See the z3c.form documentation on add forms for more details.

When we learned about configuring the Dexterity FTI, we saw the klass attribute and how it could be used to refer to either the Container or Item content classes. These classes are defined in the plone.dexterity.content module, and represent container (folder) and item (non-folder) types, respectively.

For most applications, these two classes will suffice. We will normally use behaviors, adapters, event handlers and schema interfaces to build additional functionality for our types. In some cases, however, it is useful or necessary to override the class, typically to override some method or property provided by the base class that cannot be implemented with an adapter override. A custom class may also be able to provide marginally better performance by side-stepping some of the schema-dependent dynamic behavior found in the base classes. In real life, you are very unlikely to notice, though.

Creating a custom class is simple: simply derive form one of the standard ones, e.g.:

from plone.dexterity.content import Item

class MyItem(Item):
    """A custom content class"""
    ...

For a container type, we'd do:

from plone.dexterity.content import Container

class MyContainer(Container):
    """A custom content class"""
    ...

You can now add any required attributes or methods to this class.

To make use of this class, set the klass attribute in the FTI to its dotted name, e.g.

    <property name="klass">my.package.myitem.MyItem</property>

This will cause the standard Dexterity factory to instantiate this class when the user submits the add form.

As an alternative to setting klass in the FTI, you amy provide your own IFactory utility for this type in lieu of Dexterity's default factory (see plone.dexterity.factory). However, you need to be careful that this factory performs all necessary initialisation, so it is normally better to use the standard factory.

Custom class caveats

There are a few important caveats when working with custom content classes:

• Make sure you use the correct base class: either plone.dexterity.content.Item or plone.dexterity.content.Container.
• If you mix in other base classes, it is safer to put the Item or Container class first. If another class comes first, it may override the __name__, __providedBy__,  __allow_access_to_unprotected_subobjects__ and/or isPrincipiaFolderish properties, and possibly the __getattr__() and __getitem__() methods, causing problems with the dynamic schemata and/or folder item security. In all cases, you may need to explicitly set these attributes to the ones from the correct base class.
• If you define a custom constructor, make sure it can be called with no arguments, and with an optional id argument giving the name.

Zope supports WebDAV, a protocol that allows content objects to be viewed, modified, copied, renamed, moved and deleted as if they were files on the filesystem. WebDAV is also used to support saving to remote locations from various desktop programs. In addition, WebDAV powers the External Editor product, which allows users to launch a desktop program from within Plone to edit a content object.

To configure a WebDAV server, you can add the following option to the [instance] section of your buildout.cfg and re-run buildout.

webdav-address = 9800

See the documentation for plone.recipe.zope2instance for details. When Zope is started, you should now be able to mount it as a WebDAV server on the given port.

Most operating systems support mounting WebDAV servers as folders. Unfortunately, not all WebDAV implementations are very good. Dexterity content should work with Windows Web Folders (open Internet Explorer, go to File | Open, type in a WebDAV address, e.g. http://localhost:9800, and then select "Open as web folder" before hitting OK) and well-behaved clients such as Novell NetDrive. 

On Mac OS X, the Finder claims to support WebDAV, but the implementation is so flakey that it is just as likely to crash Mac OS X as it is to let you browse files and folders. Use a dedicated WebDAV client instead, such as Cyberduck.

Default WebDAV behaviour

By default, Dexterity content can be downloaded and uploaded using a text format based on RFC (2)822, the same standard used to encode email messages. Most fields are encoded in headers, whilst the field marked as "primary" will be contained in the body of the message. If there is more than one primary field, a multi-part message is created.

A field can be marked as "primary" using the primary() directive from plone.directives.form. For example:

class ISession(form.Schema):
    """A conference session. Sessions are managed inside Programs.
    """
    
    title = schema.TextLine(
            title=_(u"Title"),
            description=_(u"Session title"),
        )
    
    description = schema.Text(
            title=_(u"Session summary"),
        )
    
    form.primary('details')
    details = RichText(
            title=_(u"Session details"),
            required=False
        )

    form.widget(presenter=AutocompleteFieldWidget)
    presenter = RelationChoice(
            title=_(u"Presenter"),
            source=ObjPathSourceBinder(object_provides=IPresenter.__identifier__),
            required=False,
        )
    
    dexterity.write_permission(track='example.conference.ModifyTrack')
    track = schema.Choice(
            title=_(u"Track"),
            source=possibleTracks,
            required=False,
        )

This will actually apply the IPrimaryField marker interface from the plone.rfc822 package to the given field(s).

A WebDAV download of this content item will by default look like this:

title: Test session
description: First session
presenter: 713399904
track: Administrators
MIME-Version: 1.0
Content-Type: text/html; charset="utf-8"
Portal-Type: example.conference.session

<p>Details <b>here</b></p>

Notice how most fields are encoded as header strings. The presenter relation field stores a number, which is the integer id of the target object. Note that this id is generated when the content object is created, and so is unlikely to be valid on a different site. The details field, which we marked as primary, is encoded in the body of the message.

It is also possible to upload such a file to create a new session. In order to do that, the content_type_registry tool needs to be configured with a predicate that can detect the type of content from the uploaded file and instantiate the correct type of object. Such predicates could be based on an extension or a filename pattern. Below, we will see a different approach that uses a custom "file factory" for the containing Program type.

Containers

Container objects will be shown as collections (WebDAV-speak for folders) for WebDAV purposes. This allows the WebDAV client to open the container and list its contents. However, representing containers as collections makes it impossible to access the data contained in the various fields of the content object.

To allow access to this information, a pseudo-file called _data will be exposed inside a Dexterity container. This file can be read and written like any other, to access or modify the container's data. It cannot be copied, moved, renamed or deleted: those operations should be performed on the container itself.

Customising WebDAV behaviour

There are several ways in which you can influence the WebDAV behaviour of your type.

• If you are happy with the RFC 2822 format, you can provide your own plone.rfc822.interfaces.IFieldMarshaler adapters to provide alternate serialisations and parsers for fields. See the plone.rfc822 documentation for details.
• If you want to use a different file representation, you can provide your own IRawReadFile and IRawWriteFile adapters. For example, if you have a content object that stores binary data, you could return this data directly, with an appropriate MIME type, to allow it to be edited in a desktop program (e.g. an image editor if the MIME type is image/jpeg). The file plone.dexterity.filerepresentation contains two base classes, ReadFileBase and WriteFileBase, which you may be able to use to make it easier to implement these interfaces.
• If you want to control how content objects are created when a new file or directory is dropped into a particular type of container, you can provide your own IFileFactory or IDirectoryFactory adapters. See plone.dexterity.filerepresentation for the default implementations.

As an example, let's register a custom IFileFactory adapter for the IProgram type. This adapter will not rely on the content_type_registry tool to determine which type to construct, but will instead create a Session object, since that is the only type that is allowed inside a Program container.

The code, in program.py, looks like this:

from five import grok

...

from zope.component import createObject
from zope.event import notify
from zope.lifecycleevent import ObjectCreatedEvent
from zope.filerepresentation.interfaces import IFileFactory

...

class ProgramFileFactory(grok.Adapter):
    """Custom file factory for programs, which always creates a Session.
    """
    
    grok.implements(IFileFactory)
    grok.context(IProgram)
    
    def __call__(self, name, contentType, data):
        session = createObject('example.conference.session', id=name)
        notify(ObjectCreatedEvent(session))
        return session

This adapter overrides the DefaultFileFactory found in plone.dexterity.filerepresentation. It creates an object of the designated type, fires an IObjectModifiedEvent and then returns the object, which will then be populated with data from the uploaded file.

To test this, you could write a text file like the one shown above in a text editor and save it on your desktop, then drag it into the folder in your WebDAV client representing a Program.

Here is a simple automated integration test for the same component:

    def test_file_factory(self):
        self.folder.invokeFactory('example.conference.program', 'p1')
        p1 = self.folder['p1']
        fileFactory = IFileFactory(p1)
        newObject = fileFactory('new-session', 'text/plain', 'dummy')
        self.failUnless(ISession.providedBy(newObject))

How it all works

The rest of this section describes in some detail how the various WebDAV related components interact in Zope 2, CMF and Dexterity. This may be helpful if you are trying to customise or debug WebDAV behaviour.

Background

Basic WebDAV support can be found in the webdav package. This defines two base classes, webdav.Resource.Resource and webdav.Collection.Collection. Collection extends Resource. These are mixed into item and container content objects, respectively.

The webdav package also defines the NullResource object. A NullResource is a kind of placeholder, which supports the HTTP verbs HEAD, PUT, and MKCOL.

Contains based on ObjectManager (including those in Dexterity) will return a NullResource if they cannot find the requested object and the request is a WebDAV request.

The zope.filerepresentation package defines a number of interfaces which are intended to help manage file representations of content objects. Dexterity uses these interfaces to allow the exact file read and write operations to be overridden without subclassing.

HEAD

A HEAD request retrieves headers only.

Resource.HEAD() sets Content-Type based on self.content_type(), Content-Length based on self.get_size(), Last-Modified based on self._p_mtime, and an ETag based on self.http__etag(), if available.

Collection.HEAD() looks for self.index_html.HEAD() and returns its value if that exists. Otherwise, it returns a 405 Method Not Allowed response. If there is no index_html object, it returns 404 Not Found.

GET

A GET request retrieves headers and body.

Zope calls manage_DAVget() to retrieve the body. The default implementation calls manage_FTPget().

In Dexterity, manage_FTPget() adapts self to IRawReadFile and uses its mimeType and encoding properties to set the Content-Type header, and its size() method to set Content-Length.

If the IRawReadFile adapter is also an IStreamIterator, it will be returned for the publisher to consume directly. This provides for efficient serving of large files, although it does require that the file can be read in its entirety with the ZODB connection closed. Dexterity solves this problem by writing the file content to a temporary file on the server.

If the IRawReadFile adapter is not a stream iterator, its contents are returned as a string, by calling its read() method. Note that this loads the entire file contents into memory on the server.

The default IRawReadFile implementation for Dexterity content returns an RFC 2822 style message document. Most fields on the object and any enabled behaviours will be turned into UTF-8 encoded headers. The primary field, if any, will be returned in the body, also most likely encoded as an UTF-8 encoded string. Binary data may be base64 encoded instead.

A type which wishes to override this behaviour can provide its own adapter. For example, an image type could return the raw image data.

PUT

A PUT request reads the body of a request and uses it to update a resource that already exists, or to create a new object.

By default Resource.PUT() fails with 405 Method Not Allowed. That is, it is not by default possible to PUT to a resource that already exists. The same is true of Collection.PUT().

In Dexterity, the PUT() method is overridden to adapt self to zope.filerepresentation.IRawWriteFile, and call its write() method one or more times, writing the contents of the request body, before calling close(). The mimeType and encoding properties will also be set based on the value of the Content-Type header, if available.

The default implementation of IRawWriteFile for Dexterity objects assumes the input is an RFC 2822 style message document. It will read header values and use them to set fields on the object or in behaviours, and similarly read the body and update the corresponding primary field.

NullResource.PUT() is responsible for creating a new content object and initialising it (recall that a NullResource may be returned if a WebDAV request attempts to traverse to an object which does not exist). It sniffs the content type and body from the request, and then looks for the PUT_factory() method on the parent folder.

In Dexterity, PUT_factory() is implemented to look up an IFileFactory adapter on self and use it to create the empty file. The default implementation will use the content_type_registry tool to determine a type name for the request (e.g. based on its extension or MIME type), and then construct an instance of that type.

Once an instance has been constructed, the object will be initialised by calling its PUT() method, as above.

Note that when content is created via WebDAV, an IObjectCreatedEvent will be fired from the IFileFactory adapter, just after the object has been constructed. At this point, none of its values will be set. Subsequently, at the end of the PUT() method, an IObjectModifiedEvent will be fired. This differs from the event sequence of an object created through the web. Here, only an IObjectCreatedEvent is fired, and only after the object has been fully initialised.

DELETE

A DELETE request instructs the WebDAV server to delete a resource.

Resource.DELETE() calls manage_delObjects() on the parent folder to delete an object.

Collection.DELETE() does the same, but checks for write locks of all children of the collection, recursively, before allowing the delete.

PROPFIND

A PROPFIND request returns all or a set of WebDAV properties. WebDAV properties are metadata used to describe an object, such as the last modified time or the author.

Resource.PROPFIND() parses the request and then looks for a propertysheets attribute on self.

If an 'allprop' request is received, it calls dav__allprop(), if available, on each property sheet. This method returns a list of name/value pairs in the correct WebDAV XML encoding, plus a status.

If a 'propnames' request is received, it calls dav__propnames(), if available, on each property sheet. This method returns a list of property names in the correct WebDAV XML encoding, plus a status.

If a 'propstat' request is received, it calls dav__propstats(), if available, on each property sheet, for each requested property. This method returns a property name/value pair in the correct WebDAV XML encoding, plus a status.

The PropertyManager mixin class defines the propertysheets variable to be an instance of DefaultPropertySheets. This in turn has two property sheets, default, a DefaultProperties instance, and webdav, a DAVProperties instance.

The DefaultProperties instance contains the main property sheet. This typically has a title property, for example.

DAVProperties will provides various core WebDAV properties. It defines a number of read-only properties: creationdate, displayname, resourcetype, getcontenttype, getcontentlength, source, supportedlock, and lockdiscovery. These in turn are delegated to methods prefixed with dav__, so e.g. reading the creationdate property calls dav__creationdate() on the property sheet instance. These methods in turn return values based on the the property manager instance (i.e. the content object). In particular:

• creationdate returns a fixed date (January 1st, 1970).
• displayname returns the value of the title_or_id() method
• resourcetype returns an empty string or <n:collection/>
• getlastmodified returns the ZODB modification time
• getcontenttype delegates to the content_type() method, falling back on the default_content_type() method. In Dexterity, content_type() is implemented to look up the IRawReadFile adapter on the context and return the value of its mimeType property.
• getcontentlength delegates to the get_size() method (which is also used for the "size" column in Plone folder listings). In Dexterity, this looks up a zope.size.interfaces.ISized adapter on the object and calls sizeForSorting(). If this returns a unit of 'bytes', the value portion is used. Otherwise, a size of 0 is returned.
• source returns a link to /document_src, if that attribute exists
• supportedlock indicates whether IWriteLock is supported by the content item
• lockdiscovery returns information about any active locks

Other properties in this and any other property sheets are returned as stored when requested.

If the PROPFIND request specifies a depth of 1 or infinity (i.e. the client wants properties for items in a collection), the process is repeated for all items returned by the listDAVObjects() methods, which by default returns all contained items via the objectValues() method.

PROPPATCH

A PROPPATCH request is used to update the properties on an existing object.

Resource.PROPPATCH() deals with the same types of properties from property sheets as PROPFIND(). It uses the PropertySheet API to add or update properties as appropriate.

MKCOL

A MKCOL request is used to create a new collection resource, i.e. create a new folder.

Resource.MKCOL() raises 405 Method Not Allowed, because the resource already exists (remember that in WebDAV, the MKCOL request, like a PUT for a new resource, is sent with a location that specifies the desired new resource location, not the location of the parent object).

NullResource.MKCOL() handles the valid case where a MKCOL request has been sent to a new resource. After checking that the resource does not already exist, that the parent is indeed a collection (folderish item), and that the parent is not locked, it calls the MKCOL_handler() method on the parent folder.

In Dexterity, MKCOL()_handler is overridden to adapt self to an IDirectoryFactory from zope.filerepresentation and use this to create a directory. The default implementation simply calls manage_addFolder() on the parent. This will create an instance of the Folder type.

COPY

A COPY request is used to copy a resource.

Resource.COPY() implements this operation using the standard Zope content object copy semantics.

MOVE

A MOVE request is used to relocate or rename a resource.

Resource.MOVE() implements this operation using the standard Zope content object move semantics.

LOCK

A LOCK request is used to lock a content object.

All relevant WebDAV methods in the webdav package are lock aware. That is, they check for locks before attempting any operation that would violate a lock.

Also note that plone.locking uses the lock implementation from the webdav package by default.

Resource.LOCK() implements locking and lock refresh support.

NullResource.LOCK() implements locking on a NullResource. In effect, this means locking the name of the non-existent resource. When a NullResource is locked, it is temporarily turned into a LockNullResource object, which is a persistent object set onto the parent (remember that a NullResource is a transient object returned when a child object cannot be found in a WebDAV request).

UNLOCK

An UNLOCK request is used to unlock a locked object.

Resource.UNLOCK() handles unlock requests.

LockNullResource.UNLOCK() handles unlocking of a LockNullResource. This deletes the LockNullResource object from the parent container.

Fields on container objects

When browsing content via WebDAV, a container object (folderish item) will appear as a folder. Most likely, this object will also have content in the form of schema fields. To make this accessible, Dexterity containers expose a pseudo-file with the name '_data', by injecting this into the return value of listDAVObjects() and adding a special traversal hook to allow its contents to be retrieved.

This file supports HEAD, GET, PUT, LOCK, UNLOCK, PROPFIND and PROPPATCH requests (an error will be raised if the user attempts to rename, copy, move or delete it). These operate on the container object, obviously. For example, when the data object is updated via a PUT request, the PUT() method on the container is called, by default delegating to an IRawWriteFile adapter on the container.