Send announcements from workflow

A common requirement of sites with registered users or members is to be able to email different groups of members with information that might be of interestor use to them.

This might be with notifications of new content, matching users' preferences and interests, or it could be more generic send an email to all of such-and-such a group business requirement. In either case, you'll probably be interested in respecting the permissions your users have given you to contact them.

This will always mean only sending emails to people who have given you permission to do so, and often matching the emails to people whose interest you know matches the thing you want to send.

Because you're running your site professionally (you are, right?), you'll also want to send emails as part of a workflow. This means that only appropriate people will have the rights to send email, and you'll have an audit trail of who sent what, when. That way, if you get complaints of spam, you can quickly find out the facts (You gave us permission to do it and have an interest in the subject, or otherwise) and respond appropriately.

Business Requirements

For a system which announces new content on a site, our system requirements are basically:

1. To be able to send email to registered users of the site.
2. To enable users to give and retract permission for you to email them, and for your email dispatch to respect that permission.
3. To enable users to register areas of interest, and for your email dispatch to only send email to users who are interested in areas which relate to what you propose to send.
4. To have emails only sent out when the content is signed off for dispatch.
5. To have the email contain
   1. A standard short message
   2. A paragraph of content-specific text (this will be a synopsis of the content, and is an existing data field in our content schema)
   3. A link to the content
6. To have the person who sends the announcement (the actor in the workflow transition, in UML-speak) receive a notification of who the email announcement went to.

We could have a requirement that emails are sent on a schedule, and pick up all new content published in the last n hours. But for simplicity, that's not how I've chosen to do it. The normal thing to say here is that that's left as an exercise to the reader.

System

The system I'll use to demonstrate this is Plone - a Zope-based CMS which provides a rich API that provides hooks for adding user data elements and state & transition workflow scripting, making all of the following extremely simple.

I'm not going to cover installing Zope, Plone, or Plone sites here - there's some reasonable install documentation on the Plone site, with some handy all-in-one installers for Windows & Mac OSX.

This should also work on basic CMF, but is untested on that platform. Caveat Emptor.

Data Requirements

The basis of the solution is (pseudo-code)

If user has emailPermission and
userInterest(any) matches contentKeyword(any) then
sendEmail

Therefore, there are four key pieces of data:

Content Data

1. Metadata Keywords
   Plone provides this out of the box, with a light Dublin Core implementation.

User Data

2. Email Address
   Self-evidently, you need this. It's an existing user attribute in Plone, and is already required.
3. Email Permission
   This is a simple user-settable boolean flag to say that the user grants permission to you to send them relevant email. Plone does not provide this out of the box, so we'll need to add it. We could use the Listed user property, but it's less confusing to keep them separate, and gives the user better control over their preferences.
4. User Interests
   This is a list data-type, each element being an area that the user is interested in. If it's not there, they're not interested.

Implementation

Adding User Data Elements

Adding data elements to users in Plone is pretty simple (unlike adding elements to content, which is a whole other story). Plone keeps its user data schema in the portal_memberdata tool. In the ZMI, navigate to there, and select the Properties tab. This will give you a list of the currently available user properties, and their default values.

You need to add two new properties (all values is case-sensitive):

The result is that all current users, and all new users, will have no interests registered, and have not granted you permission to email them. This is A Good Thing, as your emails will now be opt-in.

Enabling Data Entry

It's no use having data elements on each user data object if the users can't enter data into the waiting slots. So we need to customise the standard Plone form that users use to personalise their experience. This form can be found at /portal_skins/plone_forms/personalize_form.

If you're not used to customising CMF/Plone sites, you'll be worried that it's not editable. This is because it's looking at your server's file system (which Zope won't write to) for the data for this folder. To enable editing, you need to transfer the HTML file to the /portal_skins/custom folder, where you can edit it. There's a handy button on the locked form page, labelled Customize. Push it... you can now edit the form. How and why this works is beyond the scope of this article. For now, accept that it just does.

Once you've hit Customize, you'll find the HTML in a normal text-area form field. Again, don't worry that it appears not to have any of your site template in. The CMS is picking the main content out and inserting it into a template slot.

Enabling emailPermission Selection

Grab the HTML and drop it into your favourite HTML editor. Look for 'div's labelled thusly:

  <div class="row">
      <div class="label">
          <span i18n:translate="label_listed_status">Listed status</span>

          <div id="listed_status_help"
               i18n:translate="help_listed_status" 
               class="help" 
               style="visibility:hidden">
          Select whether you want to be listed on the public membership listing
          or not.       Remember that your Member folder will still be publicly accessible unless
          you change its security settings, even if you select 'unlisted' here.
          </div>                
      </div>
      <div class="field"
         tal:define="listed python:request.get('listed', member.listed);
                       tabindex tabindex/next;">
          <input type="radio" 
                 class="noborder" 
                 name="listed" 
                 value="on"
                 id="cb_listed" 
                 checked="checked"
                 tabindex=""
                 onfocus="formtooltip('listed_status_help',1)" 
                 onblur="formtooltip('listed_status_help',0)"
                 tal:attributes="tabindex tabindex;" 
                 tal:condition="listed" 
                 />

          <input type="radio" 
                 class="noborder" 
                 name="listed" 
                 value="on"
                 id="cb_listed"
                 tabindex=""
                 onfocus="formtooltip('listed_status_help',1)" 
                 onblur="formtooltip('listed_status_help',0)"
                 tal:attributes="tabindex tabindex;" 
                 tal:condition="not: listed" 
                 />

          <label for="cb_listed" i18n:translate="label_member_listed">Listed</label>

          <br />

          <input type="radio" 
                 class="noborder" 
                 name="listed" 
                 value=""
                 id="cb_unlisted"
                 tabindex=""
                 onfocus="formtooltip('listed_status_help',1)" 
                 onblur="formtooltip('listed_status_help',0)"
                 tal:attributes="tabindex tabindex;" 
                 tal:condition="listed" 
                 />

          <input type="radio" 
                 class="noborder" 
                 name="listed" 
                 value=""
                 id="cb_unlisted" 
                 checked="checked"
                 tabindex=""
                 onfocus="formtooltip('listed_status_help',1)" 
                 onblur="formtooltip('listed_status_help',0)"
                 tal:attributes="tabindex tabindex;" 
                 tal:condition="not: listed " 
                 />                

          <label for="cb_unlisted" i18n:translate="label_member_unlisted">Unlisted</label>
      </div>
  </div>

(The indentation isn't significant, just useful)

This is the field for the Listed field. We're going to crib it somewhat to produce radio buttons that give the user an opt-in/out mechanism, selecting and deselecting the emailPermission data element.

Let's unpack that a bit:

  <div class="row">

Each field is enclosed within a div with this class:

  <div class="label">
      <span i18n:translate="label_listed_status">Listed status</span>

The label class encapsulates both the field label and the dHTML tooltip. There's also support for auto-translation, but if you're using this, for your own fields, you'll need to add your own translations for the new content:

  <div class="field"
       tal:define="listed python:request.get('listed', member.listed);
                   tabindex tabindex/next;">

Now we're into Zope Page Templating. We're setting variables with scope of this div, and the key one is getting the listed property out of this user's data.

Next up, we have a couple of radio buttons. Actually, we have code for two pairs of radio buttons, but there's some conditionalising going on, so only the ones which apply to the current state appear and have the appropriate selection data. Here's the button to make the user unlisted, with non-significant values removed:

  <input type="radio" 
         name="listed" 
         value="on"
         checked="checked"
         tal:condition="listed" 
         />
  <input type="radio" 
         name="listed" 
         value="on"
         tal:condition="not: listed" 
         />                   

We have a checked button which only appears if the member's listing property is set, and an unchecked one which only appears if the property is not set. For the other radio button, the values are reversed.

With all this knowledge, it should be fairly simple to construct our own radio button form field. Simply replace all references to listed with emailPermission (ie the data element name you added to the member), and reword the labelling. Here's my code - I've also added some more explanatory text as it's a sensitive issue:

  <div class="row">
      <div class="label">
          Contact Permission
          <div id="permission_status_help"
               i18n:translate="help_emailPermission_status" 
               class="help" 
               style="visibility:hidden">
            Select whether you want us to send you relevant 
            information by email. 
          </div>                
      </div>
      <div style="margin:0px;">
          We would like to send you email, announcing 
          new content that's relevant to your interests. 
          Please select whether we have your permission to 
          do this.
      </div>
      <div class="field"
           tal:define="emailPermission python:request.get('emailPermission', member.emailPermission);
                       tabindex tabindex/next;">
          <input type="radio" 
                 class="noborder" 
                 name="emailPermission" 
                 value="on"
                 id="cb_emailPermission" 
                 checked="checked"
                 tabindex=""
                 onfocus="formtooltip('permission_status_help',1)" 
                 onblur="formtooltip('permission_status_help',0)"
                 tal:attributes="tabindex tabindex;" 
                 tal:condition="emailPermission" 
                 />

          <input type="radio" 
                 class="noborder" 
                 name="emailPermission" 
                 value="on"
                 id="cb_emailPermission"
                 tabindex=""
                 onfocus="formtooltip('emailPermission_status_help',1)" 
                 onblur="formtooltip('emailPermission_status_help',0)"
                 tal:attributes="tabindex tabindex;" 
                 tal:condition="not: emailPermission" 
                 />

          <label for="cb_emailPermission" 
                 i18n:translate="label_member_emailPermission">
                     You may send alerts by email
          </label>

          <br />

          <input type="radio" 
                 class="noborder" 
                 name="emailPermission" 
                 value=""
                 id="cb_not_emailPermission"
                 tabindex=""
                 onfocus="formtooltip('emailPermission_status_help',1)" 
                 onblur="formtooltip('emailPermission_status_help',0)"
                 tal:attributes="tabindex tabindex;" 
                 tal:condition="emailPermission" 
                 />

          <input type="radio" 
                 class="noborder" 
                 name="emailPermission" 
                 value=""
                 id="cb_not_emailPermission" 
                 checked="checked"
                 tabindex=""
                 onfocus="formtooltip('emailPermission_status_help',1)" 
                 onblur="formtooltip('emailPermission_status_help',0)"
                 tal:attributes="tabindex tabindex;" 
                 tal:condition="not: emailPermission " 
                 />                

          <label for="cb_not_emailPermission">
              You may <em>not</em> send alerts by email
          </label>
      </div>
  </div>

Drop this in a suitable place in your customised form and test that your selection saves in your member data and is retrieved when you reload the form. I also added a customised member_search_results page to let me inspect all the member data while I was testing - you may find this useful too for diagnostics.

Enabling the Member Interest Selection

We're going to let members select their interests by means of checkboxes. This is a bit harder than radio buttons as we don't have existing form code to copy, but knowing how to retrieve data values, it's not hard.

Remember that the selections are going to end up as a list data type? Zope is going to help you out in a big way here. Zope has a wonderful shortcut to constructing list data - if you label your fields as: name="foo:list" Zope will auto-magically bundle all the data together and make it available as a list called foo. Neat, eh?

So all we have to do to save the data is make sure that all our checkboxes are named interest:list and when we submit, we'll get a list saved in the member's interest data element.

Retrieving the data is also pretty simple. We're using a basic Python list function for testing whether a value is a member of a list, and if it is, we're writing in the checked attribute. I've only shown three checkboxes, but you can have as many as you like in your own layout. As long as they're within the <div>, it'll work fine:

  <div class="row">
      <div class="label">Areas of Interest</div>
      <div class="field"
           tal:define="interestAreas python:request.get('interestAreas', member.interest)">
          <input type="checkbox" 
                 name="interest:list" 
                 value="advertising_promotion" 
                 tal:attributes="checked python:test('advertising_promotion' in interestAreas, 'checked', '')" />
                 Advertising &amp; Promotion
          <input type="checkbox" 
                 name="interest:list" 
                 value="brand_marketing"
                 tal:attributes="checked python:test('brand_marketing' in interestAreas, 'checked', '')" />
                 Brand Marketing
          <input type="checkbox" 
                 name="interest:list" 
                 value="category_development"
                 tal:attributes="checked python:test('category_development' in interestAreas, 'checked', '')" />
                 Category Development
      </div>
  </div>

Enabling the Content Metadata

This bit's really easy. Just take keywords you used in the checkboxes above and add them to the content, either through the properties tab when viewing Plone, or via the portal_metadata tool in the ZMI.

Note that these have to be exactly the field values you used for the checkboxes. An easy mistake is to use the field labels.

Workflow Scripting

Now that all the data's in place, we're on the home straight. All we need to do is add a script that compares the content and user data, checks that a member has given us permission to email them and fire off a few emails.

We'll do this in the standard workflow tool, which is the bundled DCWorkflow product. This is a 'states-and-transitions' type of workflow. You set up states that a content object can be in - with permissions attached to each state - and define transitions between those states. DCWorkflow also lets you apply scripts to execute before and/or after each transition, which is what we need.

Rather than simply adding a script to an existing transition, we're going to add a new state and transition specifically for email announcement. This will ensure that sending announcements is logged in the standard workflow audit trail, so we know whether a piece of content has been announced, and if so, when.

Adding a Workflow State

Go to the portal_workflow tool in the ZMI and select the Contents tab. This will give you the workflows that your site is currently using. Unless you've done any customisation already, you'll have 2: a folder workflow and a Plone workflow. The Plone workflow is the default one which controls most normal content, so it's this one we'll be editing.

Select that workflow and head to the States tab. Add a state called announced. This is the destination state that the content will be in after the emails have been sent. Set the permissions to a duplicate of the Published state. You need to make sure that your email recipients can view the content that you're announcing, so you'll want to set View and Access Contents Information permissions for Anonymous and Authenticated users, and once the content's announced, you don't want it being edited without further workflow, so only give the Manager role the permission of Modify Portal Content.

Next, select which transitions Announced content can then undergo. Again, I'd duplicate the Published state, and only permit the Reject and Retract transitions.

Adding a Workflow Transition

Back up to the Plone Workflow, and select the Transitions tab and add a new transition called announce. The important properties to set are that there's a role guard - only the Manager role should be able to send email announcements - and in the Display in actions box fields have a sensible name (eg Announce by email) and the category workflow. Also make sure that the destination state is announced and that the trigger is a user action. We'll add a workflow script after the script is set up.

Enabling the transition

Site managers will only be able to use the new transition if it's enabled as a permitted transition from an existing state. Go to the published state and add the announce transition to the possible transitions available from the published state.

Adding a Workflow Script

Workflow scripts can be Python scripts, page templates, DTML documents or any other executable content you can add via the ZMI. We're going to use a Python script, so go to the workflow's Script tab and add a new script called email_announce.

This will take one parameter, review_state (which refers to the transition currently underway). Here's the script. Note that as with all Python coding, the indenting is significant.

#This script has been designed to send email to cmf users 
#with appropriate preferences. The script should be used 
#in conjunction with the workflow tool.  
#parameters review_state
# Set up a empty list of email addresses
# loop through the portal membership, pass memberId to check for
# Member role. If successful, check to see if the member has given
# permission to send email, and an area of business interest that 
# coincides with a content keyword. If successful, append the 
# list of email addresses and send them email
# Get the content object we're publishing
contentObject = review_state.object
# A nifty little function, which checks to see whether there 
# are any elements that match between two lists, and returns 
# the number of matches. Result: if the function returns true, 
# you've got a match
def isIn(list1, list2):
     y=0
     for x in list1:
         if x in list2:
             y += 1
     return y
# Start with an empty list, ready to be filled with the addressed
# of people we're dispatching to
mailList=[]
# Iterate through all the site's users
for item in context.portal_membership.listMembers():
    memberId = item.id
    # Remember that a real name is not mandatory, so fall back to the username
    if item.fullname:
        memberName = item.fullname
    else:
        memberName = memberId
    # Get a list of this member's interests...
    memberInterests = item.interest
    # ...and another that's the keywords of this object
    contentKeywords = contentObject.subject
    # Check to see if there's a match between the two
    isInterestedIn = isIn(memberInterests, contentKeywords)
    # This is the key condition:
    # If the user has the Member role and
    #             we have an email address and
    #             the user's interested in this content and
    #             we have permission to email them
    if Member in context.portal_membership.getMemberById(memberId).getRoles() \
        and (item.email !='') and isInterestedIn and item.emailPermission:
        # add them to the list of people we're emailing
        mailList.append(item.email)
        # check that we can send email via the Zope standard Mail Host
        try:
           mailhost=getattr(context, context.portal_url.superValues(Mail Host)[0].id)
        except:
           raise AttributeError, "Cannot find a Mail Host object"
        # Let's write an email:
        mMsg = Dear  + memberName + ,\n\n
        mMsg += 'We thought you\d be interested in hearing about:\n
        mMsg += contentObject.TitleOrId() + \n\n
        mMsg += Description: \n + contentObject.Description() + \n\n
        mMsg += More info at:\n + contentObject.absolute_url() + \n
        mTo = item.email
        mFrom = you@yoursite.com
        mSubj = New Content available
        # and send it
        mailhost.send(mMsg, mTo, mFrom, mSubj)
# The change in indentation signals the end of the loop, so we've
# now sent all the emails. Let's now send a confirmation that we've done it.
# We'll be building the email as a string again, but we have to convert our
# list data elements into a string before we can append the information
recipients = string.join(mailList, sep=\n)
keywordsString = string.join(contentKeywords, sep=\n)
mTo = you@yourdomain.com
mMsg = The following people were sent a link to\n
mMsg += contentObject.absolute_url() + \n\n
mMsg += recipients + \n\n
mMsg += The keywords were:\n + keywordsString
mSubj = Content announcement email confirmation
mailhost.send(mMsg, mTo, mFrom, mSubj)

Once you have the script set up, go back to your announce transition and select it in the before transition slot - that way, you'll only complete the transition if the email all gets sent.

Additional Notes

Notes based on comments to this article:

• The personalize form is at /portal_skins/plone_prefs/personalize_form
• Sometimes the script text has dashed lines, for example, in email_announce.  According to vinsci on irc, "Wherever you see a dashed border, this actually means put single quotes before and after."
• Where the checkbox value="brand_marketing" (for example), you just go to the front page of the portal in Plone (not in the ZMI) as the admin user, click on the "Properties" tab (the one between Edit and Sharing) and add to the keywords there in the "New Keywords" box. Each keyword will be the value of the checkbox. So, the new keyword will be brand_marketing. Repeat for each checkbox value, and you're good!
• For the python script to run correctly you will need to add quotes to a lot of Mail Host and all of the msg messages.

Summary

That looks a lot, but it's not that much really. What we've done is:

1. Added a boolean email permission field to user data
2. Added a list-type area of interest field to user data
3. Amended the personalisation form so that users can store their preferences in the new fields
4. Added appropriate keywords to content
5. Added a new workflow state and transition to the workflow
6. Added a script to select suitable members and send them email announcing the new content

With a bit of modification, this could be modified to allow the content metadata to also be matched against user roles, which would help the site management define user groups in addition to user self-selection.