Adding Charts To Your Custom Content Type

Step-by-step instructions for adding Python-generated charts to your custom Archetype. I describe a general method that allows you to use any Python graphing libraries that output images (png, gif, jpg, etc.). I use PyChart in this specific example.

Introduction

Ok, so you want to add graphs to your custom content type, but you're not sure exactly how to do it. You may have searched for 3rd party Zope products to do the job, but you've found they're out of date, or don't do exactly what you want to do. On the other hand, you already know how to generate graphs with a favourite Python graphing package (or at least you know they exist). You just want to know how to use these packages with Plone, and generate graphs for your content views. This How-To will discuss the background concepts, then show you an example in a custom Archetype.

Storing Images On The ZODB

ZODB Background

If you haven't noticed by now, Zope is built upon the ZODB (which I pronounce "ZopeDB"). This is a persistent database, that magically keeps all of your objects around. Any time the server is up, your objects are around, as you last left them. This really simplifies your programming model, since your objects are persistent. You don't have to explicitly pull data out of files, or relational databases. You're objects are just there.

As you may know, this ZODB works great for pickleable objects, but things like real files (i.e., things on the real file-system) require special steps to access, like external methods. The point is, there's ususally no real reason to use real files. In fact, it's preferred to keep things in the ZODB instead.

StringIO Files

Ok, if we're not going to use real files, how do we store the image files that the graphics programs want to generate? For example, how do we handle PyChart's canvas.init(outfile) call? The answer is simple, we use StringIO objects. These behave like files, but are pickleable, and work with the ZODB.

Originally, I tried using the Image object from Products.CMFDefault.Image, but I got errors because the object was not file-like. I found that the StringIO class works fine, except that I have to manually set the MIME type (e.g., to 'image/png') Otherwise, the AT mutuator for the ImageField sets it to 'application/octet-stream'.

Python Graphing Packages

There are several full-featured graphing libraries for Python. I personally use PyChart, for no special reason, and I'm satisfied with the flexibility and power. It makes great pie charts with minimal work. Other people may favor Matplotlib or GNUPlot for Python. Here are some links: PyChartMatplotlibGNUPlot.py

Installation

To install the graphics packages, you'll mostly follow the package-specific instructions. But remember, you're installing the package into your Zope's python (in the Zope installation directory tree) and not your system's python (e.g., /usr/bin/python, /sw/bin/python, etc.). Typically, a python package is set up so you can simply run

$ZOPEBIN/python setup.py install
as user from the package folder, where the ZOPEBIN environment variable is set to the path of python in your Zope installation tree.

Your graphics package may require other graphics libraries like PIL (Python Image Library). If they're not already installed in your Zope instance, then install them following the directions as above.

Example Code

All anyone really needs are commented code examples. This content type contains an ImageField, and the MakePlot() function generates a bar graph.

from Products.Archetypes.public import BaseContent, BaseSchema, Schema, registerType
from Products.Archetypes.public import ImageField
from cStringIO import StringIO
# import from the graphics package
from pychart import theme, canvas, area, axis, bar_plot, error_bar

class MyContentType (BaseContent):
""" An example custom Archetype that generates a PNG graph that's viewed
correctly by all browsers. Call PlotImage(), then look at base_view,
or aim the browser directly at the URL of the myImage.
"""
# Add an ImageField to your schema
# Read up on the deails of ImageField, it has nice features like scaling
schema = BaseSchema.copy() + Schema((
ImageField('myImage',
default_content_type='image/png',
widget=ImageWidget(label="PyChart Generated Image",),
)
))

def MakePlot (self,):
""" Call this function (e.g., by URL from the browser) to draw the plot.
After you call this function, you can see the plot on the base_view page,
or directly by URL. """

# use a StringIO to save the image, because it must be pickleable for the ZODB
imageFile = StringIO()
# show that the mimetype starts out as image/png
print "default content type", self.schema['myImage'].getContentType()

# This is a PyChart example that makes a Bar Plot
# replace this section with your graphing code
theme.use_color = True
theme.output_format="png"
theme.reinitialize()
chart = canvas.init(imageFile) # output the image to our imageFile
data = [(10, 20, 5, 5), (20, 65, 5, 5),
(30, 55, 4, 4), (40, 45, 2, 2), (50, 25, 3, 3)]
ar = area.T(x_axis = axis.X(label = "X label"),
y_axis = axis.Y(label = "Y label"))
ar.add_plot(bar_plot.T(label="foo", data = data,
fill_style = None,
error_bar = error_bar.bar3,
error_minus_col = 2,
error_plus_col = 3))
ar.draw(chart) # draw the area onto our canvas
chart.close() # close the output file

# use the built-in mutator
self.setMyImage(imageFile)
# show that the mutator changed the content type
print "content type after mutator", self.schema['myImage'].getContentType()
# finally, set the correct mimetype (note the first argument)
self.schema['myImage'].setContentType(instance=self, value='image/png')

RegisterType(MyContentType,'YourProductName')

Matplotlib adaptation

Posted by Yves Moisan at Jun 02, 2005 08:32 PM
Thanx for this How To! I'd been using an external method to gain access to Matplotlib and I knew I had to shove it into AT. Here are the modifications I did to it to integrate matplotlib. Four comments :

1) AFAIK, matplotlib's savefig does not support writing to a StringIO object ;( You have to use a temporary file.

2) Instead of resetting the mimetype in a separate line, you can force it in the setMyImage method (see code below).

3) I had to add ImageWidget to the from Products.Archetypes.public import ImageField line

4) Change PlotImage for MakePlot in the doc string

Also, for those for whom accessing the method via a URL is not all that clear (like me!), you create a new instance of your object, then go .../myObject/MakePlot and just go to .../muObject/base_view to see your image.

Thanx again!

Yves Moisan

from cStringIO import StringIO
# import from the graphics package
import matplotlib
matplotlib.use('Agg')
from pylab import *

class mpl (BaseContent):
  """ An example custom Archetype that generates a PNG graph that's viewed
      correctly by all browsers. Call PlotImage(), then look at base_view,
      or aim the browser directly at the URL of the myImage.
  """
  # Add an ImageField to your schema
  # Read up on the deails of ImageField, it has nice features like scaling
  schema = BaseSchema.copy() + Schema((
    ImageField('myImage',
               default_content_type='image/png',
               widget=ImageWidget(label="Image générée par Matplotlib",),
              )
  ))
  def MakePlot (self,):
    """ Call this function (e.g., by URL from the browser) to draw the plot.
        After you call this function, you can see the plot on the base_view page,
        or directly by URL. """

    #imageFile = StringIO()
    # show that the mimetype starts out as image/png
    #print "default content type", self.schema['myImage'].getContentType()
    
    
    t = arange(0.0, 2.0, 0.01)
    s = sin(2*pi*t)
    plot(t, s, linewidth=1.0)

    xlabel('time (s)')
    ylabel('voltage (mV)')
    title('Graphique dans Archetypes')
    grid(True)
    #savefig(imageFile)
    savefig("C:\\temp\\test.png")
    fh = open("C:\\temp\\test.png", "rb")
    data = fh.read()
    self.setMyImage(data,mimetype='image/png')
    #self.schema['myImage'].setContentType(instance=self, value='image/png')
    fh.close()
    clf()
    close("all")

Sorry about CR/LF

Posted by Yves Moisan at Jun 02, 2005 08:34 PM
I cut and pasted from Notepad++ and apparently that screwed CR/LF. Sorry about that.

No pylab for web applications

Posted by Yves Moisan at Jun 08, 2005 01:25 PM
Following comments on the mpl site, pylab should not be used for a web application (like I showed in my reply to this How To). So I used the code in the webapp_demo.py file in the examples dir of mpl, which basically amounts to replacing the imports. Not using pylab did make a few things not work (e.g. I had to explicitly "import numarray as nx" insterad of "import numerix as nx", as shown in the webapp_demo.py file), but overall it wroks well. Last glitch is to remove the need to write the PNG to a temp file ...

Removing the temp file...

Posted by SaschaGL at Jun 21, 2005 06:49 PM
On page 61 of the Matplotlib user's guide I found the following code:

import sys
import matplotlib
matplotlib.use('Agg')
from pylab import *
plot([1, 2, 3])
savefig(sys.stdout)

So it seems possible to get rid of the temp file... haven't tried myself, though.

Hope this helps.

Sascha

Still need temp, but cleanly done

Posted by Yves Moisan at Aug 18, 2005 06:32 PM
Sasha,

Since there seems we can't avoid using a temp file, I used the tempfile module to get sth cleaner. First I tried :

...
    canvas = FigureCanvasAgg(fig)
    tempplotfilename = tempfile.TemporaryFile(suffix='.png')
    canvas.print_figure(tempplotfilename.name, dpi=150)
...

but since I'm on windows this won't work because print_figure opens the file and the file is already open and "Whether the name can be used to open the file a second time, while the named temporary file is still open, varies across platforms (it can be so used on Unix; it cannot on Windows NT or later"

so I ended up with :

    canvas = FigureCanvasAgg(fig)
    tempplotfilename = tempfile.mkstemp(suffix='.png')
    canvas.print_figure(tempplotfilename[1], dpi=150)
    data = os.read(tempplotfilename[0],os.fstat(tempplotfilename[0]).st_size)
    self.setLastPlot(data, mimetype='image/png')
    os.close(tempplotfilename[0])
    os.unlink(tempplotfilename[1])

which transparently creates a temp file and cleans itself. Under Unix, TemporaryFile will make things easier.

Cheers!

Yves

*Really* no more temp file...

Posted by SaschaGL at Sep 02, 2005 09:53 PM
Thanks to a tip from Nicolas Young on the MPL mailing list, my app now does no longer need a temp file.

Here's how it's being done:

canvas = FigureCanvasAgg(fig)
canvas.draw()
size = canvas.get_width_height()
buf=canvas.tostring_rgb()
im=PILImage.fromstring('RGB', size, buf, 'raw', 'RGB', 0, 1)
imdata=StringIO()
im.save(imdata, format)

imdata now contains the PNG image data that can be rendered on a page.

Hope this helps,
Sascha

Urgh... forgot something...

Posted by SaschaGL at Sep 02, 2005 09:58 PM
fig contains the current figure (e.g. fig=pylab.gca() or fig=Figure() etc.) and format is a string (e.g. 'PNG' or 'JPG').

Sascha

Update chart when accessing the image field

Posted by lpiguet at Dec 12, 2005 03:45 PM
Thank you for this howto that was quite usefull. I managed to implement pychart and chartDirector libraries in my product type. I would like to have the image up-dated every time I get the object. I made new accessor for the image field and called the MakePlot() method before getting the image field.
It seems that the standard mutator called in MakePlot() to up-date the field content calls somehow the accessor, the code enters therefore in an endless recursion loop. Do you have a solution for this problem?