#126 — UnicodeDecodeError in PT_pt_render

by (anonymous) — last modified Feb 28, 2009 06:52 AM
State Resolved
Version: 1.1.1
Area Functionality
Issue type Bug
Severity Medium
Submitted by (anonymous)
Submitted on Feb 15, 2008
Responsible Ricardo Newbery
Target release: 1.2.1
Return to tracker
With CacheFu installed and enabled:

2008-02-15 19:36:09 ERROR Zope.SiteErrorLog http://.../collage_view
Traceback (innermost last):
  Module ZPublisher.Publish, line 119, in publish
  Module ZPublisher.mapply, line 88, in mapply
  Module ZPublisher.Publish, line 42, in call_object
  Module Shared.DC.Scripts.Bindings, line 313, in __call__
  Module Shared.DC.Scripts.Bindings, line 350, in _bindAndExec
  Module Products.CMFCore.FSPageTemplate, line 216, in _exec
  Module Products.CacheSetup.patch_cmf, line 48, in FSPT_pt_render
  Module Products.CacheSetup.patch_cmf, line 120, in PT_pt_render
  Module zope.tal.talinterpreter, line 271, in __call__
  Module zope.tal.talinterpreter, line 346, in interpret
  Module zope.tal.talinterpreter, line 891, in do_useMacro
  Module zope.tal.talinterpreter, line 346, in interpret
  Module zope.tal.talinterpreter, line 536, in do_optTag_tal
  Module zope.tal.talinterpreter, line 521, in do_optTag
  Module zope.tal.talinterpreter, line 516, in no_tag
  Module zope.tal.talinterpreter, line 346, in interpret
  Module zope.tal.talinterpreter, line 957, in do_defineSlot
  Module zope.tal.talinterpreter, line 346, in interpret
  Module zope.tal.talinterpreter, line 536, in do_optTag_tal
  Module zope.tal.talinterpreter, line 521, in do_optTag
  Module zope.tal.talinterpreter, line 516, in no_tag
  Module zope.tal.talinterpreter, line 346, in interpret
  Module zope.tal.talinterpreter, line 861, in do_defineMacro
  Module zope.tal.talinterpreter, line 346, in interpret
  Module zope.tal.talinterpreter, line 957, in do_defineSlot
  Module zope.tal.talinterpreter, line 346, in interpret
  Module zope.tal.talinterpreter, line 536, in do_optTag_tal
  Module zope.tal.talinterpreter, line 521, in do_optTag
  Module zope.tal.talinterpreter, line 516, in no_tag
  Module zope.tal.talinterpreter, line 346, in interpret
  Module zope.tal.talinterpreter, line 536, in do_optTag_tal
  Module zope.tal.talinterpreter, line 525, in do_optTag
  Module zope.tal.talinterpreter, line 346, in interpret
  Module zope.tal.talinterpreter, line 949, in do_defineSlot
  Module zope.tal.talinterpreter, line 346, in interpret
  Module zope.tal.talinterpreter, line 861, in do_defineMacro
  Module zope.tal.talinterpreter, line 346, in interpret
  Module zope.tal.talinterpreter, line 536, in do_optTag_tal
  Module zope.tal.talinterpreter, line 521, in do_optTag
  Module zope.tal.talinterpreter, line 516, in no_tag
  Module zope.tal.talinterpreter, line 346, in interpret
  Module zope.tal.talinterpreter, line 534, in do_optTag_tal
  Module zope.tal.talinterpreter, line 516, in no_tag
  Module zope.tal.talinterpreter, line 346, in interpret
  Module zope.tal.talinterpreter, line 745, in do_insertStructure_tal
  Module Products.PageTemplates.Expressions, line 221, in evaluateStructure
  Module zope.tales.tales, line 696, in evaluate
   - URL: file:/.../parts/development-products/Collage/skins/Collage/collage_view.pt
   - Line 57, Column 5
   - Expression: <PathExpr standard:u'here/@@renderer'>
   - Names:
      {'container': <PloneSite at /...>,
       'context': <Collage at /.../index>,
       'default': <object object at 0xb7d32528>,
       'here': <Collage at /.../index>,
       'loop': {},
       'nothing': None,
       'options': {'args': ()},
       'repeat': <Products.PageTemplates.Expressions.SafeMapping object at 0xb01de6ac>,
       'request': <HTTPRequest, URL=http://.../collage_view>,
       'root': <Application at >,
       'template': <FSPageTemplate at /.../collage_view used for /.../index>,
       'traverse_subpath': [],
       'user': <PropertiedUser 'admin'>}
  Module zope.tales.expressions, line 217, in __call__
  Module Products.PageTemplates.Expressions, line 161, in _eval
  Module Products.PageTemplates.Expressions, line 123, in render
  Module Products.Five.browser.metaconfigure, line 417, in __call__
  Module Shared.DC.Scripts.Bindings, line 313, in __call__
  Module Shared.DC.Scripts.Bindings, line 350, in _bindAndExec
  Module Products.PageTemplates.PageTemplateFile, line 129, in _exec
  Module Products.CacheSetup.patch_cmf, line 120, in PT_pt_render
  Module zope.tal.talinterpreter, line 271, in __call__
  Module zope.tal.talinterpreter, line 346, in interpret
  Module zope.tal.talinterpreter, line 824, in do_loop_tal
  Module zope.tal.talinterpreter, line 346, in interpret
  Module zope.tal.talinterpreter, line 534, in do_optTag_tal
  Module zope.tal.talinterpreter, line 516, in no_tag
  Module zope.tal.talinterpreter, line 346, in interpret
  Module zope.tal.talinterpreter, line 745, in do_insertStructure_tal
  Module Products.PageTemplates.Expressions, line 221, in evaluateStructure
  Module zope.tales.tales, line 696, in evaluate
   - URL: index
   - Line 2, Column 2
   - Expression: <PathExpr standard:'item'>
   - Names:
      {'container': <Collage at /.../index>,
       'context': <Collage at /.../index>,
       'default': <object object at 0xb7d32528>,
       'here': <Collage at /.../index>,
       'loop': {'item': <Products.PageTemplates.Expressions.PathIterator object at 0xb0674f2c>},
       'nothing': None,
       'options': {'args': (<Products.Five.metaclass.SimpleViewClass from /.../parts/development-products/Collage/browser/templates/collage_renderer.pt object at 0xb063452c>,)},
       'repeat': <Products.PageTemplates.Expressions.SafeMapping object at 0xb0665f0c>,
       'request': <HTTPRequest, URL=http://.../index/collage_view>,
       'root': <Application at >,
       'template': <ImplicitAcquirerWrapper object at 0xb06345ac>,
       'traverse_subpath': [],
       'user': <PropertiedUser 'admin'>,
       'view': <Products.Five.metaclass.SimpleViewClass from /.../parts/development-products/Collage/browser/templates/collage_renderer.pt object at 0xb063452c>,
       'views': <zope.app.pagetemplate.viewpagetemplatefile.ViewMapper object at 0xb0674a0c>}
  Module zope.tales.expressions, line 217, in __call__
  Module Products.PageTemplates.Expressions, line 161, in _eval
  Module Products.PageTemplates.Expressions, line 123, in render
  Module Products.Five.browser.metaconfigure, line 417, in __call__
  Module Shared.DC.Scripts.Bindings, line 313, in __call__
  Module Shared.DC.Scripts.Bindings, line 350, in _bindAndExec
  Module Products.PageTemplates.PageTemplateFile, line 129, in _exec
  Module Products.CacheSetup.patch_cmf, line 74, in PT_pt_render
  Module Products.CacheSetup.patch_utils, line 9, in call_pattern
  Module Products.PageTemplates.PageTemplate, line 89, in pt_render
  Module zope.pagetemplate.pagetemplate, line 117, in pt_render
  Module zope.tal.talinterpreter, line 271, in __call__
  Module zope.tal.talinterpreter, line 346, in interpret
  Module zope.tal.talinterpreter, line 536, in do_optTag_tal
  Module zope.tal.talinterpreter, line 521, in do_optTag
  Module zope.tal.talinterpreter, line 516, in no_tag
  Module zope.tal.talinterpreter, line 346, in interpret
  Module zope.tal.talinterpreter, line 824, in do_loop_tal
  Module zope.tal.talinterpreter, line 346, in interpret
  Module zope.tal.talinterpreter, line 536, in do_optTag_tal
  Module zope.tal.talinterpreter, line 521, in do_optTag
  Module zope.tal.talinterpreter, line 516, in no_tag
  Module zope.tal.talinterpreter, line 346, in interpret
  Module zope.tal.talinterpreter, line 534, in do_optTag_tal
  Module zope.tal.talinterpreter, line 516, in no_tag
  Module zope.tal.talinterpreter, line 346, in interpret
  Module zope.tal.talinterpreter, line 745, in do_insertStructure_tal
  Module Products.PageTemplates.Expressions, line 221, in evaluateStructure
  Module zope.tales.tales, line 696, in evaluate
   - URL: index
   - Line 27, Column 6
   - Expression: <PythonExpr ( renderer(columns))>
   - Names:
      {'container': <CollageRow at /.../index/2>,
       'context': <CollageRow at /.../index/2>,
       'default': <object object at 0xb7d32528>,
       'here': <CollageRow at /.../index/2>,
       'loop': {'columns': <Products.PageTemplates.Expressions.PathIterator object at 0xb065e4ac>},
       'nothing': None,
       'options': {'args': (<Products.Five.metaclass.SimpleViewClass from /.../parts/development-products/Collage/browser/views/row_largeleft.pt object at 0xb0662d4c>,)},
       'repeat': <Products.PageTemplates.Expressions.SafeMapping object at 0xb0662f0c>,
       'request': <HTTPRequest, URL=http://.../index/collage_view>,
       'root': <Application at >,
       'template': <ImplicitAcquirerWrapper object at 0xb0662a2c>,
       'traverse_subpath': [],
       'user': <PropertiedUser 'admin'>,
       'view': <Products.Five.metaclass.SimpleViewClass from /.../parts/development-products/Collage/browser/views/row_largeleft.pt object at 0xb0662d4c>,
       'views': <zope.app.pagetemplate.viewpagetemplatefile.ViewMapper object at 0xb0662d8c>}
  Module zope.tales.pythonexpr, line 59, in __call__
   - __traceback_info__: ( renderer(columns))
  Module <string>, line 0, in ?
  Module Products.Five.browser.metaconfigure, line 417, in __call__
  Module Shared.DC.Scripts.Bindings, line 313, in __call__
  Module Shared.DC.Scripts.Bindings, line 350, in _bindAndExec
  Module Products.PageTemplates.PageTemplateFile, line 129, in _exec
  Module Products.CacheSetup.patch_cmf, line 120, in PT_pt_render
  Module zope.tal.talinterpreter, line 271, in __call__
  Module zope.tal.talinterpreter, line 346, in interpret
  Module zope.tal.talinterpreter, line 824, in do_loop_tal
  Module zope.tal.talinterpreter, line 346, in interpret
  Module zope.tal.talinterpreter, line 534, in do_optTag_tal
  Module zope.tal.talinterpreter, line 516, in no_tag
  Module zope.tal.talinterpreter, line 346, in interpret
  Module zope.tal.talinterpreter, line 745, in do_insertStructure_tal
  Module Products.PageTemplates.Expressions, line 221, in evaluateStructure
  Module zope.tales.tales, line 696, in evaluate
   - URL: index
   - Line 3, Column 4
   - Expression: <PathExpr standard:'item'>
   - Names:
      {'container': <CollageRow at /.../index/2>,
       'context': <CollageRow at /.../index/2>,
       'default': <object object at 0xb7d32528>,
       'here': <CollageRow at /.../index/2>,
       'loop': {'item': <Products.PageTemplates.Expressions.PathIterator object at 0xb06545ec>},
       'nothing': None,
       'options': {'args': (<Products.Five.metaclass.SimpleViewClass from /.../parts/development-products/Collage/browser/templates/row_renderer.pt object at 0xb06626ec>,
                            [<Products.ZCatalog.Catalog.mybrains object at 0xb0688bbc>,
                             <Products.ZCatalog.Catalog.mybrains object at 0xb06882fc>])},
       'repeat': <Products.PageTemplates.Expressions.SafeMapping object at 0xb065ef2c>,
       'request': <HTTPRequest, URL=http://.../index/collage_view>,
       'root': <Application at >,
       'template': <ImplicitAcquirerWrapper object at 0xb065e92c>,
       'traverse_subpath': [],
       'user': <PropertiedUser 'admin'>,
       'view': <Products.Five.metaclass.SimpleViewClass from /.../parts/development-products/Collage/browser/templates/row_renderer.pt object at 0xb06626ec>,
       'views': <zope.app.pagetemplate.viewpagetemplatefile.ViewMapper object at 0xb065ee4c>}
  Module zope.tales.expressions, line 217, in __call__
  Module Products.PageTemplates.Expressions, line 161, in _eval
  Module Products.PageTemplates.Expressions, line 123, in render
  Module Products.Five.browser.metaconfigure, line 417, in __call__
  Module Shared.DC.Scripts.Bindings, line 313, in __call__
  Module Shared.DC.Scripts.Bindings, line 350, in _bindAndExec
  Module Products.PageTemplates.PageTemplateFile, line 129, in _exec
  Module Products.CacheSetup.patch_cmf, line 74, in PT_pt_render
  Module Products.CacheSetup.patch_utils, line 9, in call_pattern
  Module Products.PageTemplates.PageTemplate, line 89, in pt_render
  Module zope.pagetemplate.pagetemplate, line 117, in pt_render
  Module zope.tal.talinterpreter, line 271, in __call__
  Module zope.tal.talinterpreter, line 346, in interpret
  Module zope.tal.talinterpreter, line 534, in do_optTag_tal
  Module zope.tal.talinterpreter, line 516, in no_tag
  Module zope.tal.talinterpreter, line 346, in interpret
  Module zope.tal.talinterpreter, line 745, in do_insertStructure_tal
  Module Products.PageTemplates.Expressions, line 221, in evaluateStructure
  Module zope.tales.tales, line 696, in evaluate
   - URL: index
   - Line 21, Column 0
   - Expression: <PathExpr standard:'here/@@renderer'>
   - Names:
      {'container': <CollageColumn at /.../index/2/1>,
       'context': <CollageColumn at /.../index/2/1>,
       'default': <object object at 0xb7d32528>,
       'here': <CollageColumn at /.../index/2/1>,
       'loop': {},
       'nothing': None,
       'options': {'args': (<Products.Five.metaclass.SimpleViewClass from /.../parts/development-products/Collage/browser/views/column_standard.pt object at 0xb065432c>,)},
       'repeat': <Products.PageTemplates.Expressions.SafeMapping object at 0xb064f1cc>,
       'request': <HTTPRequest, URL=http://.../index/collage_view>,
       'root': <Application at >,
       'template': <ImplicitAcquirerWrapper object at 0xb06659ac>,
       'traverse_subpath': [],
       'user': <PropertiedUser 'admin'>,
       'view': <Products.Five.metaclass.SimpleViewClass from /.../parts/development-products/Collage/browser/views/column_standard.pt object at 0xb065432c>,
       'views': <zope.app.pagetemplate.viewpagetemplatefile.ViewMapper object at 0xb064f96c>}
  Module zope.tales.expressions, line 217, in __call__
  Module Products.PageTemplates.Expressions, line 161, in _eval
  Module Products.PageTemplates.Expressions, line 123, in render
  Module Products.Five.browser.metaconfigure, line 417, in __call__
  Module Shared.DC.Scripts.Bindings, line 313, in __call__
  Module Shared.DC.Scripts.Bindings, line 350, in _bindAndExec
  Module Products.PageTemplates.PageTemplateFile, line 129, in _exec
  Module Products.CacheSetup.patch_cmf, line 120, in PT_pt_render
  Module zope.tal.talinterpreter, line 271, in __call__
  Module zope.tal.talinterpreter, line 346, in interpret
  Module zope.tal.talinterpreter, line 824, in do_loop_tal
  Module zope.tal.talinterpreter, line 346, in interpret
  Module zope.tal.talinterpreter, line 534, in do_optTag_tal
  Module zope.tal.talinterpreter, line 516, in no_tag
  Module zope.tal.talinterpreter, line 346, in interpret
  Module zope.tal.talinterpreter, line 745, in do_insertStructure_tal
  Module Products.PageTemplates.Expressions, line 221, in evaluateStructure
  Module zope.tales.tales, line 696, in evaluate
   - URL: index
   - Line 2, Column 2
   - Expression: <PathExpr standard:'item'>
   - Names:
      {'container': <CollageColumn at /.../index/2/1>,
       'context': <CollageColumn at /.../index/2/1>,
       'default': <object object at 0xb7d32528>,
       'here': <CollageColumn at /.../index/2/1>,
       'loop': {'item': <Products.PageTemplates.Expressions.PathIterator object at 0xb0668b4c>},
       'nothing': None,
       'options': {'args': (<Products.Five.metaclass.SimpleViewClass from /.../parts/development-products/Collage/browser/templates/column_renderer.pt object at 0xb06a716c>,)},
       'repeat': <Products.PageTemplates.Expressions.SafeMapping object at 0xb06a5c8c>,
       'request': <HTTPRequest, URL=http://.../index/collage_view>,
       'root': <Application at >,
       'template': <ImplicitAcquirerWrapper object at 0xb06a7dcc>,
       'traverse_subpath': [],
       'user': <PropertiedUser 'admin'>,
       'view': <Products.Five.metaclass.SimpleViewClass from /.../parts/development-products/Collage/browser/templates/column_renderer.pt object at 0xb06a716c>,
       'views': <zope.app.pagetemplate.viewpagetemplatefile.ViewMapper object at 0xb06a5bac>}
  Module zope.tales.expressions, line 217, in __call__
  Module Products.PageTemplates.Expressions, line 161, in _eval
  Module Products.PageTemplates.Expressions, line 123, in render
  Module Products.Five.browser.metaconfigure, line 417, in __call__
  Module Shared.DC.Scripts.Bindings, line 313, in __call__
  Module Shared.DC.Scripts.Bindings, line 350, in _bindAndExec
  Module Products.PageTemplates.PageTemplateFile, line 129, in _exec
  Module Products.CacheSetup.patch_cmf, line 74, in PT_pt_render
  Module Products.CacheSetup.patch_utils, line 9, in call_pattern
  Module Products.PageTemplates.PageTemplate, line 89, in pt_render
  Module zope.pagetemplate.pagetemplate, line 117, in pt_render
  Module zope.tal.talinterpreter, line 271, in __call__
  Module zope.tal.talinterpreter, line 346, in interpret
  Module zope.tal.talinterpreter, line 824, in do_loop_tal
  Module zope.tal.talinterpreter, line 346, in interpret
  Module zope.tal.talinterpreter, line 536, in do_optTag_tal
  Module zope.tal.talinterpreter, line 521, in do_optTag
  Module zope.tal.talinterpreter, line 516, in no_tag
  Module zope.tal.talinterpreter, line 346, in interpret
  Module zope.tal.talinterpreter, line 536, in do_optTag_tal
  Module zope.tal.talinterpreter, line 521, in do_optTag
  Module zope.tal.talinterpreter, line 516, in no_tag
  Module zope.tal.talinterpreter, line 346, in interpret
  Module zope.tal.talinterpreter, line 534, in do_optTag_tal
  Module zope.tal.talinterpreter, line 516, in no_tag
  Module zope.tal.talinterpreter, line 346, in interpret
  Module zope.tal.talinterpreter, line 745, in do_insertStructure_tal
  Module Products.PageTemplates.Expressions, line 221, in evaluateStructure
  Module zope.tales.tales, line 696, in evaluate
   - URL: index
   - Line 11, Column 4
   - Expression: <PathExpr standard:'item_view'>
   - Names:
      {'container': <CustomType at /.../ZZZ>,
       'context': <CustomType at /.../ZZZ>,
       'default': <object object at 0xb7d32528>,
       'here': <CustomType at /.../ZZZ>,
       'loop': {'el': <Products.PageTemplates.Expressions.PathIterator object at 0xb06a5e2c>},
       'nothing': None,
       'options': {'args': (<Products.Five.metaclass.SimpleViewClass from /.../xyz.collage/xyz/collage/browser/views/news_listing.pt object at 0xb06a354c>,)},
       'repeat': <Products.PageTemplates.Expressions.SafeMapping object at 0xb06a592c>,
       'request': <HTTPRequest, URL=http://.../index/collage_view>,
       'root': <Application at >,
       'template': <ImplicitAcquirerWrapper object at 0xb06a37cc>,
       'traverse_subpath': [],
       'user': <PropertiedUser 'admin'>,
       'view': <Products.Five.metaclass.SimpleViewClass from /.../src/xyz.collage/xyz/collage/browser/views/news_listing.pt object at 0xb06a354c>,
       'views': <zope.app.pagetemplate.viewpagetemplatefile.ViewMapper object at 0xb06a5ccc>}
  Module zope.tales.expressions, line 217, in __call__
  Module Products.PageTemplates.Expressions, line 161, in _eval
  Module Products.PageTemplates.Expressions, line 123, in render
  Module Products.Five.browser.metaconfigure, line 417, in __call__
  Module Shared.DC.Scripts.Bindings, line 313, in __call__
  Module Shared.DC.Scripts.Bindings, line 350, in _bindAndExec
  Module Products.PageTemplates.PageTemplateFile, line 129, in _exec
  Module Products.CacheSetup.patch_cmf, line 149, in PT_pt_render
   - Warning: Macro expansion failed
   - Warning: exceptions.UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 1: ordinal not in range(128)
  Module StringIO, line 271, in getvalue
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 1: ordinal not in range(128)



With CacheFu not installed or not enabled, no problem. Therefore I suspect this *might* be a problem with CacheFu.

Plone 3.0.5, CacheFu from SVN.
Steps to reproduce:
I can reproduce. I can't provide a recipe to reproduce.

I would be happy to help debug. Just give me a bit of guidance.
Added by Ricardo Newbery on Apr 04, 2008 01:21 AM
Issue state: unconfirmed → postponed
Responsible manager: (UNASSIGNED) → newbery
Target release: 1.1.2 → None
I can't reproduce this. Thank you, anonymous, for offering to help debug... but you will need to log in and post another response before we can start as I don't know how to contact you.

Added by David Byers on Apr 04, 2008 05:50 AM
I forgot to log in? Silly me.

I have debugged a little further. The problem appears only with some of my custom views that contain non-ascii characters. They work without CacheFu though. In PT_pt_render, there is a code path where CacheFu renders the page, and I assume that is done differently in CacheFu than in Zope/CMF. I haven't been able to figure out how to change my views to avoid the problem.

It's hard to say if this is a bug in CacheFu or a bug in my views, but if it is the latter, then CacheFu still behaves sufficiently differently from stock Plone that it exposes problems that would otherwise be hidden, and I don't think that's a particularly good thing.
Added by Ricardo Newbery on Apr 20, 2008 07:35 AM
Issue state: postponed → in-progress
Target release: None → 1.1.2
David, can you try applying the fix in...

http://dev.plone.org/collective/changeset/63107

and let me know if it fixes this issue for you?

Added by Stefan H. Holek on May 09, 2008 04:14 PM
I can confirm this happening in Plone 3.0.6 + CacheSetup 1.2 with a ViewPageTemplateFile in UTF-8 format containing non-ascii characters. For me it happens in line 153 and the traceback is a lot shorter. :-)

2008-05-09T14:58:18 ERROR Zope.SiteErrorLog http://foo.com:8082/plone/profile_delete
Traceback (innermost last):
  Module ZPublisher.Publish, line 119, in publish
  Module ZPublisher.mapply, line 88, in mapply
  Module ZPublisher.Publish, line 42, in call_object
  Module Products.Breakfast.browser.profile, line 60, in __call__
  Module Shared.DC.Scripts.Bindings, line 313, in __call__
  Module Shared.DC.Scripts.Bindings, line 350, in _bindAndExec
  Module Products.PageTemplates.PageTemplateFile, line 129, in _exec
  Module Products.CacheSetup.patch_cmf, line 153, in PT_pt_render
   - Warning: Macro expansion failed
   - Warning: exceptions.UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 21: ordinal not in range(128)
  Module StringIO, line 271, in getvalue
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 21: ordinal not in range(128)
Added by Chan Phap Luu on Aug 11, 2008 08:45 PM
We have found a similar problem after enabling CacheSetup on our Plone site at http://deerparkmonastery.org.

Our setup:
    * Plone 3.1.3
    * CMF 2.1.1
    * Zope (Zope 2.10.6-final, python 2.4.5, linux2)
    * Python 2.4.5 (#1, Jul 25 2008, 23:09:10) [GCC 4.2.3 (Ubuntu 4.2.3-2ubuntu7)]
    * PIL 1.1.6

As soon as we enable CacheSetup in the configuration page, we cannot access our site except through the ZMI, and the following error comes up:

Site Error

An error was encountered while publishing this resource.

UnicodeDecodeError
Sorry, a site error occurred.

Traceback (innermost last):

    * Module ZPublisher.Publish, line 202, in publish_module_standard
    * Module Products.LinguaPlone.patches, line 67, in new_publish
    * Module ZPublisher.Publish, line 150, in publish
    * Module plone.app.linkintegrity.monkey, line 21, in zpublisher_exception_hook_wrapper
    * Module Zope2.App.startup, line 221, in zpublisher_exception_hook
    * Module ZPublisher.Publish, line 119, in publish
    * Module ZPublisher.mapply, line 88, in mapply
    * Module ZPublisher.Publish, line 42, in call_object
    * Module Products.CMFFormController.FSControllerPageTemplate, line 90, in __call__
    * Module Products.CMFFormController.BaseControllerPageTemplate, line 31, in _call
    * Module Shared.DC.Scripts.Bindings, line 313, in __call__
    * Module Shared.DC.Scripts.Bindings, line 350, in _bindAndExec
    * Module Products.CMFCore.FSPageTemplate, line 216, in _exec
    * Module Products.CacheSetup.patch_cmf, line 48, in FSPT_pt_render
    * Module Products.CacheSetup.patch_cmf, line 120, in PT_pt_render
      Warning: Macro expansion failed
      Warning: exceptions.KeyError: 'edit_macros'
    * Module zope.tal.talinterpreter, line 271, in __call__
    * Module zope.tal.talinterpreter, line 346, in interpret
    * Module zope.tal.talinterpreter, line 891, in do_useMacro
    * Module zope.tal.talinterpreter, line 346, in interpret
    * Module zope.tal.talinterpreter, line 536, in do_optTag_tal
    * Module zope.tal.talinterpreter, line 521, in do_optTag
    * Module zope.tal.talinterpreter, line 516, in no_tag
    * Module zope.tal.talinterpreter, line 346, in interpret
    * Module zope.tal.talinterpreter, line 891, in do_useMacro
    * Module zope.tal.talinterpreter, line 346, in interpret
    * Module zope.tal.talinterpreter, line 536, in do_optTag_tal
    * Module zope.tal.talinterpreter, line 521, in do_optTag
    * Module zope.tal.talinterpreter, line 516, in no_tag
    * Module zope.tal.talinterpreter, line 346, in interpret
    * Module zope.tal.talinterpreter, line 536, in do_optTag_tal
    * Module zope.tal.talinterpreter, line 521, in do_optTag
    * Module zope.tal.talinterpreter, line 516, in no_tag
    * Module zope.tal.talinterpreter, line 346, in interpret
    * Module zope.tal.talinterpreter, line 534, in do_optTag_tal
    * Module zope.tal.talinterpreter, line 516, in no_tag
    * Module zope.tal.talinterpreter, line 346, in interpret
    * Module zope.tal.talinterpreter, line 745, in do_insertStructure_tal
    * Module Products.PageTemplates.Expressions, line 221, in evaluateStructure
    * Module zope.tales.tales, line 696, in evaluate
      URL: file:/opt/Plone-3.1/dpm/parts/plone/CMFPlone/skins/plone_templates/main_template.pt
      Line 180, Column 8
      Expression: <StringExpr u'plone.portalfooter'>
      Names:

      {'container': <PloneSite at /Plone>,
       'context': <CacheTool at /Plone/portal_cache_settings>,
       'default': <object object at 0x7f1729a76200>,
       'here': <CacheTool at /Plone/portal_cache_settings>,
       'loop': {},
       'nothing': None,
       'options': {'args': (),
                   'state': <Products.CMFFormController.ControllerState.ControllerState object at 0xc4fac10>},
       'repeat': <Products.PageTemplates.Expressions.SafeMapping object at 0xc52ccf8>,
       'request': <HTTPRequest, URL=http://test.dpweb.org/[…]/cache_tool_config>,
       'root': <Application at >,
       'template': <FSControllerPageTemplate at /Plone/cache_tool_config used for /Plone/portal_cache_settings>,
       'traverse_subpath': [],
       'user': <PloneUser 'brstream'>}

    * Module Products.Five.browser.providerexpression, line 37, in __call__
    * Module plone.app.viewletmanager.manager, line 151, in render
    * Module plone.app.viewletmanager.manager, line 80, in render
    * Module zope.app.pagetemplate.simpleviewclass, line 44, in __call__
    * Module Shared.DC.Scripts.Bindings, line 313, in __call__
    * Module Shared.DC.Scripts.Bindings, line 350, in _bindAndExec
    * Module Products.PageTemplates.PageTemplateFile, line 129, in _exec
    * Module Products.CacheSetup.patch_cmf, line 153, in PT_pt_render
      Warning: Macro expansion failed
      Warning: exceptions.UnicodeDecodeError: 'ascii' codec can't decode byte 0xef in position 0: ordinal not in range(128)
    * Module StringIO, line 271, in getvalue

UnicodeDecodeError: 'ascii' codec can't decode byte 0xef in position 0: ordinal not in range(128) (Also, the following error occurred while attempting to render the standard error message, please see the event log for full details: 'ascii' codec can't decode byte 0xef in position 0: ordinal not in range(128))

We would be grateful for any help with this. Our site has LinguaPlone installed and is in three languages (English, Spanish and Vietnamese.)

Best,
Brother Phap Luu
Deer Park Monastery
Escondido, CA 92026
Added by esperanto on Aug 14, 2008 03:51 PM
I had the same problem with a copy sign in the footer. (version 1.2 of cachefu)
Added by Luca Fabbri on Nov 10, 2008 04:17 PM
Can someone give some more info about this? If the patch linked above still valid?

This is a very bad bug for non english users! If we can help in debugging we are here!
Added by Christoph Glaubitz on Nov 11, 2008 10:18 AM
Hi Luca,

the issue is still valid with the patch, linked above. This is the error message:

best regards, chris

2008-11-11 10:57:24 ERROR portlets Error while rendering <plone.app.portlets.manager.ColumnPortletManagerRenderer object at 0xd2e8bac>
Traceback (most recent call last):
  File "/home/joerg/.buildout/eggs/plone.app.portlets-1.1.3-py2.4.egg/plone/app/portlets/manager.py", line 65, in safe_render
    return portlet_renderer.render()
  File "/home/joerg/Instances/gt-portal-fischer/parts/zope2/lib/python/Shared/DC/Scripts/Bindings.py", line 313, in __call__
    return self._bindAndExec(args, kw, None)
  File "/home/joerg/Instances/gt-portal-fischer/parts/zope2/lib/python/Shared/DC/Scripts/Bindings.py", line 350, in _bindAndExec
    return self._exec(bound_data, args, kw)
  File "/home/joerg/Instances/gt-portal-fischer/parts/zope2/lib/python/Products/PageTemplates/PageTemplateFile.py", line 129, in _exec
    return self.pt_render(extra_context=bound_names)
  File "/home/joerg/.buildout/eggs/Products.CacheSetup-1.2-py2.4.egg/Products/CacheSetup/patch_cmf.py", line 154, in PT_pt_render
    result = output.getvalue()
  File "/home/joerg/local/Python-2.4.5/lib/python2.4/StringIO.py", line 271, in getvalue
    self.buf += ''.join(self.buflist)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 101: ordinal not in range(128)
2008-11-11 10:57:24 ERROR Zope.SiteErrorLog http://develop:8081/gt/startpage_view
Traceback (innermost last):
  Module plone.app.portlets.manager, line 65, in safe_render
  Module Shared.DC.Scripts.Bindings, line 313, in __call__
  Module Shared.DC.Scripts.Bindings, line 350, in _bindAndExec
  Module Products.PageTemplates.PageTemplateFile, line 129, in _exec
  Module Products.CacheSetup.patch_cmf, line 154, in PT_pt_render
   - Warning: Macro expansion failed
   - Warning: exceptions.UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 101: ordinal not in range(128)
  Module StringIO, line 271, in getvalue
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 101: ordinal not in range(128)
Added by Christoph Glaubitz on Nov 13, 2008 03:23 PM
Hi again Luca,

I have a quick and dirty solution for this Problem. I just used the FasterStrionIO out of Products.CMFPlone.patches.unicodehacks instead of the standard StringIO. FasterStringIO catches the exception and decodes the string in utf-8.

But I'm really not happy with this solution. Because the "wrong" data will be appended by the TALInterpreter :-(

Please have a look at the appended patch file and let me know what you think about it.

Thanks so far, chris
Added by Christoph Glaubitz on Nov 13, 2008 04:07 PM
Ok. One step back. In my case there was an german special character (not html-decoded) in a template.
TALInterpreter just read in a string, and mixes it with the utf-8 StringIO in patch_cmf.py. Without a special char, no error occures. However, in fact it was a defect in a self made template.

So... There is no defect in CacheFu. The implementation without the patch (http://dev.plone.org/collective/changeset/63107) should mostly work. Even, if there is no template, containing a character what is over ascii. The change to utf-8 does not solve the issue, because TALInterpreter still just read strings.

In my opinion, it would be great to see FasterStringIO in patch_cmf.py, because it's not easy to associate the error message with this marginal failure in the template.

chris
Added by Luca Fabbri on Nov 14, 2008 09:12 AM
Thank you for replying!

In any case we fixed the problem replacing the non ascii char with the HTML entity.

In the past I got some problem with using tal:content with unicode strings. For what I remember using tal:content="string:unicodevalue" raise errors. But using tal:content="python:unuicodevalue" works!

My only problem is that often we (or power users customer) build template with italian (non ascii) characters and in those cases we can rely directly on CacheFU is there is the possibility that enable this brakes the site!
Added by Rafael Cabral Coutinho on Jan 13, 2009 06:43 PM
Same problem here (Plone 3.1.7/CacheFu 1.2) in a viewlet template.

CMFPlone applies a patch in zope.pagetemplate to use its own version of the FasterStringIO, which is missing in the patch of the CacheFu, causing the error when a template with non-ascii chars is used. So, I really think CacheFu should use the FasterStringIO too.

Not sure, but I think that the encoding mixing happens due to Products.PageTemplate.PageTemplateFile not being unicode aware (which is used by Five when a viewlet is rendered), and FasterStringIO "solves" this mixing.
Added by Ricardo Newbery on Feb 28, 2009 06:52 AM
Issue state: In progress → Resolved
Target release: 1.1.2 → 1.2.1
Fixed in svn... http://dev.plone.org/collective/changeset/67165

No responses can be added.