#7 — Curly quotes getting mangled when Descriptions are built

State Resolved
Version:
Area Functionality
Issue type Bug
Severity Medium
Submitted by Jon Stahl
Submitted on Nov 12, 2006
Responsible Maurits van Rees
Target release: 1.0


Using r33404, I see that feedfeeder is now populating the Description field with an excerpt of body text. Excellent!

However, if that body text happens to contain curly quotes or apostrophes, those are being turned into unicode entities in the description text, e.g.:

Christian “MrTopf” Scholz interviewed me at the tail end of the Plone Conference, and the video is now up on his blog. Not surprisingly, we chatted about the Plone Conference, the Plone community, and ONE/Northwest.

Instead of...

Christian "MrTopf" Scholz interviewed me at the tail end of the Plone Conference, and the video is now up on his blog. Not surprisingly, we chatted about the Plone Conference, the Plone community, and ONE/Northwest.

Perhaps you can do some sort of appropriate text transform or filter when building the Description field?
Added by Sig on Jan 05, 2009 01:14 PM
The issue is not fully solved : sure the description does not display the entity codes when you view the feedfolder but the entity codes can still be seen in portlets (e.g. review portlet) and in a collection.
Would you reopen this issue and consider finding a "deeper" solution ?
Added by Sig on Jan 05, 2009 01:15 PM
Note that this happens when using feedfeeder 1.0RC3 on Plone 3.x
Added by Maurits van Rees on Jan 08, 2009 08:56 AM
Issue state: ResolvedConfirmed
Added by Reinout van Rees on Jan 23, 2007 10:45 AM
Target release: None1.0
Responsible manager: (UNASSIGNED)reinout
Opening the issue. Should be fixed in 1.0
Added by Reinout van Rees on Jan 23, 2007 11:12 AM
Issue state: unconfirmedopen
now really opening it :-)
Added by (anonymous) on Jul 12, 2007 11:02 AM
any news about fixing this issue?
Added by Reinout van Rees on Sep 10, 2007 01:08 PM
I finally figured out where it is happening: in the library (feedparser) that we're using.

I haven't the time at the moment to delve deeper, so I'm leaving this comment here for now. It could be either a genuine bug in feedparser or it could be the way we display the description field. Perhaps we just need to put "structure" in the tal:content.

Do you see the &abcd; verbatim in the rendered page?
Added by Reinout van Rees on Sep 10, 2007 05:57 PM
Issue state: openresolved
I applied the "structure" fix on trunk. Closing the issue. If it doesn't solve it for you, could you reopen it?

(I assume svn trunk is ok for you, that is)
Added by Maurits van Rees on Feb 18, 2009 12:02 AM
Issue state: ConfirmedIn progress
Responsible manager: reinoutmaurits
[Note that the order of responses in this issue has gone wrong; look at the dates instead.]

We could do a transform from html to plain text before storing the summary/description. Or use stripogram, as stated in one of the responses to issue #1.
Added by Maurits van Rees on May 11, 2009 11:18 PM
I started a branch to try to resolve this better:
http://svn.plone.org/[…]/maurits-cleaner-entityrefs-in-description

This changeset more or less does the trick: r85867. But I am not completely happy with it; it feels a bit hacky.

If someone wants to play around with this, please also take a look at 'old_handle_entityref' and 'debug_convert_summary' which are not used but can be handy during testing.
Added by Maurits van Rees on Sep 22, 2009 12:34 AM
Issue state: In progressResolved
I merged the branch and released Products.feedfeeder 1.0rc on pypi. Please test.

No responses can be added.