#7 — Curly quotes getting mangled when Descriptions are built

State	Resolved
Version:
Area	Functionality
Issue type	Bug
Severity	Medium
Submitted by	Jon Stahl
Submitted on	Nov 12, 2006
Responsible	Maurits van Rees
Target release:	1.0

Return to tracker

Last modified on Sep 22, 2009 by Maurits van Rees

Using r33404, I see that feedfeeder is now populating the Description field with an excerpt of body text. Excellent!

However, if that body text happens to contain curly quotes or apostrophes, those are being turned into unicode entities in the description text, e.g.:

Christian “MrTopf” Scholz interviewed me at the tail end of the Plone Conference, and the video is now up on his blog. Not surprisingly, we chatted about the Plone Conference, the Plone community, and ONE/Northwest.

Instead of...

Christian "MrTopf" Scholz interviewed me at the tail end of the Plone Conference, and the video is now up on his blog. Not surprisingly, we chatted about the Plone Conference, the Plone community, and ONE/Northwest.

Perhaps you can do some sort of appropriate text transform or filter when building the Description field?

Added bySigonJan 05, 2009 01:14 PM

The issue is not fully solved : sure the description does not display the entity codes when you view the feedfolder but the entity codes can still be seen in portlets (e.g. review portlet) and in a collection.
Would you reopen this issue and consider finding a "deeper" solution ?

Added bySigonJan 05, 2009 01:15 PM

Note that this happens when using feedfeeder 1.0RC3 on Plone 3.x

Added byMaurits van ReesonJan 08, 2009 08:56 AM

Issue state: Resolved → Confirmed

Added byReinout van ReesonJan 23, 2007 10:45 AM

Target release: None → 1.0

Responsible manager: (UNASSIGNED) → reinout

Opening the issue. Should be fixed in 1.0

Added byReinout van ReesonJan 23, 2007 11:12 AM

Issue state: unconfirmed → open

now really opening it :-)

Added by(anonymous)onJul 12, 2007 11:02 AM

any news about fixing this issue?

Added byReinout van ReesonSep 10, 2007 01:08 PM

I finally figured out where it is happening: in the library (feedparser) that we're using.

I haven't the time at the moment to delve deeper, so I'm leaving this comment here for now. It could be either a genuine bug in feedparser or it could be the way we display the description field. Perhaps we just need to put "structure" in the tal:content.

Do you see the &abcd; verbatim in the rendered page?

Added byReinout van ReesonSep 10, 2007 05:57 PM

Issue state: open → resolved

I applied the "structure" fix on trunk. Closing the issue. If it doesn't solve it for you, could you reopen it?

(I assume svn trunk is ok for you, that is)

Added byMaurits van ReesonFeb 18, 2009 12:02 AM

Issue state: Confirmed → In progress

Responsible manager: reinout → maurits

[Note that the order of responses in this issue has gone wrong; look at the dates instead.]

We could do a transform from html to plain text before storing the summary/description. Or use stripogram, as stated in one of the responses to issue #1.

Added byMaurits van ReesonMay 11, 2009 11:18 PM

I started a branch to try to resolve this better:
http://svn.plone.org/[…]/maurits-cleaner-entityrefs-in-description

This changeset more or less does the trick: r85867. But I am not completely happy with it; it feels a bit hacky.

If someone wants to play around with this, please also take a look at 'old_handle_entityref' and 'debug_convert_summary' which are not used but can be handy during testing.

Added byMaurits van ReesonSep 22, 2009 12:34 AM

Issue state: In progress → Resolved

I merged the branch and released Products.feedfeeder 1.0rc on pypi. Please test.

No responses can be added.