#10 — get_size throws exception if the blob file is missing

by Brit Minor — last modified Feb 04, 2011 01:18 AM

State	Resolved
Version:	1.0
Area	Functionality
Issue type	Bug
Severity	Important
Submitted by	Brit Minor
Submitted on	Dec 16, 2008
Responsible	Andreas Zeidler
Target release:	1.0

Return to tracker

Due to migration errors or problems with file uploads in the past, we have cases where a physical blob file does not exist. This error is not handled by plone.app.blob.field.get_size(). A fix is easy enough: catch POSKeyError at line 66 in field.py and return 0.

Call stack:

2008-12-16T11:41:52 ERROR Zope.SiteErrorLog https://col3dev.tnc.org:54321/[…]/manage_reindexIndex
Traceback (innermost last):
  Module ZPublisher.Publish, line 119, in publish
  Module ZPublisher.mapply, line 88, in mapply
  Module ZPublisher.Publish, line 42, in call_object
  Module Products.ZCatalog.ZCatalog, line 501, in manage_reindexIndex
  Module Products.ZCatalog.ZCatalog, line 468, in reindexIndex
  Module Products.CMFPlone.CatalogTool, line 384, in catalog_object
  Module Products.ZCatalog.ZCatalog, line 535, in catalog_object
  Module Products.ZCatalog.Catalog, line 360, in catalogObject
  Module Products.PluginIndexes.common.UnIndex, line 235, in index_object
  Module Products.PluginIndexes.common.UnIndex, line 244, in _index_object
  Module Products.PluginIndexes.common.UnIndex, line 274, in _get_object_datum
  Module Products.CMFPlone.CatalogTool, line 110, in __getattr__
  Module Products.CMFPlone.CatalogTool, line 214, in getObjSize
  Module Products.ATContentTypes.content.base, line 204, in get_size
  Module plone.app.blob.field, line 194, in get_size
  Module plone.app.blob.field, line 66, in get_size
  Module ZODB.Connection, line 811, in setstate
  Module ZODB.Connection, line 876, in _setstate
  Module ZODB.Connection, line 1270, in loadBlob
  Module ZEO.ClientStorage, line 933, in loadBlob
POSKeyError: 'No blob file'

Steps to reproduce:: Not sure about this, but I'd guess that deleting a blob file from disk, then getting size on that file would suffice.

Added by Andreas Zeidler on Dec 17, 2008 11:15 AM

Issue state: unconfirmed → open

Target release: None → 1.0

Responsible manager: (UNASSIGNED) → witsch

thanks for reporting this. i was planning on making the whole implementation robust against missing blob files anyway (as this will greatly help when having to copy a storage for debugging :)), but having a ticket will help in following up on it... :)

Added by Andreas Zeidler on May 08, 2009 03:28 PM

Severity: Medium → Important

up'ing priority as this will have to be resolved for the first release candidate...

Added by Andreas Zeidler on May 12, 2009 11:25 PM

Issue state: Confirmed → Resolved

fixed in http://dev.plone.org/plone/changeset/26910

Added by Andreas Zeidler on May 15, 2009 10:37 PM

the fix has been released in http://pypi.python.org/pypi/plone.app.blob/1.0b3

Added by Andreas Zeidler on Jan 15, 2010 03:45 PM

fyi, i've reverted the changes from r26910 again now — please see http://dev.plone.org/plone/changeset/33101 for more info.

Added by David Glick on Feb 01, 2010 05:20 AM

I agree with raising an error by default, but can we make this something it's possible to opt in to, in case I need to do some non-blob-related debugging on a local copy of a database, and don't want to copy all the blobs?

Added by Andreas Zeidler on Feb 04, 2010 09:24 PM

yes, and as mentioned above (and probably also in the commit messages :)) that was my original intention. however, i think not raising an error and instead silently going on will it make it even harder to find and track down bugs. well, i said that already, didn't i? :)

anyway, i thought maybe we could make it dependent on whether or not the instance is running in debug-mode. if so, the missing blob wouldn't break the site, but a big error message should be issued... what do you think?

Added by David Glick on Feb 04, 2010 09:37 PM

From our IRC conversation:

davisagli: um, well, if *I'm* the one running in debug mode and I have an up-to-date blobstorage rsynced from the server, then I probably want things to break so I know when there are problems
[1:42pm] davisagli: witsch: but I want to be able to hand a Data.fs to a designer and have them work on something without having to send all the blobs too
[1:43pm] davisagli: witsch: whether or not I want things to fail hard depends more on whether or not I my lack of an up-to-date blobstorage is intentional or not
[1:43pm] davisagli: which you can't tell just by whether we're in debug mode

just a new setting in zope.conf would be great (not sure how hard it is to add one of those), because then I could just set it in our standard development.cfg buildout variant

Added by mintsauce on Nov 02, 2010 04:04 PM

Just to ping this issue again, I get a similar error (POSKeyError: 'No blob file') when trying to rebuild my catalog.

As missing blobs (as mentioned above) are usually just developers working without the blob directory, I feel this would be much better reported by a warning, rather than full error.

In the meantime, how can I work around it?

Added by Alejandro Pereira on Nov 05, 2010 12:08 AM

Hi, i get the same error of mintsauce. I'm doing a migration from plone 3.3.5 to plone 4 and give me that error with the index "SearchableText" (ZCTextIndex type). I could not find a solution, can someone help?
This is the full error log: http://pastie.org/1273753.

Thanks.

Added by Alejandro Pereira on Nov 16, 2010 07:38 PM

I solve it, it was not an error of the product. Was a misunderstanding with a partner of mine. Thanks...

Added by Dan Thomas on Feb 04, 2011 01:18 AM

The key problem with the error message "POSKeyError: 'No blob file'" is that it does not identify the missing file(s).

I do not know which files are missing, and I cannot catalog my site, copy it, or export it.

I hope you can help me.

Dan

No responses can be added.