#4 — FSS content is not correctly returned when dispatched inline

Date: 2007-11-24 03:26
Sender: dahoste
Logged In: YES
user_id=1943480
Originator: NO

oops -- I mean 'self.streamsize'.

And it's moot anyway - I haven't been able to tweak the iterator to work.
For the time being, I'm just doing what the original poster did -- I just
deleted the index_html() implementation from FileSystemStorage.py, allowing
the base class implementation to handle index_html() calls.
                            

Date: 2007-11-24 03:18
Sender: dahoste
Logged In: YES
user_id=1943480
Originator: NO

It's looking like range_filestream_iterator hasn't actually been tested.
I was doing some more experimenting and when the iterator was actually
exercised, it exposed a bug in line 122:

data = self.read(self.sizeread)

There's no 'sizeread' property. Looks like it's supposed to be
'self.size'.
                            

Date: 2007-11-23 23:21
Sender: dahoste
Logged In: YES
user_id=1943480
Originator: NO

There appears to be a good example of full Range support in
<zope>/lib/python/OFS/image.py -- the File class. See the following
functions:

_range_request_handler()
index_html()

Notice how populates multiple blocks in the response, one for each range
requested, each with its own 'Content-Range' value. This is consistent
with the definition of the Range header field in the HTTP1.1 spec:

http://www.w3.org/[…]/rfc2616-sec14.html#sec14.35


Judging from that, I don't think the attempted support for Range in the
range_filestream_iterator and index_html() of FileSystemStorage.py is
viable -- i.e. I don't think a single iterator can provide compliant
support for the Range requests. It needs to do the RESPONSE.setHeader()
and RESPONSE.write() for each range block itself.

When you look at OFS/image.py, it will be fairly clear what's supposed to
happen.
                            

Date: 2007-11-23 10:42
Sender: yboussard
Logged In: YES
user_id=1807619
Originator: NO

Hello,
You have right, there is effectively a bug.
The problem is fss return an filestream iterator for performance reason
(no all data in memory)
But when you return an iterator, zope don't do the right job if request
have an http range header
So we have implemented an file stream iterator with range support . But it
only work with one range.
If you have multiple range, body concerned about range (boundary range
body) is write by the publisher before the data is sending by index_html
(and i don't know why and how).
We are working to found a solution to this problem
Regards Youenn Boussard.


                            

Date: 2007-11-23 10:42
Sender: yboussard
Logged In: YES
user_id=1807619
Originator: NO

Hi,


We'd love to answer your request but we'd need more information
about your problem, especially about your platform.

Can you provide us the following information:

 - OS/Platform

 - Python version

 - Zope version (see Zope's control panel)

 - Plone version

 - Installed products (copy/paste your Plone's
portal_quickinstaller information page).

We'll be able to answer as soon as we'll receive this information !

Regards,


The Ingeniweb Support Team.

                            

Date: 2007-11-22 20:52
Sender: dahoste
Logged In: YES
user_id=1943480
Originator: NO

Ok, the index_html() function in FileSystemStorage.py has a bad bug in it.
 

Line 248: [(start,end)]=parseRange(REQUEST.environ.get('HTTP_RANGE'))

That calls the ZPublisher.HTTPRangeSupport.parseRange() function, which
states that it will "Convert a range header to a list of slice indexes,
returned as (start, end) tuples".

Tuple*S* -- plural! I.e., more than one tuple in the returned list. From
the error trace that I was getting from my PDF access, I see the following
value for the 'HTTP_RANGE' value:

HTTP_RANGE 'bytes=1817708-1818731,1810540-1817707,65536-1810539'

Which, in fact, has 3 range pairs. So the list returned from parseRange()
is:

[(1817708,1818731), (1810540,1817707), (65536,1810539)]

A list of length=3 cannot be assigned to a list of length=1. Hence, the
following error in the trace:

Exception Value too many values to unpack

I don't know if that means there is a fundamental misunderstanding on the
part of the index_html() implementation (i.e. does what it's trying to do
even make sense when there are multiple ranges in the request?). Or if
it's merely an oversight -- i.e. maybe it only needs the first range, and
so it's just a matter of fixing the variable assignment.

I don't know anything yet about the range_filestream_iterator that is
created using the range value, so I'm hesitant to just guess at what should
happen when multiple ranges are provided in HTTP_RANGE.

Clearly, this needs a real fix from someone who understands the context.

                            

Date: 2007-11-22 20:23
Sender: schrei5
Logged In: YES
user_id=1778147
Originator: NO

I disable the index_html method a few month ago.
I don't know the internal consequences but it worked so far :-)
                            

Date: 2007-11-22 18:50
Sender: nobody
Logged In: NO

(followup to the previous comment) Here's a trace of the error I have when
accessing a PDF directly (i.e. index_html):

Request URL http://localhost/plone/testfolder/test.pdf/index_html
Exception Type ValueError
Exception Value too many values to unpack

Traceback (innermost last):

    * Module ZPublisher.Publish, line 115, in publish
    * Module ZPublisher.mapply, line 88, in mapply
    * Module ZPublisher.Publish, line 41, in call_object
    * Module Products.ATContentTypes.content.file, line 118, in
index_html
    * Module Products.ATContentTypes.content.base, line 404, in
index_html
    * Module Products.FileSystemStorage.FileSystemStorage, line 248, in
index_html

ValueError: too many values to unpack


                            

Date: 2007-11-22 18:35
Sender: nobody
Logged In: NO

I think this problem persists in FSS 2.6.1. I'm encountering
dysfunctional behavior for PDF files stored using an FSS-based AT type.
Accessing them directly (i.e. via implicit index_html) just times out - the
file contents are never delivered to the browser. Accessing them via a
right-click/save-as (i.e. implicit download) works just fine, and the file
arrives in tact to the client filesystem.

This failure is repeatable on both linux and windows(xp), using IE7 and
firefox (2.0.0.9).

Is there a work around for this? Should I do as the original poster
suggested and disable the index_html method? Is there a fix being
developed for this? 2.6.1 appeared to address some issues in both
index_html and the filestream_iterator, but obviously this problem is not
solved.

Plone 2.5.3
FSS 2.6.1 (2.6.0 behaves similarly)
Zope 2.9.7
Python 2.4.4
Apache 2.2.6
Adobe Reader 7.0.9-r1

                            

Date: 2007-04-25 09:01
Sender: schrei5
Logged In: YES
user_id=1778147
Originator: NO

Hi,

I have the same problem with some PDF files.
Do you know what the index_html method does?