#16: Attachments Quota and Upload file size limit

Contents
  1. Motivation
  2. Proposal
  3. Implementation
by Wichert Akkerman — last modified Jan 22, 2010 06:30 AM

Add configuration option to set a global ploneboard quota limit in order to control zope database and therefore disk space growth.

Proposed by
Pete <peteredhair@gmail.com>
Proposal type
Architecture
Assigned to release
State
draft

Motivation

Forums and Boards are usually one of the most active active spaces of a website.

Conversation and debate occur on a daily basis and usually documents are shared in the middle of a conversation and specially images (it is said that a picture/image is worth a thousand words).

Many times threads have several dozens or hundreds of messages, which if we consider that 5% of them has attachments (I'm being conservative and truly think this is a very moderate percentage) we can see where this leads in terms of uncontrolled database growth and disk space usage.

This proposal is for the implementation of a simple mechanism to control disk space and zope database growth using Ploneboard forums.

Proposal

The idea is to add a mechanism with two kind of settings to Ploneboard configuration.
 
One specifying the maximum size all attachments can take up in Ploneboard (everything, all forums), eventually have a mail alert sent to the site administrator when the usage percentage exceeds x%.

There are two options for the second one:

The most usefull one would be a table with with maybe 3 columns and where you'd add lines specifying type in the first column (type could be image, file,...., by file extension) the second column would be to specify the file extension if that was the type selected in the first column, and the third column to specify the maximum size.

While writing this occurred me that the types not specified would not be accepted adding another control to what type of content can be added as attachment.

If adding file size limit by file type is too dificult or complicated to implement then a single max. upload file size would limit all attachments, not so usefull but still a better than no control mechanism att all.

Implementation

  • Implement Attachments Quota and Upload file size limit.
  • Add configuration option to set a global ploneboard quota limit in order to control zope database and therefore disk space growth.
  • Add configuration option to limit the upload file size. This option can be global (max. size for each upload file) or a more useful setting to specify max. size for each file type/extension.
  • File type could be specified by extension (ex: .jpg = 60 Kb, .mpg=2 Mb, .pdf = 1 Mb, .zip = 500 Kb) or "plone type" (ex: image = 60 Kb, file = 2Mb,...)
  • Suggest using a table format with a add button to specify file type/extension limits.


+1 kind of

Posted by Marshall Mayer at Jun 29, 2006 01:56 PM
I generally support this, because having three upload options on all comment forms (even the quick reply !?!) will definitely encourage uploads and ZODB bloat. But think it's way to complex as presented. I think there should a global setting for the maximum size of any file to upload, regardless of file type, using Ploneboard. Default could be 100K but manager definable. If someone wants to upload something larger, there are other ways to do it.

multiple attachments

Posted by Wichert Akkerman at Jun 29, 2006 01:59 PM
The 3 attachment fields for comments that we ave now really should be removed and replaced with something simpler. I was thinking of a single input field and some javascript magic that adds more fields if a user wants to.

+1, Wiggy

Posted by Jon Stahl at Jun 29, 2006 02:38 PM
Wiggy,

The three attachment fields is definitely non-standard UI and rather confusing. I definitely agree that a single attachment upload field, with additional ones appearing as you fill the first one, with a configurable maxiumu would be optimal.

limits per file type

Posted by Wichert Akkerman at Jun 29, 2006 02:23 PM
I agree with Marshall here: having different limits per file type does not make a lot of sense. Having different limits per forum seems sensible.

Why by file type/extension

Posted by Pete at Jun 29, 2006 10:48 PM
Just to justify and give an example of usage limit by file type consider this:

Imagine users can post image files as attachments. You may wan't to allow pictures that serve the demonstration purpose and can take up to 80Kb in 800x600 px.

But users could also add some demonstrating videos which could be acceptable up to a maximum of 2Mb .mpeg files or a .pdf manual up to that file size limit.

With single max. file size in order to allow for the .mpeg files to be attached you will have to set the limit at 2Mb this allows for 2Mb pictures at 1600x.... and higher resolutions with no chance to prevent this.

Many times it's not user abuse, many times they just don't know the consequences on the server side or don't know how to reduce image size, or, simply have 5 images they want to attach and don't want to bother resizing all of them.

Also by specifying file type/extension and limit you limit the type of files uploadable, if it's not on the list than you can't upload that type.

That's the reason why the proposal included a by file type limit.

Surely it might be more complicated to implement but this would IMHO be a more usefull feature. Having a single upload limit blind to the file type is better that none at all.

Like many of you i've run several forums/boards, with phpBB, and found the quota limit control invaluable whatever the way it was implemented.

too complicated

Posted by Wichert Akkerman at Jun 29, 2006 02:34 PM
This proposal is too complicated. I think two per-forum setting suffice for the vast majority of users: a maximum number of attachments per comments (the current default of 3 is silly) and a single maximum attachment size.

+1

Posted by Jon Stahl at Jun 29, 2006 02:42 PM
I agree. Total maximum size (with a warning to the admin) and a max attachment size is a simple, useful way to go. It can always be elaborated later.

I do think this is a pretty important potential security/abuse issue that should be nipped in the bud if possible.

I wonder, would there be a point in trying to make such a mechanism more generalizable/reusable throughout Plone? I can think of other content types that allow attachments that might want to be able to take advantage of such a feature (at least the max size per attachment). This could make it much easier to open up such content types to large numbers of semi-trusted or not-very-trusted users.

I haven't really looked at quota solutions before.

More on Quotas

Posted by Jon Stahl at Jun 29, 2006 02:44 PM
Ok, looks like Nate Aune did some background research on this a while back.

http://www.plone4artists.org/development/teams/quota

Quotes

Posted by Wichert Akkerman at Jun 29, 2006 02:52 PM
This sounds like an excellent place where adaptation can be used. An adapter which adapts an object to an interface which can return the size for the object itself or to an interface which can do that recursively would be very useful. You could combine that with a z3 add event to trigger quota-checks and send out emails. And the beauty is that this will work everywhere, not just for ploneboard.

This means we have two orthogonal projects:

1. implement the max-number-of-attachments and max-attachment-size limits on ploneboard

2. write a new product to check and possibly enforce quota rules

The second is generic and should not be done as part of Ploneboard. The first is a quite simple change that we can schedule for Ploneboard 1.0 or 1.1

1.0 possibility

Posted by Marshall Mayer at Jun 29, 2006 05:04 PM
Perhaps we could create a Ploneboard Setup option that enables file attachments on a Ploneboard-wide basis (all forums) as well as a file size limit). Default is off. If on, an option appears where appropriate, and if one is attached another one can be.

That would solve a problem now, knowing that it might not be the final solution. If a generic solution is developed, independent of Ploneboard, the 1.0 functionality would not be hard to replace.

Marshall

Current implementation

Posted by Wichert Akkerman at Jul 02, 2006 09:55 AM
I implemented a per-forum attachment size limit. I think that finishes implementing the parts of this PLIP on which we have consensus. A more extensive and generic quota solution can be developed outside of Ploneboard and then used for both Ploneboard and other content.