#86: Catalog Based Folder Listings

Contents

1. Motivation
2. Proposal
3. Deliverables
4. Participants

Proposed by

Alexander Limi

Seconded by

Alec Mitchell

Proposal type

Architecture

Repository branch

uiteam-plip86-catalog-based-folder-listings

State

completed

Motivation

Plone's folder listings as currently implemented call a folder method which retrieves all objects in a folder (with some optional but expensive filtering). For large folders, this results in high memory usage and slowness because many complex objects will be moved into cache, despite the unlikelihood that more that a small fraction of those objects are needed by the user. The current method requires that screening of content by permission/workflow be done explicitly in the templates or scripts, though all of the necessary information for doing this screening is in the catalog.

Changing the folder listing implementations to use the catalog prevents waking up the objects on listing, so that only those objects which the user explicitly views are loaded into memory. Also, using the catalog allows for efficient complex filtering and sorting based on any catalog index, including the content permissions/roles.

This also makes it possible for tools that depend on the catalog for advanced behaviour - like LinguaPlone, which has parallel objects, and should only return objects in the correct language based on properties in the catalog - to work transparently.

Proposal

We need to create new versions of folder_contents.pt and folder_listing.pt which use catalog queries to generate the content list and may pass along query parameters inherited from the template or the request. Doing this requires installing ExtendedPathIndex from the collective in order to restrict the query to items in the current folder. Additionally, supporting the current features of folder_contents.pt requires adding new indexes for the size and for folder ordering of the contained objects.

We will also need to handle explicit sort parameters passed into the template in conjunction with the current explicit ordering mechanism. Initially this can be done by simply disabling the ordering column when an explicit sort is passed into the query. Persistent folder ordering along with combined explicit and catalog sort_on based ordering will be the subject another PLIP.

Deliverables

• Change the existing templates to use a catalog search and catalog brains without calls to getObject().
• Modify plone_scripts/getObjSize.py to support getting the size of the object that is the current context.
• Add plone_scripts/folder_order.py python script to return the integer folder order of the object.
• Add catalog indexes for getObjSize, folder_order, and convert path to use ExtendedPathIndex.
• Add unit tests for the presence of new indexes, ensure that existing content is properly reindexed, and that queries return expected content.

Participants

Alexander Limi (limi)
Danny Bloemendal (_ender_)
Chad Whittaker (whit357)
Alec Mitchell (alecm)