#131: Document Staging

Contents

1. Definitions
2. Motivation
3. Assumptions
4. Proposal
5. Implementation
6. Risks

Proposed by

Maik Röder

Proposal type

Architecture

State

rejected

Definitions

Canonical

  A document that does not have an "isTranslationOf" reference is canonical.

Document Staging

    Work on a development version of a document while the live version is published.

Document Versioning

Motivation

In Plone, it is necessary to retract a document before being able to modify
it. This is annoying, because this means the document is not visible while
it is being edited, and is going through the approval workflow. Of course a
member can work on a private copy of a document, submit it to a Reviewer, who
then needs to replace the old version with the new version manually, but this
way of versioning emulation is too complicated and error prone. This calls
for an integrated versioning system for Plone.

Assumptions

  o Comparison of historic versions of a document field by field is out of the
    scope. CMFDiffTool may be used later.

  o Version comments and history are those of the normal Plone workflow.

  o This proposal does not try to find a solution to the problem of lost
    references when copying documents. A solution in the concerned
    products, or in Plone needs to be found.

  o We only concentrate of making the system compatible with Plone 2.1. Support
    for Plone 2.0 is not part of this PLIP.
   
  o No locking is implemented for the moment. A user can create more than one
    working version, with all the risks of having to merge later.

Proposal

  We propose a simple solution for versioning that is accessible through
  an action drop-down like it is proposed for working on languages in LinguaPlone.
  From there, a user can create a working version of a document.
 
  While editing this working copy, the published version of the document remains
  available for consultation.

  The working version of the document and any past versions can not be
  found through catalog searches, in folder listings or folder content
  listings. For the moment, the only way to access past or working versions
  of a document is as a link that is provided in the versioning drop-down from the
  current document.

  As the normal workflow associated to the working document is used, this
  also means that whatever review process is defined on the working document
  is taken into account.

  On versioned documents, the Owner and Reviewer can

    o view the version history in a portlet

    o browse old versions depending on permissions

  On a published document without a working copy, the Owner can

     o create a new working copy of the document

  On a working copy of a document, the Owner can

    o make this document the current version

Integration with LinguaPlone

  Let's consider the following use cases to see how versioning can be integrated
  with LinguaPlone.


  The user wants to know the version state and history of the current document:
 
    1. The user goes to the document
   
    2. A special portlet indicates which document the current document is derived
       from, if any, and which are the other languages the document has been translated
       to, and what their versions are. (The level of detail needs to be decided yet)

    3. The user can directly go to any of the documents listed in the portlet

    4. The portlet has a link for quickly adding a version for the current
       document. This way the current user can directly see when he is allowed
       to add a version, without having to go to the versioning pull-down.


  The user wants to work on a non-editable document written in the canonical language
 
    1. The user goes to the canonical document

    2. The user sees whether there is already a working version in the version
       status and history portlet.

    3. If the user is allowed to add this kind of document in the context, the
       "Create new version" action is available from the version drop down or
       in the history portlet.
      
    4. The user clicks on the "create new version" link

    5. The user is warned if there is already a working version of the document
       for this language.

    6. The user edits the document. The LinguaPlone interface should be the
       same as usual. (The canonical document on the left, and the working copy
       in edit mode on the right)

    7. The user saves the changes to the working document
   
    8. The user follows the workflow transitions of the working document
   
    9. Any other users who are responsible as defined in the workflow
       can work on the document, going through any necessary workflow
       transitions.
   
    10. Once the workflow state of the document has reached the same state as
       the canonical document, the user can make the document the current
       document.


  The user wants to work on an editable document written in the canonical language:

    1. The user goes to the canonical document

    2. The user goes to the "edit" tab
   
    3. The user is warned about any existing working versions that it would be
       very hard to merge with the current document.

    4. The user modifies the document
   
    5. The user stores the document


  The user wants to translate a working version to another language:

    1. The user goes to the working version
   
    2. The user goes to the LinguaPlone drop-down
   
    3. The user selects "Translate into (language)"

    4. A new document is created
   
    5. The LinguaPlone interface offers the choice of language/version
       translate from
      
    6. The user sees a view of the working version he wants to translate from on the
       left, and can translate to the new language on the right.
   
    7. The user saves, edits and publishes the document as usual.


  The user wants to change the language of a working document.
 
    1. The user goes to the working document
   
    2. The user clicks on the "Manage translations" link in the LinguaPlone
       drop-down.
      
    3. The user selects the language he wants to convert the document into
       and clicks on 'change language'

    4. If there is already a working version for that language, the user
       is warned.
      
    5. The language of the document is changed


  The user wants to delete a document:
 
    1. The user goes to the canonical document
   
    2. The user clicks on the "Manage translations" link in the LinguaPlone
       drop-down.

    3. The user selects the version to delete
   
    4. If the user wants to delete the canonical document, all versions
       are deleted, otherwise the document is deleted


  The user wants to delete a current document in the Plone interface:
 
    1. The user goes to the folder that contains the document
   
    2. The user selects the document in the folder listing
   
    3. The user deletes the document
   
    4. If it is a canonical document, all versions are deleted, otherwise
       just the document is deleted
   

  A Reviewer sees a document in his review box, and wants to publish it

    1. The Reviewer clicks on the working document in his Review box. The
       version number is added to its title.

    2. The Reviewer sees the version history and state portlet
   
    3. The Reviewer publishes the document.
   
    4. The version to be made current appears in the "Working versions" box


  A Reviewer sees a document in his review box, and wants to publish and make it current
 
    1. The Reviewer clicks on the working document in the Review box

    2. The Reviewer sees the version history and state portlet for the document,
       which shows that the document is not the current one.
   
    3. The Reviewer publishes the document.

    4. If the user has the "Make working copy current" permission on the document,
       he sees it appear in the "Working versions" box. This way, the Reviewer
       is reminded that the document still awaits to be made current.
   
    5. The Reviewer makes the document current in the versions drop-down


  A user has made a working version of a document, and wants to make it current.
 
    1. If the user has the "Make working copy current" permission, he
       sees the document in a special "Working Versions" box

    2. The user clicks on the document

    3. The user sees the version history and state portlet that identify the
       document as a working document.
   
    4. The user makes the document current


  A user wants to make a document private, for which a working version exists:

    1. The user goes to a published document
   
    2. The user makes the document private
   
    3. The working version is not concerned by this operation.


  The user wants to translate a set working copiesin different languages
 
    1. The user makes a versions for all languages he needs
   
    2. The user works on the versions in the different languages

    3. The user has the choice of the source for the translation.
       
    4. The documents can go through different workflow states individually, and
       in parallel

  Conclusion:
 
    The use cases demonstrate that a tight integration with LinguaPlone is
    fundamental to work with versions. The user needs the choice of language
    and version to translate from, just like he is used to in LinguaPlone.
   
    The version aspect needs to be very clear for the user, so that he doesn't
    confuse a working copy with a current version. This is partly done in the
    version history portlet, but this may not be enough.
   
    It is important to note that besides the Review process, there is a separate
    process which governs the replacement of existing versions with working
    versions.
   
    A fundamental point is that the system fully relies on existing workflows,
    and never touches the security of the documents. Working content is filtered
    out at the lower levels, and only the current documents appear in the
    official site.

Implementation

  Dealing with LinguaPlone References

    References are used in LinguaPlone to point from translations to canonical
    documents. These references need to be unique per language, because otherwise
    LinguaPlone doesn't have a clue which is the current document.
   
    We need to make LinguaPlone understand that there can be multiple versions
    for a document in one Language, and that the current one is the one with
    the version_state = 'current'.

    Creating a working version of a canonical document
   
      1. We copy the canonical document.
   
      2. The copy does not have the "isLanguageOf" references any more
     
      3. An "isTranslationOf" reference is added that points from the working
         version to the canonical document.

    Creating a working copy of a non-canonical document
   
      1. We copy the non-canonical document.

      2.The existing "isTranslationOf" reference to the canonical document
        is retained.

    Making a working version of a canonical document current
   
      1. We need to redirect all "isTranslationOf" back-references targetting
         the canonical version to the working version.
        
      2. The canonical document gets an "isTranslationOf" reference to the working
         version.
        
      3. We need to make the working version the current version

      4. The canonical documents becomes a past version.

    Making a working version of a non-canonical document current
   
      1. We need to remove the "isTranslationOf" reference from the non-canonical
         version.
        
      2. We need to add an "isTranslationOf" reference from the non-canonical
         document to the canonical document
        
      3. We need to make an "isTranslationOf" reference from the working
         document to the canonical document
        
      4. We need to make the working version the current version

      5. We need to make the non-canonical version the past version

    Removing a working version
   
      1. We delete the working version
     
      2. The references automatically get removed

    Changing the language of a working version
   
      1. The "isTranslationOf" reference remains
     
      2. The language of the document is changed

  Adapting LinguaPlone templates
 
    translate_item.cpt
   
      o needs to work with working versions from which another working version
        is translated
   
      o needs to display the working versions as a source in addition to the
        current versions
   
    manage_translations_form.pt
   
      o The user can not select the target language, when there is a working
        version for that target language

  Reviewers
 
    Reviewers need to be able to identify that a document is a working
    version when they look at their Review box. The Review box needs to
    display the version number behind the document title.
   

  Working versions box

    The Reviewer as well as the Owner of a document need to have a
    "Working versions" box, in which all documents appear that are not yet
    made current.


  Version history and state box for current documents
 
    - Lists all documents, the canonical document was translated to. For all
      documents, displays their versions by following the "isTranslationOf"
      references references.


  Version workflow
 
    The version workflow is defined as follows:
 
      o All permissions need to be removed from this workflow. A new permission
        "Make working copy current" is added for the Owner and the Reviewer roles.
     
      o A "version" variable needs to be added to the Variables
     
      o The workflow state of this workflow needs to be renamed to version_state
       
      o The version_workflow has three workflow states:
     
        - past
       
        - current
       
        - working
 
      o The version_workflow has three transitions:
     
        - id: past, description: "Make past", states: current
       
        - id: working, description: "Create new working copy version", states: current
       
        - id: current, description: "Make this the current version", state: working

    When a working document is created, the following happens:
   
      1. The copy takes on whatever workflow state is defined as the initial
         state of the first workflow in the workflow chain.
        
      2. The version_workflow assigns the "working" workflow state to the
         version_state variable, which hides the document from the visible site.)
   
      3. The version number is incremented

  Portal Catalog, Workflow chains and patch to catalog search
 
    We add the version_state to the catalog as a Field Index and in metadata

    We add the version to the catalog as a Field Index. and in metadata

    We need to patch the catalog search to filter on the version_state.
    By default, we only return documents in the current state. Past and working
    versions are excluded from search results, and also from folder contents and
    folder listing views.

    The following line needs to be added at the end of the LinguaPlone patch
    to searchResults::
   
    By default:
   
      o  kw['version_state'] = 'current' # Always search in current documents

    To search for past versions:

      o  kw['version_state'] = 'past'

    To search for working versions:

      o  kw['version_state'] = 'working'

    This makes it necessary to always have the value of all content show up
    with 'current', which can only be done by setting the following workflow
    chain for all content types:

      o workflow chain: plone_workflow, versioned_workflow

    XXX It would be better to set the versioned_workflow only where the versioning
    should be effective, but I couldn't come up with a catalog search for
    that.

Risks

  We have to generalise LinguaPlone to make it understand that there can be
  more than one language version of a document, and that only the current
  one is shown. We have to work closely with the LinguaPlone developers to
  carefully make LinguaPlone extensible.
 
  The searchResults method in LinguaPlone needs to be extended to make it possible
  to extend enforced searching to other criteria than the labguage (e.g. "lang='en').
  For our needs we need to be able to tell LinguaPlone that it needs to search for
  "version_state='current'" by default. Still we need to be able to search for
  "version_state='past'" and "version_state='working'" when needed.
 
  Any products that still use objectValues or objectIds to access their content
  will uncover past and working versions. These products should start to use a
  catalog search like it is done in Plone 2.1 folder listings and folder contents.

  Working copies are not necessarily secured, meaning that if you know its URL you can
  access it if the workflow allows it.

  Transitions made to the current document while the working document is being modified are
  not transferred to the working document. The history of the current document needs
  be interpreted as the history of all its versions.

  When the user publishes the document, he still needs to replace the
  current document with the working copy in another workflow transition,
  which is possibly one step more than necessary?

  Nothing stops a user from modifying the current document while a working
  copy is being worked on. This can lead to lost information.
 
  When making a current document current, this does not necessarily mean that
  the document is published. If the current document is pending, and the
  working copy is pending as well, the working document can take the place
  of the current document. The document then still needs to be published.