Working off the file system

by Joel Burton last modified Nov 12, 2009 04:10 PM

The benefits of working on the file system as opposed to in the ZMI.

Use your favorite editing environment

Don't underestimate the value of this. I use vim, which launches
quickly (so it can be used easily with ExternalEditor TTW), and
which practically every server already has installed (so I can use
through ssh). However, even considering that, I find myself almost
twice as productive being able to work in my exact editing
environment, with my macros, scripts, paths, etc., already set.

One word: *grep*!

grep and find are the godsends of working on the filesystem. Once
you've reacquainted yourself with them after years of ignoring them
for the ZMI, you'll wonder what you were thinking.

Source code management

You can use your usual source code management tools.

Staging/replication

It is easy to replicate your setup across multiple instances.

Goal: no skins or scripts in ZODB. Only config results and content.

Our end goal is that the only thing in the ZODB will be
our actual content objects and configuration settings made by
scripts. All of our skins and all of our scripts will be on the
filesystem.

Sometimes, it's very helpful to have scripts or skins be placeful.
For example, you might have a different logo in different parts
of your site, and a common Zope practice for this would be to have
'logo.jpg' in the root of your site, and a different one in
'/about'. However, this leaves a piece of skin in the site, and
ruins our ability to maintain it well. Better is to have the root
folder and '/about' folder have a property--say, logo_name, which
tells us what logo to use in this area. Then, we can keep all of
these logos on the filesystem, and have achieved our goal of just
keeping the configuration part in the ZODB.


Adding New FS-Stored Content

Follow the examples of content types in CMFPlone.

In CMFPlone, there are examples of all of the types that can be
stored on the filesystem including feature like proxy settings,
titles, security settings. 'grep' is your friend here: 'grep -r
--include="*.metadata" proxy *' Will give you an example of using a
proxy setting in .metadata files.

There's no example of a working ZSQLMethod shipped with
CMFPlone, CMFDefault, or CMFPlone. For a discussion of ZSQLMethods
and an example of a filesystem-stored ZSQLMethod, see http://joelburton.com/resources/reldb.pdf.

'.metadata' files for title, security, proxy roles

These are the accompanying files that hold all the "other stuff" for objects,
titles, security settings, proxy roles for PythonScripts, and more.
These replace the .properties files in earlier versions of the CMF.

One common mistake is to customize a skin object file (say,
'foo.py') by copying it to a new directory, but not copying any
'foo.py.metadata'. Note that the .metadata file **must** be in the
same directory, so in this case, the customized foo object is used,
and it does not get the settings in its metadata file. If this
contained important things, like security or proxy settings, this
could be disastrous.

Debug mode and FS-stored content

In debug mode, changes to filesystem-stored content are seen
immediately. If you're not in debug mode, you'll have to either
restart the Zope server or use product refresh.

Note that if you're using SpeedPack under debug mode, you'll see
changes to skin objects, as long as they don't get copied from one
directory to another. However, if you customize a skin object to a
new directory and are running SpeedPack under debug mode, Zope has
already cached the old location of the skin object, and won't use
the new location version. Restart or use product refresh to have
Zope notice this.

Metadata File Example

From 'register.cpy.metadata' ::

[default]
title=Register a User
proxy=Manager,Anonymous

[validators]
validators = validate_registration

[actions]
action.failure=traverse_to:string:join_form
action.success=traverse_to:string:registered
action.prefs=traverse_to:string:prefs_users_overview



New FS-Stored Types

It is possible to create your own FS-stored types. If there are non-contentish objects you use often, it's worthwhile
creating the small script that will allow it to be stored on the
filesystem. You can look at the code in CMFCore for storing the
existing types (PageTemplates, PythonScripts, Files, etc.)

An example of this is 'FSExternalMethod'. It is taking an existing Zope object type and creating a
FS-stored capable version of it. Found in the product "FileSystemSite".


Atonement for Your Sins - FSDump

Sometimes, it's not possible to work on the filesystem. You may
only have access to a web browser or have created many existing skin
objects in the ZODB. FSDump will take existing
skin objects and dump them out to the filesystem.

FSDump:

* Dumps existing ZODB stuff to FS

* You need to write dumpers for your FS-Stored types.

If you've written custom FS-stored types, you'll have to write
your own Dump plug-in for this. This is quite easy to do--see the
code in FSDump for examples of how it dumps the basic types.

At the very least — use FSDump for backups