Personal tools
You are here: Home Documentation How-tos Migrating data from one field to another
Support

Get Help

Join our chat rooms or support forums if you have more specific questions.

Plone Training
Learn how to design, build, and deploy a website in Plone through one of the numerous Plone training sessions around the world.
Find Plone training…
 
Document Actions

Migrating data from one field to another

This How-to applies to: Any version.
This How-to is intended for: Developers

An example of an ad-hoc script which can be used to migrate data from one field to another within a content type.

Lets say we have content type called MyContentType that has a field category with a keyword index. Suppose we then change our minds and want to use Plone's internal subject (aka Keyword) field for this purpose instead. If there are existing content types in your site, doing this manually would be a pain. Luckily, it's quite simple to write a quick python script to migrate the data from one field to another.

Even more luckily, there is now a component out there to help you called contentmigration!

The script method below may be all you need, but contentmigration, which leverages ATContentTypes' migration framework, used internally in Plone, is very flexible and allows you to make changes like the one below with less code. The tool is described in the context of other migrations in the RichDocument tutorial - see the last section of this page in particular.

If you would rather do it manually with a script, the original steps are below:

  1. Backup your database
  2. Create a script in you custom skin folder copyKeywords.py
  3. Paste in the script below
  4. In the catalog search put the name of you contenttype where it says MyContentType. This parameter ensures that the script only searches for your specific content type.
  5. In this case, we are using getCategory() as the accessor for the old value, and setSubject() as the mutator for the new value. You may have different fields, and you may need to manipulate the value before writing it to the new field.
  6. To run the script, click on the test tab in the zmi

Note: If an exception was raised, the transaction will be rolled back, and no data will be changed. If the script executes successfully, you can use the "Undo" tab at the root of your Plone site in the ZMI to undo the running of the script, if you change your mind

the script:

    request = container.REQUEST
    RESPONSE =  request.RESPONSE

    brains = context.portal_catalog.searchResults( REQUEST=request,
                                                   Type = 'myContent')

    for brain in brains:
        obj = brain.getObject()
        keywords = obj.getCategory()
        obj.setSubject(keywords)
        print "Updated", obj.absolute_url()

    return printed

There are many variations on this script. You may for example perform calculations or other modifications before saving the new value, or save the value to a different content object altogether. Remember that you are modifying your database, so be careful, don't forget the backup, and use the Undo tab if things go wrong!

by Christof Haemmerle last modified February 6, 2006 - 14:32
Contributors: batlogg, berhard (snizek.com), Martin Aspeli (optilude)
All content is copyright Plone Foundation and the individual contributors.

Reindexing

Posted by Pavel Bogdanovic at May 20, 2006 - 18:12

Thx a lot, exactly what I needed. After migrating something to subject aka keywords, you should reindex the catalog index "Subject".

Extended version

Posted by Guy C. Heckman at September 21, 2008 - 14:01
First off thanks for this little snippet, it works fabulously. I extended it a bit to provide more verbose output and to prevent overwrites of the new field in the event this gets run against content where the new field may already have values or the script had bee previously run. I also elected to zero the old field after the migration. There definitely places where the code could be more compact, but I prefer readability.



request = container.REQUEST
RESPONSE = request.RESPONSE

brains = context.portal_catalog.searchResults(REQUEST=request,
Type = 'YOURCONTENTTYPE')
obj_count = 0

for brain in brains:
obj = brain.getObject()
obj_url = obj.absolute_url()
oldvaluelist = obj.YOUROLDFIELD()
newvaluelist = list(obj.YOURNEWFIELD())

if oldvaluelist:
print "Checking", obj_url
updated = 0
for value in oldvaluelist:
if value not in newvaluelist:
newvaluelist.append(value)
print " Adding:", value
updated = updated + 1
if updated:
print " Setting:", newvaluelist
obj.setDataTypes(newvaluelist)
print " Updated with %i value(s)" % updated
else:
print " No values were migrated for this object."
obj_count = obj_count + 1
else:
print "Skipping", obj_url

# You may want to remove this if you do not want your old field zeroed.
obj.setDataType([])

print "\nTotal objects processed:", obj_count
print "Total batch objects:", len(brains)

return printed

Appendum

Posted by Guy C. Heckman at September 21, 2008 - 14:06
Didn't realize comments clear preceeding whitspace - watch your indenting. Also replace all occurrences of setDataTypes with setYOURNEWFIELD and replace setDataType with setYOUROLDFIELD. Sorry, accidently hit return while I was still editing it.

For any issues with the web site functionality, please file a ticket.

Please consult the policy on plone.org content if you want your content published on this site.

Servers and hosting by