« Chandler, and the hardness of software | Main | Sawing off the handle »

Using TextIndexNG3 with Archetypes and automated testing

I'm currently working on the latest version of our Business Directory product for Business Link Kent (it should be going live on 8th Feb, in fact).

Because of the complex searching requirements plain old ZCTextIndex or TextIndex just aren't enough, and we've incorporated the very powerful TextIndexNG3 instead.

Unfortunately the installation documentation and so on doesn't really deal with implementing new indexes using the product. Nor does it show how to programatically convert the pre-existing indexes upon installation of your product; instead it is described as manual steps.

Manual steps do not play nice with either our automated testing strategy or the fact that we blow away and rebuild our development environments 3 or 4 times a day.

In your Install.py (yes, this is a Plone 2.1 site and no, I'm not using Generic Setup for this one) put the following function (I'm using stemming here, so don't copy this blindly if you don't want the effects of stemming):

Then call updateIndexes(out) from your install method and lo! your indexes are converted by your install script, rather than having to do it manually each time.

To implement new TextIndexNG3 indexes from within your Archetypes schema definition is very simple. I'm still debating whether I should be making use of the broader ability of TextIndexNG3 to search across a subset of the fields that it indexes (meaning that I might not need to create multiple TextIndexNG3 indexes), but for right now I'm going the simple route and defining them on the schema as I would for any other type of index.

Once you've installed TextIndexNG3 have a look at Products/TextIndexNG3/src/config.py. In there are all the parameters that you can pass at the point of index creation. To do this in Archetypes it's exactly the same as it would be for any other index; any non-default creation parameters should be comma separated after the index type is specified. For example:

So far the product looks very powerful, and it's definitely sorted all my complex searching requirements (ranges, booleans and so forth) that until now had me completely stumped.

Comments

"""
Unfortunately the installation documentation and so on doesn't really deal with implementing new indexes using the product. Nor does it show how to programatically convert the pre-existing indexes upon installation of your product; instead it is described as manual steps.
""""

This is not entirely true. The standard Zope APIs for createing new indexes apply to TextIndexNG3 as well. In addition there is code triggered by the related Plone configlet to convert the textindexes of a Plone instance to TextIndexNG3 instances. The uninstall code of TXNG3 also restores the original state.

Understood - I was merely pointing out that the documentation itself doesn't cover it explicitly and those not familiar with the source may well have got rather stuck. I was just writing up the steps that I had to follow to automate the process.

I hope you didn't take it as a criticism - I'm extremely happy with the product and grateful that it exists!

If I get stuck with something and can't find anything when googling I always think it's worth writing up so that someone else suffers less than me :)

Post a comment

(If you haven't left a comment here before, you may need to be approved by the site owner before your comment will appear. Until then, it won't appear on the entry. Thanks for waiting.)