Running a Django (or any other) dev instance over HTTPS

Being able to run your dev instance over HTTPS is really useful: you might spot some weird bug that would have bitten you in production, and if you do find one, you can debug it much more easily. Googling for this subject resulted in several different tutorials using stunnel, but all of them broke in some way on my machine running Ubuntu Maverick. So here is how I got stunnel working – perhaps it will help someone else too:

sudo aptitude install stunnel
sudo su -
cd /etc
mkdir stunnel
cd stunnel
openssl req -new -x509 -days 365 -nodes -out stunnel.pem -keyout stunnel.pem
openssl gendh 2048 >> stunnel.pem
chmod 600 stunnel.pem

Now create a file called dev_https with the following text:

debug = 7


Note: this assumes your web server is running on port 8000. If it’s not, change the value of “connect” to the appropriate port.

Finally, run:

sudo stunnel4 dev_https

Now if you go to https://localhost:8443/, you should see your HTTPS-enabled dev instance!

Note: To properly simulate a HTTPS connection in Django, you should also set an environment variable HTTPS=on. Without this, request.is_secure() will return False. You could set it at the same time as starting your dev instance e.g:

HTTPS=on python runserver

About us: Isotoma is a bespoke software development company based in York and London specialising in web apps, mobile apps and product design. If you’d like to know more you can review our work or get in touch.

Photoshop guidelines for web designers

We often work with external design agencies. Sometimes they work with wireframes we produce, and sometimes not. These are my standard guidelines (or wishlist, if you will) that I try to send to designers before they start. I don’t think I’ve ever received any that meets all these criteria, but it’s always good to aim for. Perhaps it’ll be a useful checklist for other designers out there.

  1. Final deliverables are layered Photoshop files (.psd), with flat file snapshots of each.
  2. When providing flat files (incl. work-in-progress snapshots), use 24-bit PNG format (not JPG).
  3. Use separate files for each distinct layout template.
  4. Photoshop Layers should be given meaningful names. Multiple identically-named layers are unnacceptable. As far as possible, remove obsolete layers. Use layer groups to organise layers
  5. Photoshop Guides should be used, matching the page layout grid. Remove obsolete guides. Guides should be snapped to pixel edges Photoshop lets you position guides at sub-pixel level, which causes confusion when working with pixel-level artwork for screen-based media. To snap guides to pixel edges, first create a marquee selection in the right place, and then snap the guide against the marquee edge..
  6. Consistency between Photoshop files is required, regarding measurements, fonts and colours.
  7. Use a meaningful and consistent file naming policy. Related files should be named in such a way that they are alphabetically adjacent (e.g. content-normal.psd, content-wide.psd
  8. Versioning: for updated versions of files, use a consistent and sensible versioning system in the file names. E.g. homepage-v01.psd. Files with version numbers should stay in order when sorted A-Z. Avoid file name suffixes like -final.psd, -updated.psd, -new.psd etc.
  9. Bear in mind CSS capabilities and work with them wherever possible. For example, horizontal and vertical lines should follow pixel edges exactly (not anti-aliased) in order to be implemented as CSS borders.
  10. When drawing vector shapes, remember to check the “Snap to Pixels” box in the shape options.
  11. Remember to specify hover (mouseover) and active (on click) states for links and buttons.
  12. Accessibility: Minimum font size is 11px. Body text should ideally be 12px or 13px.
  13. Accessibility: Test for adequate colour contrast and colour blindness
  14. Text in Photoshop should be sized in px units (not pt), and in integer sizes (not fractions)
  15. Specify the actual leading in the Character palette, and, for multi-paragraph text areas, spaces above and below paragraphs in the Paragraph palette, that should be implemented using CSS. Do not use double linebreaks or font sizing to set paragraph spacing
  16. Use web fonts as far as possible (core web fonts or from a commercial web font service). Only use non-web fonts (e.g. corporate fonts) where absolutely necessary (they’ll be implemented using images). This is extra important for text that will be content-managed or translated.
  17. Embedded images such as logos should be production quality, as the Photoshop file will be used to cut out final images. (Tip: embed logos as ‘smart objects’, or avoid resizing them multiple times.)
  18. If working from wireframes, try to avoid a “wireframe aesthetic” (monochrome, everything in boxes). If a wireframe puts something in a box, it just means that information should visually stick together. Use gestalt principles to group information visually.
    Edit 3 Mar 2014: Good series on gestalt by Andy Rutledge: 1, 2, 3, 4, and 5.

Edit Feb 2014: Here are 2 excellent resources for designers producing website mockups in Photoshop:

About us: Isotoma is a bespoke software development company based in York and London specialising in web apps, mobile apps and product design. If you’d like to know more you can review our work or get in touch.

HTML5 input type=number and decimals/floats in Chrome

As you may already know, HTML5 has introduced a number of new input types, one of which is the “number” type. As you might expect, this is a form field which accepts numeric input. So what happens in Chrome with the following HTML when we try to enter a decimal (floating point) number and submit the form?

<input type="number" />

Answer: Chrome pops up a validation error:

Decimal number failing validation

So what’s going on here? Is it a bug? It doesn’t fail in Firefox.

Well actually it’s not a bug; the form field is behaving as defined by the W3C. Numeric input fields can take additional attributes “min” and “step”, which constrain the range of values allowed in your input. The “min” attribute is fairly obvious: it’s the minimum value your number can be. The “step” attribute is less intuitive: by playing with different values, you would most likely work out that it controls the increase/decrease when clicking the up/down buttons on the field. If you input the number 1 and click the up arrow, it will increase to 2. This is because the default step is 1. So far, so obvious. However, the step attribute also determines which values are valid, so a step of 1 means you can enter 1, 2, 3 etc. and a step of 2 means you can enter 2, 4, 6 etc, and when you click the up/down buttons the number will increase/decrease by 2 each time, but entering 3 or 5 in the box will cause a validation error. You can also use a decimal value: for example, a step of 0.3 will allow values such as 0.3, 0.6, 0.9 etc, but not 1 or 2.

But what if you want all the numbers to be valid, integers and decimals alike? In this case, set step to “any”:

<input type="number" step="any" />

Now you don’t get a validation error. Yay! Also note that if you only want to accept positive numbers, you’ll want to add min=”0″.

So the lesson here is that the “step” attribute is linked to both the up/down buttons and the range of values allowed in the field. When step=”any”, the up/down buttons will increase/decrease the number by 1. As far as I can tell, there is no way to have step=”any” and an increase/decrease of more (or less) than 1 when clicking the up/down buttons. Feel free to enlighten me in the comments though!

About us: Isotoma is a bespoke software development company based in York and London specialising in web apps, mobile apps and product design. If you’d like to know more you can review our work or get in touch.

Content Strategy Forum writeup

On 5-6 September my colleague Stephen and I attended the second CS Forum in London.

“Content strategy” is the latest buzz-phrase gaining traction in our industry, joining usability, information architecture, interaction design, user experience, accessibility and the like. As with the others, there is a great deal of overlap, but it’s a term I think will endure, as it gets to the nub of the biggest unsolved problem in web projects.

Put simply, it’s that content remains the weak link in most projects. Online content sucks much of the time. There’s too much uninteresting content, not enough useful content, and responsibility for creating and improving it is diffuse. Poor content often exposes the most convincing information architecture and wireframes as a fantasy of wishful thinking.

Content strategy brings out the pessimist in me. There are many reasons why it’s such a hard problem:

  • Getting good quality content out of organisations that do not see themselves as publishers is like getting blood out of a stone.
  • Most content strategy problems are human – political, organisational – on the client side; those you (as external agency/consultant) are least equipped to solve, often going beyond your remit.
  • It’s always easier to make content strategy the client’s problem – you don’t want to accept responsibility for something that is so likely to go wrong.
  • Well-structured content is a Good Thing, but the better structured your data (both information architecture and content types), the worse problems you face when the structure is no longer fit for purpose, and editors try to get around the structure you’ve imposed.

Conference themes

The following themes occurred repeatedly during the conference:

  • We forget to consider the user experience of the content creators (usually in a CMS), in focusing on only the site users
  • Fragmented organisational structures, e.g. lack of communication between developers, designers, copywriters and client
  • Not knowing the CMS well enough, or choosing a CMS without consulting content creators
  • Not learning the lessons of separating content from presentation – content should be produced from a “create once, publish anywhere” approach
  • Lorem ipsum is a problem – it’s always better to design from the content out

The following are not so much summaries of all the talks I attended, as the most interesting things I took away from them. Some writeups are therefore much longer than others.

Gerry McGovern – Manage the tasks, not the content

Back in 2003, his books “Content Critical” and “The Web Content Style Guide” had a huge influence on me and helped guide the content strategy on, which I was working on at the time.

He reminded us that people go to website to do specific things on a website; any content that is not related to those essential tasks should be cut. He particularly dislikes marketing content. “Support is the new marketing”, he says, meaning that the “customer service” content should be the main focus of a site, rather than marketing content.

Avoid unhelpful navigational labels like “Articles” or “FAQs”, and link text should be written to make the keywords clear. (All principles familiar from information architecture best practices.)

Finally, Gerry made a plea for measuring content success, with the emphasis on useful metrics. Figures like visitor numbers, time on page, eye-tracking “fixation time” are all relatively useless – why do we assume that more = better? A/B testing yields more credible insights.

Melissa Rach – Content strategy methodology: a DIY project

Melissa talked about methodology – what strategists do – such as

  • Create clarity
  • Facilitate smart decisions
  • Align stakeholders
  • Help operationalize change

These steps have both content components and people components. But like many talks, it mainly deals with the what (none surprising), but not the how, which is where the the problems lie in my experience.

Margo Bloomstein – First things first: message matters

This talk recommended establishing a “message architecture” as a first step. Via card-sorting, help the client establish

  • Who we are
  • Who we’re not
  • Who we’d like to be

…and tell the story of those aspirations. The message architecture drives the copywriting, the user experience and design, and becomes the benchmark by which to evaluate success later.

This is valid advice, but not always within my remit, and also not really where the worst problems originate.

Karen McGrane – The way forward: what’s next for content strategy

This gets my vote for the best talk of the conference, as it homes in on all the biggest problem areas. You can watch it on Vimeo

Problems usually don’t lie in technology, but in company structure. She describes her role as “corporate therapy”. Companies suffer from fragmentation:

  • fragmented content management
  • fragmented organisation structures
  • fragmented devices and platforms

Content strategy – part of UX – should function as the bridge between marketing and technology. Content strategy is often about organisational change management.

Karen acknowledged to me in conversation afterwards that solutions to these problems are often out of your grasp and if she could do certain projects over again, she wouldn’t necessarily be able to prevent them.

The CMS – “the enterprise software that UX forgot” – is another major problem area. Why do we care more about the conversion funnel than the CMS workflow? Why do we care about the user experience of site visitors, but not editors? We need to work with the developers implementing the CMS from the beginning. And even before that, CMS procurement should be on the basis of usability – workflow that matches editors’ mental model – rather than feature checklists. A better CMS fosters better content.

As for the developing content for a fragmented landscape of devices and platforms, Karen compared the approaches of Condé Nast and NPR. Condé Nast’s strategy, by creating iPad apps for their magazines like Wired, is unsustainable. Ethan Marcotte: “Fragmenting content across ‘device-optimised’ experiences is a losing proposition” . By contrast, NPR pursued a “Create Once, Publish Anywhere” approach, by building an API that enables their content to be deployed in a multitude of contexts. This is why we need flexible, structured content, and why Ethan Resnick claimed “Metadata is the new art direction”.

These are all familiar principles to information architects, but as Karen said, Mobile is a great wedge to bring the argument up again.

Lisa Welchman – On All the Different “Web Governances” in the Universe

Governance refers to a policy and standards. Some key points:

  • A policy that no-one knows exists, or reads, doesn’t exist
  • Policy that you don’t have the authority to implement doesn’t make sense
  • Don’t be a barrier to organisational change that needs to take place (I suspect this is aimed more at insiders than consultants)

Lisa then introduced the recently-formed Web Governance Journal.

Eric Reiss – Content strategists: the men and women of a new renaissance

Eric summed up the big problem I have with content strategy: “Strategy is easy. The rest is tactics.” We often know what needs to be done, but making it happen is where the problems lie.

Erin Kissane – Making sense of the (new) new content landscape

Erin invoked the old IA favourite, Christopher Alexander, who in his treatises on architecture described enduring patterns such as balance, interconnection and stability, and fundamental user needs that products are useful, accessible, findable, searchable, portable and usable in many ways.

We need to preserve the life of our strategic decisions to enhance the life of our visitors.

Des Traynor – The language of software: the role of content strategy in software development

Des talked about websites (or web applications) with strong social elements or user-generated content. In such cases, there may not be much content to begin with, but what we do control, is

  • The user interface
  • The blank slate (what users start with)
  • The content definition

Thus “you get the content you deserve”.

  1. User interface: “Language influences behaviour”. Consider the ramifications of labeling decisions like “tweet” (not share, update, set, publish, post…) or “like” (not love, or appreciate…) E.g you can “+1” something where “like” isn’t appropriate. The Zune’s coolest feature, instant sharing via wifi, was arguably killed at birth by the term “squirt”.
  2. Blank slate: Consider the different quality of user-generated content on YouTube, Yahoo Answers, Quora, Get Satisfaction. By subtle elements in how they let you compose, the results can be idiotic or serious, disrespectful or civil. Google Wave undermined itself with unconvincing sample/seed content.
  3. Content definition: Compare the quality of TripAdvisor reviews with those on the Apple app store (where ratings don’t differentiate between e.g. technical problems or product suitability.)

Des showed an example of a “microcopy framework” for an application. This is a spreadsheet containing all messages in the application, and their aims and expected tone of voice.

Content is “always an opportunity to delight your user”. +1

Richard Ingram – How did we all get here?

Richard revealed the results of several surveys on professionals who consider themselves content strategists, in the form of gratuitous infographics :) which you can view here.

His blog’s title is a veiled entreatment to abandon Lorem Ipsum.

Martin Belam ( @currybet – Content strategy for people who think they already have one

As expected, another stand-out talk, but probably one of the least applicable to me. Journalistic organisations tend not to have the CS problems I find most intractable (they actually have qualified writers and editors), although they have different problems relating to the business model – extracting the greatest value from content – and publishing across a ever-fragmenting landscape.

The Guardian is also an ideal organisation in many respects – doing their development in-house with good collaboration between editors (a fascinating story of “domain driven design” in itself), IAs and developers, and using a custom-built CMS that dispenses with a lot of complex workflow in favour of trust. Having a controlled taxonomy of keywords with a single manager is also something that on most projects I can but dream of.

In keeping with one of the main themes of the conference, Martin stressed the benefits of an API approach to deal with platform fragmentation – create once, publish anywhere.

I liked Martin’s advice to teach using a “portfolio of errors” – showing the mistakes others have made before, rather than criticising your colleagues’ efforts.

Martin ended with a healthy reminder for all “content strategists” not to forget their Information Architecture, and the decade-plus of prior art. (The vast majority of the conference would have sat comfortably under an IA banner.)

Sophie Dennis @sophiedennis – How web designers can stop worrying and learn to love content strategy

I enjoyed Sophie’s talk, which grappled with the all-too-familiar problem their own agency website, which no-one likes but never gets fixed. 

Sophie pointed out that “lorem ipsum” dummy text is specifically intended not to be read, hence it results in designs that are simply not reader-friendly, where the text is the least important component of the design.

When we start designing with real content, we become better designers. However, this is usually not realistic. The pressure to design first typically comes from clients – “I’ll know what to write when I see it”.

In response, Sophie proposed an agile approach, where neither comes first. For their own agency site redesign, she turned off CSS on the existing site and, working with the HTML rather than wireframes, started rewriting – this made it easy for everyone to focus on the content. Then, it became easy to start adding style and “grow the brand out from there”.

Sophie is also a fan of spreadsheets instead of tree-diagram sitemaps, as they are much better at communicating scope. Sitemaps can give a misleading mental model of the amount of content on a site. Spreadsheets are also more accessible and collaborative.

Kate Kenyon @kate_kenyon – Content strategy and CMSs

Valuable reminders never to ignore the CMS. As the content strategist, you need to own it, and understand the publishing process. Find out where it fits in the technical ecosystem. Ideally, the CMS should be procured from the user requirements of the editors.

Good reminders for me. For a long time I intentionally ignored the CMS, believing that it was my duty to focus on the user experience and not be “captured” by the tech constituent.

Cleve Gibbons @cleveg – Strategist and executioner

We’re used to doing “customer journeys”, Cleve said, but we hardly ever do “author journeys”. Optimise the author journey, focus on making authors proficient and productive.

More exhortations to bring technologists to the table when content management is discussed. Developers shouldn’t just “take the ticket and implement” something; they should ask “Why?” more often – what’s the aim of the requirement?

(I agree, from painful experience – many well-intentioned features I designed turned out, after implementation, to result in such editor overhead in the CMS that the features went unused.)

Lisa Moore @writebyteuk – Agile and Content Strategy

Lisa espoused Agile principles such as “putting deliverables on a diet” – keeping documentation to a minimum (I agree), developing early, and testing often.

The team structure she described had separate content strategists and copywriters – she recommends that these people should be involved in all meetings with IAs and developers. However, in most of my projects these don’t exist as separate roles.

Lisa also recommended using real content as far as possible in wireframes – besides benefiting design, another advantage is that it’s likely to be included in user tests.

Noz Urbina @nozurbina – B2B content strategy

Noz made a compelling case for replacing the traditional marketing content with customer service content – i.e. the stuff you usually got after the sale is then driving the sale (echoing Gerry McGovern from earlier.)

B2B (as opposed to B2C) compels you to focus on customer retention rather than acquisition.

Noz saw our role as “consultant within the enterprise” – that if everyone just stuck to the org chart, eventually it’ll be the customer who suffers.

Two accessibility gotchas

A few weeks ago I watched a live web conference in which the open source Plone content management system was put through its paces by a blind person using a screen reader (JAWS). Two important issues stuck with me, since they affect most websites, are quite serious, but do not seem to be very widely known.

Popups are usually inaccessible

(Ticket 12123) Popups (by which I mean dynamic overlays rather than popup browser windows) are usually inaccessible. For example, in Plone the login button invoked a login form inside an overlay. Popups are also commonly used for modal dialogs. It is common practice to put popup DIVs at the end of the HTML body, but this means screen reader users are usually not aware of their appearance, nor have any way to get to their contents. (Especially if they do not contain forms; forms in popups are still accessible via JAWS “forms mode”.)

There are two alternative solutions:

  • The trigger element should have an anchor link to the popup contents.
  • The popup should be inserted in the markup immediately after the trigger element.

Both have some associated problems, but I prefer the first solution and will try to implement it in future.

Form field help should be part of the label

(Ticket 7212) In the commonly-used “forms mode” in JAWS, the screen reader reads only labels, legends, input fields, select fields and textarea fields, but no other elements in the form, such as paragraphs, DIVs, etc. That means any help text (usually positioned below or alongside the field) is ignored. This also applies to validation error messages.

The recommendation is to include the help text inside the LABEL element, and use CSS positioning to put it in the right place in the layout. (I can see how this can cause problems where the text should appear above or below fields.)

Update: Here’s an example (see also the ARIA enhancements, not quite reliable cross-platform according to the associated Alistapart article)

Update 19/10/2011: This article recommends the use of aria-describedby for both help text and validation message. It advocates telling users of non-ARIA supporting screen readers to upgrade (but claims that JAWS has supported it since version 10 in 2008)

It was good to see a content management system take accessibility so seriously. Here are some more tickets.

Buildout Basics Part 1

Introduction to the series

This is the first in a 3 part series of tutorials about creating, configuring and maintaining buildout configuration files, and how buildout can be used to deploy and configure both python-based and other software.
During the course of this series, I will cover buildout configuration files, some buildout recipes and a simple overview structure of a buildout recipe. I will not cover creating a recipe, or developing buildout itself.

For a very good guide on the python packaging techniques that we will be relying on, see this guide:

All code samples will be python 2.4+ compatible, system command lines will be debian/ubuntu specific, but simple enough to generalise out to most systems (OSX and Windows included).
Where a sample project or code is required, I’ve used Django as it’s what I’m most familiar with, but this series is all about the techniques and configuration in buildout itself, it is not Django specific, so don’t be scared off if you happen to be using something else.

Buildout Basics

So, what’s this buildout thing anyway?

If you’re a python developer, or even just a python user, you will probably have come across either easy_install or pip (or both). These are pretty much two methods of achieving the same thing; namely, installing python software

$> sudo easy_install Django

This is a fairly simple command, it will install Django onto the system path, so from anywhere you can do

>>> import django

While this is handy, it’s not ideal for production deployment. System installing a package will lead to problems with maintenance, and probably also lead to version conflict problems, particuarly if you have multiple sites or environments deployed on the same server. One environment may need Django 1.1, the newest may need 1.3. There are significant differences in the framework from one major version to another, and a clean upgrade may not be possible. So, system-installing things is generally considered to be a bad idea, unless it’s guaranteed to be a dedicated machine.

So what do you do about it?
One answer is to use a virtualenv. This is a python package that will create a ‘clean’ python environment in a particular directory:

$> virtualenv deployment --no-site-packages

This will create a directory called ‘deployment’, in which is a clean python intepreter with only a local path. This environment will ignore any system-installed packages (--no-site-packages), and give you a fresh, self contained python environment.

Once you have activated this environment, you can then easy_install or pip install the packages you require, and then use them as you would normally, safe in the knowledge that anything you install in this environment is not going to affect the mission-critical sales (or pie-ordering website) process that’s running in the directory next door.

So, if virtualenv can solve this problem for us, why do we need something else, something more complex to solve essentially the same problem?

The answer, is that buildout doesn’t just solve this problem, it solves a whole lot more problems, particuarly when it comes to ‘how do I release this code to production, yet make sure I can still work on it, without breaking the release?’

Building something with buildout

The intent is to show you the parts for a buildout config, then show how it all fits together. If you want to see the final product, then dissemble it to find the overall picture, scroll to the end of this, have a look, then come back. Go on, it’s digital, this will still be here when you come back….

Config Files

Pretty much everything that happens with buildout is controlled by its config file (this isn’t quite true, but hey, ‘basics’). A config file is a simple ini (ConfigParser) style text file; that defines some sections, some options for those sections, and some choices in the options.

In this case, the sections of a buildout config file (henceforth referred to as buildout.cfg) are generally referred to as parts. The most important of these parts is the buildout part itself, which controls the options for the buildout process.

An absolute minimum buildout part looks something like this:

parts = noseinstall

While this is not a complete buildout.cfg, it is the minimum that is required in the buildout part itself. All is is doing is listing the other parts that buildout will use to actually do something, in this case, it is looking for a single part named noseinstall. As this part doesn’t exist yet, it won’t actually work. So, lets add the part, and in the next section, see what it does:

parts = noseinstall

recipe = zc.recipe.egg
eggs = Nose

An aside about

We now have a config file that we’re reasonably sure will do something, if we’re really lucky, it’ll do something that we actually want it to do. But how do we run it? We will need buildout itself, but we don’t have that yet. At this point, there are two ways to proceed.

  1. sudo apt-get install python-zc.buildout
  2. wget && python

For various reasons, unless you need a very specific version of buildout, it is best to use the file. This is a simple file that contains enough of buildout to install buildout itself inside your environment. As it’s cleanly installed, it can be upgraded, version pinned and generally used in the same manner as in a virtualenv style build. If you system-install buildout, you will not be able to easily upgrade the buildout instance, and may run into version conflicts if a project specifies a version newer than the one you have. Both approaches have their advantages, I prefer the second as it is slightly more self contained. Mixing the approaches (using with a system-install is possible, but can expose some bugs in the buildout install procedure).

The rest of this document is going to assume that you have used to install buildout.

Running some buildout

Now we have a method of running buildout, it’s time to do it in the directory where we left the buildout.cfg file created earlier:

$> bin/buildout

At this point, buildout will output something along the lines of:

Getting distribution for 'zc.recipe.egg'.
Got zc.recipe.egg 1.3.2.
Installing noseinstall.
Getting distribution for 'Nose'.
no previously-included directories found matching 'doc/.build'
Got nose 1.0.0.
Generated script '/home/tomwardill/tmp/buildoutwriteup/bin/nosetests-2.6'.
Generated script '/home/tomwardill/tmp/buildoutwriteup/bin/nosetests'.

Your output may not be exactly similar, but should contain broadly those lines.

The simple sample here is using the zc.recipe.egg recipe. This is probably the most common of all buildout recipes as it is the one that will do the heavy work of downloading an egg, analysing its for dependencies (and installing them if required), and then finally installing the egg into the buildout path for use. Recipes are just python eggs that contain code that buildout will run. The easiest way to think of this is that while a recipe is an egg, recipe contains instructions for the buildout process itself, and therefore will not be available to code at the end.

An analysis of the buildout output shows exactly what it has done. It has downloaded an egg for zc.recipe.egg and run the noseinstall part. Let’s take a closer look at that noseinstall part from before:

recipe = zc.recipe.egg
eggs = Nose

So, we can see why buildout has installed zc.recipe.egg, it is specified in the recipe option of this part, so buildout will download it, install it and then run it. We will take a closer look at the construction of a recipe in a later article, but for now, assume that buildout has executed a bunch of python code in the recipe, and we’ll carry on.
The python code in this case will look at the part that it is in, and look for an option called eggs. As we have specified this option, it will then look at this as a list, and install all the eggs that we have listed; in this case, just the one, the unittest test runner Nose.
As you can see from the bottom of the buildout output, the recipe has downloaded Nose, extracted it and created two files; bin/nosetests and bin/nosetests-2.6. Running one of those files like so:

$> bin/nosetests

Ran 0 tests in 0.002s


We can see that this is nose, as we expect it to be. Two files have been generated because that is that the for Nose defines, a base nosetest executable, and one for the specifc python version that we have used (python 2.6 in my case). These are specified in the that makes up the nose egg, which will be covered in a later article.


We can install buildout into a development environment, and use a simple config file to install a python egg. The next article will cover a development example for using with django, and some niceties such as version pinning and running python in our buildouted environment.

Practical collective.xdv

Last year I published an overview of collective.xdv. A new and very promising way of theming Plone. Now that we’ve used it a few times I thought I’d document some the process we went though.

Firstly read the documentation:

It’s very good and shows you the best way to structure your theme.

Our setup:

  • Plone 3.3.5
  • CMF 2.1.2
  • PIL 1.1.6

Setting Up Collective.xdv Best Practice

The collective.xdv documentation has a very good description of the best way to setup  collective.xdv read it. I won’t go in to the full details as they are well documented but the basic points are:

  • Setup a Plone product
  • Use css registry to serve the css, instead of serving through XDV. This gives you the all the advantages of managing CSS through CSS registry (compression etc).
  • Create your html/css templates.

In our setup I turned off all stylesheets but kupustyles.css, authors.css, kupuplone.css, kupudrawerstyles.css when serving through XDV. I also created a new stylesheet to pull in what we needed from the admin theme. By adding back in on a case-by-case basis you can avoid CSS conflicts with your themes styles. Of course if you’re completely mad you could decide to restyle the admin interface (good luck with that).


There were a few snags we ran into. Here’s how we fixed them:

no styling with portal_factory

We ran in to an issue with our Plone setup and portal_factory. The transformation stopped working when you added content, leaving you seeing the default plone theme. Very confusing. Apparently it’s fixed in 3.3. but for whatever reason we had to use this monkey patch:

Here are some great tips for using collective.xdv as well:

IE conditionals

By default I was removing comments from both the theme and content. (<rules />) . As the IE conditionals are wrapped in comment tags I had to add them back based on their order.

<append theme='/html/head' content='/html/head/comment()[3]' />
<append theme='/html/head' content='/html/head/comment()[4]' />

Lack of applyPrefix support for Plone 3

Plone 4 has a option to parse a stylesheet for relavtive URLs and apply an absolute prefix. You’ll have to do this manually with Plone 3.

Styling the Search Box

I’m not a fan of Plone search box so it was nice to be able to just pull out the action attribute to use:

<prepend theme='//*[@id="search-form"]' content="//*[@id='portal-searchbox']/form/@action" />

Using Conditionals to Select Templates

Using Plone’s default body classes or defining your own you can specify which template to use based on class name. This is a big win as you no longer have mess around with macros or the main template. For example:

<theme href="one-column.html" css:if-content="" />
<theme href="two-column.html" css:if-content="body.two-column" />
<theme href="three-column.html" css:if-content="body.three-column" />

Pulling in Content

You could bring in content piece-by-piece – writing rules for each element. I found this was time consuming and difficult. In the end it made more sense to pull in the all content from each column. If you want to re-work the HTML of a standard portlet it’s better to use inline XSL (more on that in the next article)

<replace content='//*[@id="portal-column-content"]/div/*' theme='//*[@id="content"]' />


From my perspective:

  • It gives control of the templating to the front-end dev / designer.
  • It allows working in parallel with Plone devs rather then them waiting for templates to be completed.
  • Using collective.xdv dramatically sped up the time we had a skinned site we could show to the client.
  • We cut the time we spent doing donkey work and increased the time we could spend on worthwhile development.

I’ll leave it to the Senior Developer to give the pros from an experienced Plone developer:

  • No mangling of reference HTML templates as they are implemented in TAL by Plone Devs
  • improved team workflow as HTML/CSS Designer/Dev can work in parallel with Plone Devs
  • lighter workload for Plone Devs as there is much less HTML to implement in TAL
  • reduced need to explicitly remove unwanted elements from TAL templates – they can mostly be ignored by xdv
  • flexibility in layout between pages – i.e. one-column, two-column, three-column layouts don’t require a mass of macros and modification of main_template to achieve. This can instead be pushed up to the XDV layer.

Jonathan Stark – “Building mobile apps with HTML, CSS and JavaScript” workshop

Jonathan began with an overview of the current mobile landscape. In a nutshell it’s a minefield of platform fragmentation and device fragmentation.

“‘It’s brutal to develop natively across platforms”

Jonathan Stark
So what to do? Well, we then looked at three approaches; native, web and hybrid. All have their advantages but hybrid looked the way to go for most mobile use cases. In Jonathan’s words the advantages are:
  • Web files bundled in native executable
  • Access to device APIs
  • Sell in native app stores
  • Not perfect, but pretty damn good

Next we looked at some useful guidelines for the mobile use case. A nice summation would be “Do one thing and do it well”.


We looked at various CSS3 features; advanced styling, transforms, transitions, animations etc. There’s a wealth of information out there on these so I won’t go into too much detail. Here are some interesting pointers:
  • android requires text-shadow to have a blur value.
  • box-shadow has performance issues, particularly on Blackberry (consider using media-query to turn off for Blackberry).
  • border-image is particularly interesting, as it does away with sliding doors css.
  • animations are hardware accelerated on iOS.
  • be careful of z-index on transforms. By default on it stacks above static positioned elements.
  • box-reflect works with form elements.
  • box-shadow takes up space like borders.
  • box-reflect works on <VIDEO> tag.
  • Talk of hardware acceleration for animation in the next version of Android  (Ice Cream Sandwich).
  • using lots of CSS3 gradients, animations, box-shadow will cause performance issues, tread careful and test often.
I liked the look of CSS animations. I have some reservations of how manageable they will become on medium to large projects. Having written code based animation sequences in Flash I know from experience it can get pretty complicated managing dependances and interactions. Using LESS, or SASS looks like an absolute must for using CSS animation on anything but the smallest projects.


Next up was looking at some of the useful offline features of HTML5. Namely; web storage, web SQL database and application cache. My personal favourite was web storage – it looked easy to use and very useful for state maintenance. Application cache also looked great but I can see huge problems debugging, even Jonathan struggled to demo it properly.

Phonegap, jQTouch and Sencha Touch

Here we looked at the various frameworks for building hybrid apps. Again there is a ton of info out there. Jonathan was obviously keen on jQTouch but I actually think Phonegap looked the more promising platform for development. It seemed to have a good balance between ease of use, learning curve and features. Sencha Touch looked really powerful, whilst  jQTouch looked lightweight and quick to build with.


In the final part of the day Jonathan showed some great debugging tools. If you’ve ever had to debug mobile CSS you know what a pain it is to debug on the device itself.  The stand out star for me was weinre, a really fantastic tool for debugging mobile apps.  The basic premise is it gives you access to the webkit inspector on your laptop which in turn is linked to the phone browser. It allows you to see exactly what is happening on the phone and allows live editing as well. Pretty amazing really.


Jonathan’s workshop has re-enforced my opinion that writing mobile apps for all formats is best done using HTML5 / CSS and JS.  The key benefits are:
  • write once*
  • The skills are already available and less costly than hiring native specific developers
  • one central code base
* They’ll always be some platform specific code needed.
All the slides and downloads from Jonathan’s talk are available here:

The Third Manifesto Implementers’ Workshop

Earlier this month I went to the Third Manifesto Implementers’ Workshop at Northumbria University in Newcastle. A group of us discussed recent developments in implementing the relational data model.

The relational data model was proposed by E. F. Codd in 1969 in response to the complex, hierarchical, data storage solutions of the time which required programs to be written and compiled for each database query. It was a powerful abstraction, but unfortunately SQL and its implementations missed out on important features, and broke it in fundamental ways. In response to this problem, and the industry’s approach towards object-databases, Chris Date and Hugh Darwen wrote “The Third Manifesto” (TTM) to put forward their ideas on how future database systems should work. I urge you to read their books (even if you’re not interested in the subject) – the language is amazing: precise, concise, comprehensive and easy to read – other technical authors don’t come close.

The relational model treats data as sets of logical propositions and allows them to be queried, manipulated and constrained declaratively. It abstracts away from physical storage and access issues which is why it will still be used a hundred years from now (and why NoSQL discussions like these are retrograde). If you’re writing loops to query your data, or having to navigate prescribed connection paths, then your abstractions are feeble and limited.

At the workshop, I talked about my project, Dee, which implements the relational ideas from TTM in Python. You can see my slides here (or here if you have an older browser).

Erwin Smout gave a couple of talks about implementing transition constraints and dispensing with data definition language.

David Livingstone walked us through the RAQUEL architecture – a layered approach along the lines of the OSI network model.

Hugh Darwen discussed the features and implementation of IBM’s Business System 12, one of the first ever relational database systems which had some surprisingly dynamic features, including key inferencing, so that view definitions could keep tabs on their underlying constraints.

Chris Date took us through his latest thoughts on how to update relational views in a generic way. The aim is for database users to be able to treat views and base tables in the same way, for both reading and writing. Lots to think about here, and my to-do list for Dee has grown another section.

Adrian Hudnott discussed a couple of research projects around optimising multiple relational assignments and tracking the source of updates so that transition constraints could be more effective.

Renaud de Landtsheer gave an insight into the work he’s been doing implementing first-order-logic constraints within Oracle databases.

I sat next to Toon Koppelaars (whose name went down well with the Geordies) and then I realised I had his book (Applied Mathematics for Database Professionals) waiting to be read on my desk at work, thanks to the eclectic Isotoma library (and Wes).

It was a packed couple of days with plenty of food for thought. Thank you to David Livingstone and Safwat Mansi for organising and hosting such an enjoyable and interesting event.

UX Lx: Day 1

Thoroughly enjoyed UX Lisbon, organised by ideas e imagens. I don’t think I’ve ever been at a conference where I didn’t regret a single session.

These notes are not necessarily complete summaries of the sessions; they focus on the aspects I found most noteworthy.

1. Prototyping with HTML5 – Todd Zaki Warfel

@zakiwarfel | Slides

Probably the workshop in which I learned least, as Zaki Warfel effectively described exactly the way I currently prototype, and for the same reasons. But was good to have this reinforced.

With all prototypes, including wireframes, set expectations with the client. Choose the level of visual and functional fidelity the project requires. I tend to choose HTML prototypes when I require a high level of functional fidelity, when I don’t know whether an interface works until I can use it.

HTML prototypes have many advantages:

  • closest to final delivery environment
  • browsers & text editors are ubiquitous – fewer things that can go wrong
  • my favourite: producing production-quality HTML for the prototype can shave 30-40% off production time. (This divided attendees as some considered it too high a hurdle. But aligns with my belief that well-structured semantic HTML is akin to IA and the IA is best placed to write it)

HTML + JavaScript is also more capable than any GUI prototyping tool like Axure. But remember, it is still more constrained than a sketch or a wireframe. When you are sketching you can invent things; you’re not constrained by the familiar toolset.

Zaki Warfel stressed the advantages of HTML5, but I came away still unconvinced of any real benefits for prototypes. Only new HTML5 form attributes (date, email, tel, url, placeholder, etc) are a no-brainer if you want to improve usability in iPhones.

In the tricky choice between HTML5‘s <section> and <article> elements, he suggested thinking of using <article> for things that you can imagine going into an RSS feed. But just “choose one road and don’t look back”. (I also find this very confusing, and agree with Jeremy Keith that article and section should be merged into a single element.)

To make HTML5 compatible with older browsers, Zaki Warfel recommended html5shiv instead of modernizr, to keep your HTML cleaner.

CSS3 he didn’t have to sell to me. I already know it saves major time not doing sliding doors or image-based corners, shadows or gradients. But note you have to go back later to fix for IE (or convince the client to accept visual compromises in IE, as Zaki Warfel, echoing Zeldman, advised.)

CSS3 selectors are also a huge time saver in prototyping, but (in my opinion) they will usually require extensive refactoring for IE later.

Zaki Warfel recommended using include files for repeated elements in your HTML, using PHP (already installed in OS X). At first I thought this was unnecessary, but then remembered that in later stages of prototyping I tend to do a lot of difficult find/replace operations across files. So I definitely want to try this in future.

Finally, Zaki Warfel demonstrated the magic of jQuery. I’ve only recently started learning it, and I think every interaction designer should. I use it for all show/hide, open/close, expand/contract, highlight, popup, etc. behaviours in HTML prototypes, and it is a pleasure being able to do this as easily as CSS.

2. Skeuomorphs: The Good, The Bad, and the Silly – Andrew Watterson

@andrewwatterson | Also summarised on Johnny Holland

Skeuomorphs refer to physical metaphors to ease transition to new technologies (mostly touch interfaces in this talk). Explicitly recommended in Apple’s HIG. Criticised by Adam Greenfield “patronizing crutches” and (implicitly) by Jakob Nielsen “Users don’t know where they can click.”

Skeuomorphs can be good:

  • They deemphasise technology in favour of utility (cf. Apple’s ads which suggest familiar usage contexts for new technology, vs. Android’s science-fiction approach.)
  • They can reassure and please, bridge gaps, ease transition. He paraphrases Don Norman’s adage that users have 2 needs, (1) to get something done, and (2) to smile.

But skeuomorphs can cause lots of problems (interestingly, most of the worst culprits are Apple apps)

  • They can mislead with inappropriate metaphors (Apple Calendar and Contacts on iPad have scrollable areas, concealed by the fact that they look like paper; Contacts has a “Groups” icon that looks like a bookmark but is actually a button; the Apple Sound Recorder shows a realistic microphone that conceals the location of the device’s actual microphone)
  • They can cause you to skip opportunities to innovate, preventing you from improving on existing tech (e.g. Apple’s Compass app, which has no more utility than a mechanical one, compared with AR Compass which uses augmented reality to give a far more useful view.)

Watterson didn’t even talk about Apple’s egregious Game Center, which boasts the worst skeuomorphic interface I’ve ever seen.

3. Serendipity: Beyond Recommendation – Pedro Fernandes


In Fernandes’ talk, serendipity refers to fortunate discoveries while looking for something unrelated.

The essence of his argument was that existing recommendation engines frequently resemble echo chambers, limited by their algorithms or the metadata they depend on. If item A always recommends items B and C, and items B and C recommend A/C and A/B respectively, then you’ll never discover items D or Z.

In e-commerce this can undermine “long tail” sales, but in social networking it can result in “cultural tribalism”, online spaces that just reinforce your preferred world view.

Fernandes showed two apps that try to inject more serendipity into the user journey: for browsing films, and Serendipicity, a mobile app for tourists. Mowid relies heavily on tag-based browsing, drawn from a rich vocabulary of tags, and Serendipicity lets you explore places primarily via photos (inherently more open to interpretation) taken in the vicinity by other users.

While I agree about the central issue, I was a bit skeptical about both apps.

Designing for Touch – Josh Clark


Definitely one of the conference highlights (all the way from the Johnny Cash opening music). Clark proved himself a consummate educator, fitting in so much detailed, actionable information and examples that the fact it was a presentation rather than a workshop didn’t matter.

When designing for mobile devices, forget pixels: think of it as designing a physical device. It’s more like industrial design.

Many design rules follow directly from physical limitations. On small touchscreen devices, follow the rule of controls at bottom, content at top – so that content is not obscured by “meat sticks”. For the same reason, two rows of buttons at the bottom is also not ideal, something which Android is unfortunately stuck with.

Josh Clark demonstrating optimal thumb range on an iPhone

Optimal thumb range

Bottom-left is the top spot – for right-handed users at least, which is what you should optimise for. But consider offering a setting for the 10-15% of users who are left-handed.

Because it’s difficult to fix buttons at the bottom of the screen using JavaScript, he suggested a design pattern for web apps where the main menu is always presented at the bottom of the content, but with a “menu” anchor link at the top of the page.

For the iPad / tablets, rules are different. There are many ways to use them, and many use contexts, with no clear preference for portrait or landscape. It is therefore harder to predict hand positioning. Top-of-screen controls are better, to avoid controls at the bottom sinking into your belly :)

The Instapaper app, with its controls in the top two corners, was praised, and The Daily, where using the page scrubber at the top obscures the thumbnail images just below it, was criticised.

What is the optimal size for a button? The answer turns out to be 44 pixels high (29 wide). This happens to be (no coincidence) the height of the iPhone menu bars, buttons, and keys on the virtual keyboard.

But what is a pixel? Due to differing pixel densities, we should stop thinking of device pixels, and think rather of a physical measure on screen. This is called

  • iOS: points
  • CSS: pixel (which is device-independent)
  • Android: density-independent pixel (dp)

So if you continue to use the px unit in CSS, it automatically does the right thing. However, you need to start producing both a normal and high-density version of all images, e.g. image.png and image@2x.png
This is easy enough to apply using the IMG element, but for CSS background images you have to specify the higher-density image thus

@media -webkit-min-device-pixel-ratio:2 {
		.class { background-image: image@2x.png;
		         background-size: 50%;

It’s not just about size, though, but also spacing: the closer together, the bigger buttons need to be. A useful tip is to invisibly increase the hit area for small elements (Remember the Milk does this for checkboxes.)

You can also use animation to draw attention to or explain buttons or other elements that might otherwise be missed or misunderstood. For example, slide in a menu that can be swiped, or pop up the primary action button (Gowalla does this.)

Single-screen interfaces (utility apps) strengthen the illusion that it is a physical device. They should pass the “glance test” – right information hierarchy at arms’ length. Two examples: Tea Round and Umbrella Today. Strip out everything not needed when rushed and distracted. Clarity trumps Density (of features).

But mobile devices are not always used when rushed and distracted! People don’t want “dumbed down”. People want uncomplicated. Full-featured, just lighter interface. (Story of initial Facebook app which was billed as a companion app, which users rejected because they expected to be able to do everything they could do on the website.) Some mobile apps need to do more than the desktop versions.

Don’t fear extra taps! Web has made us squeamish about number of clicks. Latency not an issue in a native/cached app. Tap quality* trumps tap quantity (*unconfusing)

Similarly, don’t fear scrolling. E.g. USA Today used an accordion interface to avoid a long list of headlines scrolling, but not all users understood it. For long lists of content, scrolling is still better.

Changing orientation: think of it not just as a change in layout, but also a change in mindset. Landscape mode can also be seen as “focused mode”. But beware of depending on it, as it is hard to discover. (Personally, I believe orientation in iOS is flawed, invoked overwhelmingly by accident. I think Ben Summers has the right idea for how it should work.)

Affordances: gestures are the keyboard shortcuts of touch interfaces, they can be hard for users to discover. Shortcuts need backup plans – never rely on all users discovering them.

When you see your app being used, look for unsuccessful gesture attempts and repetitive interactions, and pave the cowpaths. (Have Apple never seen users try to swipe the Calendar app to change day?)

Multi-touch (on phones, not iPad) and the shake gesture generally considered bad.

Don’t get carried away with an impressive, glitzy interface. Showcase the content, not the form. Don’t underestimate the power of the humdrum and familiar – see NY Times and Flipboard apps. Similarly, skeuomorphs, while still window-dressing, can enhance a design; familiarity and intimacy invite touch. But if you’re aping a physical object, choose the right metaphor.

But try to avoid buttons altogether. Buttons are a hack. Look at how a toddler uses an iPhone/iPad – they try to interact directly with the content. Wherever possible, make the content the interface. (Example: see how Twitter for iPad eliminated the Back button.)

Clark signed off on an optimistic note, and an encouragement to experiment: new platforms don’t appear very often. This is the coolest job in the world.