Category Archives: Testing

Announcing FuncBrows – A browser testing tool

For the short version: System web browser testing abstraction layer. Get it here

For the last month or so, I have been heavily investigating various functional browser testing tools, with the aim of adding them to our Continuous Integration build.

The History

I settled on using zc.testbrowser for writing quick, functional tests that can be added to the end of a unit test run. As testbrowser isn’t a full browser in itself, it’s fast to run tests with, and can easily be integrated with the Zope and Plone functional tools.
However, for a full suite of integration tests, testbrowser isn’t a great fit. It doesn’t support javascript, and won’t tell you that ‘The login button is missing in IE6’, or that ‘Firefox 3 can’t access the profile changer’. For this type of test, the current leader is Selenium, which can instrument and drive real browsers (IE, Firefox, Chrome, Safari), in order to perform more accurate real world tests.
Selenium is currently undergoing a massive revamp in order to add better functionality and clean up the API (including a mode that will work similar to testbrowser), however this means that we are currently stuck with the older, stable version, as it has python bindings and more documentation.

So, given these two tools, I wrote a simple suite of tests for a project. They registered a user, logged in, logged out, and changed a user profle. Not massively complex, but it’s often surprising how many times such simple processes can be broken, and you won’t notice as you tend to always use the same test user details.

This was all well and good, and everyone was happy and it was sunny.

The Problem

The problem then became, that although developers can write testbrowser scripts fairly easily, and run them quickly, selenium is a lot more heavyweight, requiring selenium installs, multiple browsers and virtual machines to run a full test.

Fundamentally, the selenium API is very different from testbrowser, and asking people to write both was never going to happen.

This meant that selenium was added to the CI build, but the developers would never usually run the tests themselves, creating a disconnect between the current tests that the developers would run as part of their unit test suite, and what would be tested more heavily with the CI build.

The Solution

I (with a lot of help from other people, some ‘creative language’, and some frustration) created FuncBrows. This is a simple, lightweight abstraction tool over both testbrowser and selenium, that can easily be extended to add more tools to it, when required (selenium v2 and twill are on the target list).
It can easily be included, and configured in one line, with a base set of tests that can then be run by every tool, as required.

This means that the developers can write fast tests for their own use, and the exact same code can then be reused for more complete browser testing later in the system tests. A quick, simple way to smoke test for browser issues

There is a samples directory in the github repository, with a simple example of how to set up the tests so that they can be run with either the python standard test runner, or nosetests.
It’s fairly simple, and can’t do any advanced stuff, it only progressed to the stage where we could dogfood our existing tests, I expect the API to grow slightly as we need more functionality from it.
Patches and issues gratefully accepted at the github page.

Get the bits here: Isotoma Github
Or ‘easy_install FuncBrows’

Using Select Boxes with WebDriver

This is a little update to my previous WebDriver post. The code below allows you to set the content of select boxes by accessing the element directly, in this case using XPath to find the element.

#!/usr/bin/env python

from webdriver_firefox.webdriver import FirefoxLauncher
from webdriver_firefox.webdriver import WebDriver

driver = WebDriver()
driver.get("http://cassandra.appspot.com/")

elements = driver.find_elements_by_xpath(
"/html/body/div[@id='container']/div[@id='search']/form[@id='searchForm']/div/select")

select_box = elements[0]
options = select_box.find_elements_by_tag_name("option")

for option in options:
    print option.get_text()

# sets the select box to last.fm Username
options[2].set_selected()

Debugging django unit tests with WING IDE

Wing is a nice IDE, it does autocompletion, project management and other fancy stuff like SVN integration.

There is a page here on how to set up Wing so that it’ll debug within a django environment, including how to make it work with the auto reloading feature, however this will not help if you wish to debug within unit tests, which can be extremely useful, particuarly if you are following TDD or test-first philosophies.

The answer to this:

  1. Set the manage.py of your django project to be the ‘Main Debug File’. Do this by finding manage.py in the Project View and right clicking on it. The option is about half way down the drop down.
  2. Set the environment variables required in the Project settings.
  3. Right click the manage.py again and go to “File Properties”
  4. In the Debug tab of File Properties, set the ‘Run Arguments’ to ‘test’ (without the quotes)
  5. Add a breakpoint somewhere in your tests
  6. Press Debug
  7. Marvel.

Hope this helps someone, I’ve just spent time getting it working, only to blast my .wpr file and have to remember how to do it again, so I thought I’d write it down.

Using WebDriver to Test Web Pages

In this post I will give a quick head first introduction into WebDriver. At the time of writing the WebDriver project is in the process of being moved into the Selenium.

Testing web pages can be difficult especially if the code is under development. An excuse commonly used is “We cant test while everything is changing”, fair enough but the basic functionality of the page probably wont change . For example allowing users to sign up, login or change their password.

This is where WebDriver can help. Interacting with a web page with WebDriver is straight forward. Get an element on the page and either read or write to it. In the “everything is changing” environment its unlikely that the CSS id of the username field or the password field will change very frequently if ever during normal development. You did give your all your fields CSS ids didnt you? If not the second test below will show you how to find your required field.

Below I give two examples using Twitter. The code below is setup to run in Python’s UnitTest using the Firefox browser. To run the code below, download WebDriver from Python Release and follow the instructions. Make sure you add

export WEBDRIVER=/home/channam/Code/python/webdriver

to your ~/.bashrc file under Linux to ensure no errors when running.

test_login_twitter_by_css_id fetches the username and password fields by their CSS ids. To find these I used Firebug for Firefox and the inspect element function.

test_login_twitter_by_xpath fetches the username and password fields by their position in an XPath query. To save writing the XPath myself I used a Firefox plugin called XPather which will display the XPath for a required element on a page.

Below gives you two separate ways for you to test logging into Twitter. The test is successful if we get back the correct title for the page once logged in. Both methods have the weaknesses, more so the XPath method where a small layout change will break the XPath location and so make the test fail. Given the choice, using the CSS id is usually a better choice.

#!/usr/bin/env python

import unittest

from webdriver_firefox.webdriver import FirefoxLauncher
from webdriver_firefox.webdriver import WebDriver

class TwitterTests (unittest.TestCase):

    def setUp(self):
        # setup web driver
        self.driver = WebDriver()

    def tearDown(self):
        # close the window, if you want to see the
        # finished page comment this out
        self.driver.quit()

    def test_login_twitter_by_css_id(self):
        # page we are testing
        self.driver.get("http://twitter.com/login")

        # find our elements
        # the html on the page with same ids
        username_element = self.driver.find_element_by_id(
                               'username_or_email')
        password_element = self.driver.find_element_by_id(
                               'session[password]')

        # use this to toggle the remember me box
        remember_me_element = self.driver.find_element_by_id(
                                  'remember_me')

        # type into the boxes
        username_element.send_keys('username')
        password_element.send_keys('password')

        # set the remember me box
        remember_me_element.toggle()

        # click Sign In and we should be logged in
        self.driver.find_element_by_id(
            'signin_submit').click()

        # check that the title of the page is
        # correct to see if we logged in
        self.assertEqual(self.driver.get_title(),
                             'Twitter / Home')

    def test_login_twitter_by_xpath(self):

        self.driver.get("http://twitter.com/login")

        # find our elements
        # the html on the page with same ids
        username_element = self.driver.find_elements_by_xpath(
                "/html/body[@id='new']/div[@id='container']/table/tbody/tr/td[@id='content']/div/form/fieldset/table/tbody/tr[1]/td/input[@id='username_or_email']")[0]
        password_element = self.driver.find_elements_by_xpath(
                "/html/body[@id='new']/div[@id='container']/table/tbody/tr/td[@id='content']/div/form/fieldset/table/tbody/tr[2]/td/input[@id='session[password]']")[0]

        # use this to toggle the remember me box
        remember_me_element = self.driver.find_element_by_id(
                                  'remember_me')

        # type into the boxes
        username_element.send_keys('username')
        password_element.send_keys('password')

        # set the remember checkbox
        remember_me_element.toggle()

        # click Sign In and we should be logged in
        self.driver.find_element_by_id(
            'signin_submit').click()

        # check that the title of the page is
        # correct to see if we logged in
        self.assertEqual(self.driver.get_title(),
                             'Twitter / Home')

if __name__ == "__main__":
    unittest.main()

I recently expanded the above to test for a sign up email for a project I was working on. When a user signed up for an account a confirmation email was sent to their email address with a link to validate their sign up. To test this I wrote a WebDriver test to sign up in a similar fashion to the ones above. The test instance on the site was configured to send all emails to a local IMAP server. Using Python’s IMAP libraries I was able to login and retrieve the confirmation URL contained in the email. I then visited the URL in the WebDriver code to test that the account creation had been successful.

WebDriver is a great tool for testing your websites so your users don’t have to.