Author Archives: channam

Monitoring Your Power Usage

I was lucky enough to attend LUG Radio and Ogg Camp recently and was inspired by a talk given by Andy Standford-Clark and his house that twitters. I had been sent a power monitor after switching power supplier, but never done much with it. Watching Andy’s energetic performance inspired my to dig it out and have a go at getting the data off it. Also the thought of adding twitter to anything was too strong to resist.

My unit is a branded version of Classic Current Cost unit with a standard transmitter. The first hurdle was to get the transmitter and the monitor talking again. The monitor has three buttons at the base, pressing and holding the middle button syncs the unit to the transmitter. It is possible to associate more than one transmitter to the monitor, the Classic model I believe can talk to 3 transmitters.

Next issue was connecting it to a PC, the unit didn’t come with a data cable. The output from the monitor is slow trickle of XML over a RJ45 serial connector. I found a suitable cable on eBay from Current Cost RJ45 to USB. They so sell the same cables on Amazon but on eBay the postage is free.

Connecting the cable is simple. To get access the feed is slightly more complex but luckily there are some great articles on the web. Connecting under Linux is fairly straight forward, there is fantastic article at http://www.linuxuk.org/2008/12/currentcost-and-ubuntu/. This explains how make the USB device available to read from as a device. Next you can use a Perl script from http://www.jibble.org/currentcost/ or read on for the Python I wrote to interpret the XML. I used the python-serial package from Karmic Koala Ubuntu.

#!/usr/bin/env python

import serial
from lxml import etree

# open the device, 2400 is baud rate
ser = serial.Serial('/dev/ttyUSB0', 2400, timeout=1)

while (True):
    reading = ser.readline()
    if reading:
        try:
            # use lxml to extract what we care about
            xml = etree.fromstring(reading)
            print xml.xpath('/msg/ch1/watts')[0].text
            print xml.xpath('/msg/tmpr')[0].text
        except:
            pass

This will print the watts and the temperature as the data is sent from the monitor via the serial cable.

Without any processing the input from the monitor looks like (replace the hrr with hr – I was losing the fight with html tags…):



    
        00002
        205102
    
    CC02037771
    01429
    00000
    00000
    18.2


For some more detailed information check out: http://mungbean.org/blog/?p=477 and also http://knolleary.net/2008/05/05/power-graphing/ for some tips on graphing.

To finish off below is a script to log the data into a sqlite database:

#!/usr/bin/env python

import serial
import twitter
import sqlite3

from sqlalchemy import *
from sqlalchemy.orm import sessionmaker, mapper

from datetime import datetime
from lxml import etree
from lxml.etree import XMLSyntaxError
ser = serial.Serial('/dev/ttyUSB0', 2400, timeout=1)

while (True):
    reading = ser.readline()
    if reading:
        try:
            xml = etree.fromstring(reading)
            watts = xml.xpath('/msg/ch1/watts')[0].text
            temp = xml.xpath('/msg/tmpr')[0].text
            now = datetime.now()
            time = now.strftime("%Y%m%d%H%M%S")

            metadata = MetaData()
            engine = create_engine('sqlite:////home/yourusername/Desktop/power.db')
            metadata.bind = engine
            Session = sessionmaker(bind=engine, autoflush=True)
            session = Session()
            session.autocommit = False

            # create a class to map to
            class Power(object): pass
            table = Table('power', metadata, autoload=True)
            mapper(Power, table)

            entry = Power()
            entry.datetime = time
            entry.watts = watts
            entry.temperature = temp
            session.add(entry)
            session.commit()
            
            break
        except XMLSyntaxError:
            pass

Code Coverage in Testing

Getting an idea of much code your test covers varies in how complex your code is. I have found coverage.py from Ned Batchelder to be a great tool.

The usage confused me slightly for a while, so I thought I`d share with you how I used it.

First step was to created my Python test. I created mine using the standard unittest framework in Python. I then ran ‘coverage -e -x test_mytest_file.py’ which executes the test.

To extract the coverage of the test run ‘coverage -b -i -d htmlcov ../module_being_tested.py’. This creates a directory of HTML output containing an index.html. Open this in your browser and you will be able to see the overall percentage of covered code. On this page is a list of the modules you tested. Clicking on a module highlights in red the statements not covered by your test.

Google Contact API

I`m quite a Google fan and enjoying playing with their APIs. For me one of the big attractions of Google is the quality and availability of the APIs.

Below is a bit of code that fetches the postal address of a contact from your Google contacts. It took me ages to figure out so I hope posting it here will cut down the learning curve and yes I`m aware its only a simple thing :) .

You will need the Google Gdata libraries available from Google. I also use lxml to process the feed. I cant recommend lxml enough. If your using Python and XML or HTML use lxml.

The following has been lifted straight from a interpreted session so fire up Python first and paste the below in remembering to change the username and password.

import atom
import gdata.contacts
import gdata.contacts.service
from lxml import etree

gd_client = gdata.contacts.service.ContactsService()
gd_client.email = 'jo@gmail.com'
gd_client.password = 'password'
gd_client.source = 'exampleCo-exampleApp-1'
gd_client.ProgrammaticLogin()

feed = gd_client.GetContactsFeed()
entry = feed.entry[0]
entry
entry.postal_address[0].text
'123 Fake Street'

Memcached in 2 Minutes

So you need to use memcached with Python? Below is a brief intro.

First install everything you need. I run Ubuntu so python-memcached memcached packages were required. python-memcached is available from www.tummy.com and the memcached site is available here

Start the memcached server if its not already running:

/usr/bin/memcached -m 64 -p 11211 -u nobody -l 127.0.0.1

The above gets you a 64Mb server, more than enough to play on.

Next some python below is from the interpreter:

>>> import memcache
>>> memc = memcache.Client(['127.0.0.1:11211'])
>>> class test():
>>>    def __init__(self):
>>>        self.message = "Hello, world"
>>>
>>> t = test()
>>> memc.set('testname', t, 120)
True
>>> got = memc.get('testname')
>>> got.message
'Hello, world'

The above instantiates an object and then saves it memcached with set() and then we get it back using get(). In the set call we specify the number 120, this is the number of seconds the object should be held in the cache. The usage is pretty simple: try and fetch from memcached if it fails fetch from your datasource and then save that document ready for next time.

Using Select Boxes with WebDriver

This is a little update to my previous WebDriver post. The code below allows you to set the content of select boxes by accessing the element directly, in this case using XPath to find the element.

#!/usr/bin/env python

from webdriver_firefox.webdriver import FirefoxLauncher
from webdriver_firefox.webdriver import WebDriver

driver = WebDriver()
driver.get("http://cassandra.appspot.com/")

elements = driver.find_elements_by_xpath(
"/html/body/div[@id='container']/div[@id='search']/form[@id='searchForm']/div/select")

select_box = elements[0]
options = select_box.find_elements_by_tag_name("option")

for option in options:
    print option.get_text()

# sets the select box to last.fm Username
options[2].set_selected()

D-Bus Fun

Use the following to make the music player Banshee queue a track ready for playback.

import dbus
bus = dbus.SessionBus()
player_queue = bus.get_object("org.bansheeproject.Banshee",
"/org/bansheeproject/Banshee/SourceManager/PlayQueue")
player_queue.EnqueueUri("/home/channam/Music/Jonathon Coulton/Code Monkey.mp3",True)

For more D-Bus info I recommend the D-Bus tutorial.

Pidgin Alerter using D-Bus

Below is a little snippet of code showing what you can do with Python and D-Bus. The code below displays an alert when you receive a message in Pidgin.

#!/usr/bin/env python

import dbus, gobject
import pynotify
from lxml import etree
from dbus.mainloop.glib import DBusGMainLoop

def my_func(account, sender, message, conversation, flags):
    xml_data = message
    root = etree.fromstring(xml_data)
    pynotify.init( "Pidgin Alert" )
    n = pynotify.Notification(sender, root[0][0].text, "dialog-info")
    # number of secs to display message
    n.set_timeout(10000)
    n.show()

dbus.mainloop.glib.DBusGMainLoop(set_as_default=True)
bus = dbus.SessionBus()

bus.add_signal_receiver(my_func,
                        dbus_interface="im.pidgin.purple.PurpleInterface",
                        signal_name="ReceivedImMsg")

loop = gobject.MainLoop()
loop.run()

If you want to explore D-Bus further I recommend using D-Feet. Its a great D-Bus enumeration program.

Using WebDriver to Test Web Pages

In this post I will give a quick head first introduction into WebDriver. At the time of writing the WebDriver project is in the process of being moved into the Selenium.

Testing web pages can be difficult especially if the code is under development. An excuse commonly used is “We cant test while everything is changing”, fair enough but the basic functionality of the page probably wont change . For example allowing users to sign up, login or change their password.

This is where WebDriver can help. Interacting with a web page with WebDriver is straight forward. Get an element on the page and either read or write to it. In the “everything is changing” environment its unlikely that the CSS id of the username field or the password field will change very frequently if ever during normal development. You did give your all your fields CSS ids didnt you? If not the second test below will show you how to find your required field.

Below I give two examples using Twitter. The code below is setup to run in Python’s UnitTest using the Firefox browser. To run the code below, download WebDriver from Python Release and follow the instructions. Make sure you add

export WEBDRIVER=/home/channam/Code/python/webdriver

to your ~/.bashrc file under Linux to ensure no errors when running.

test_login_twitter_by_css_id fetches the username and password fields by their CSS ids. To find these I used Firebug for Firefox and the inspect element function.

test_login_twitter_by_xpath fetches the username and password fields by their position in an XPath query. To save writing the XPath myself I used a Firefox plugin called XPather which will display the XPath for a required element on a page.

Below gives you two separate ways for you to test logging into Twitter. The test is successful if we get back the correct title for the page once logged in. Both methods have the weaknesses, more so the XPath method where a small layout change will break the XPath location and so make the test fail. Given the choice, using the CSS id is usually a better choice.

#!/usr/bin/env python

import unittest

from webdriver_firefox.webdriver import FirefoxLauncher
from webdriver_firefox.webdriver import WebDriver

class TwitterTests (unittest.TestCase):

    def setUp(self):
        # setup web driver
        self.driver = WebDriver()

    def tearDown(self):
        # close the window, if you want to see the
        # finished page comment this out
        self.driver.quit()

    def test_login_twitter_by_css_id(self):
        # page we are testing
        self.driver.get("http://twitter.com/login")

        # find our elements
        # the html on the page with same ids
        username_element = self.driver.find_element_by_id(
                               'username_or_email')
        password_element = self.driver.find_element_by_id(
                               'session[password]')

        # use this to toggle the remember me box
        remember_me_element = self.driver.find_element_by_id(
                                  'remember_me')

        # type into the boxes
        username_element.send_keys('username')
        password_element.send_keys('password')

        # set the remember me box
        remember_me_element.toggle()

        # click Sign In and we should be logged in
        self.driver.find_element_by_id(
            'signin_submit').click()

        # check that the title of the page is
        # correct to see if we logged in
        self.assertEqual(self.driver.get_title(),
                             'Twitter / Home')

    def test_login_twitter_by_xpath(self):

        self.driver.get("http://twitter.com/login")

        # find our elements
        # the html on the page with same ids
        username_element = self.driver.find_elements_by_xpath(
                "/html/body[@id='new']/div[@id='container']/table/tbody/tr/td[@id='content']/div/form/fieldset/table/tbody/tr[1]/td/input[@id='username_or_email']")[0]
        password_element = self.driver.find_elements_by_xpath(
                "/html/body[@id='new']/div[@id='container']/table/tbody/tr/td[@id='content']/div/form/fieldset/table/tbody/tr[2]/td/input[@id='session[password]']")[0]

        # use this to toggle the remember me box
        remember_me_element = self.driver.find_element_by_id(
                                  'remember_me')

        # type into the boxes
        username_element.send_keys('username')
        password_element.send_keys('password')

        # set the remember checkbox
        remember_me_element.toggle()

        # click Sign In and we should be logged in
        self.driver.find_element_by_id(
            'signin_submit').click()

        # check that the title of the page is
        # correct to see if we logged in
        self.assertEqual(self.driver.get_title(),
                             'Twitter / Home')

if __name__ == "__main__":
    unittest.main()

I recently expanded the above to test for a sign up email for a project I was working on. When a user signed up for an account a confirmation email was sent to their email address with a link to validate their sign up. To test this I wrote a WebDriver test to sign up in a similar fashion to the ones above. The test instance on the site was configured to send all emails to a local IMAP server. Using Python’s IMAP libraries I was able to login and retrieve the confirmation URL contained in the email. I then visited the URL in the WebDriver code to test that the account creation had been successful.

WebDriver is a great tool for testing your websites so your users don’t have to.