Category Archives: Javascript

Black box defect hunting

I’ve made a bit of a bad habit of looking for / finding security bugs in other people’s software. The things I start with before looking through requests and front-end code in more detail I’ve distilled down to a few algorithmic bits of grepping and analysis based on a few simple rules.

  1. Consider all user-input dirty – this doesn’t just include in form fields, but:
    1. Any previously saved data loading back from your server;
    2. URLs;
    3. Data that may be sent to your server even if it’s invalid;
    4. Any communication between frames.
  2. If it’s not over HTTPS, even if you’re getting data from your own service, it could be dirty.
  3. Even if it’s over HTTPS, trust no-one to do the right thing.

And all these rules basically come down to one: sanitize everything.

Things to watch out for in JavaScript:

  • $.html(foo), $(foo);: you almost never want to use this except for hard-coded values. If an API is returning HTML for your front-end to use it’s doing it wrong. Construct it yourself out of the other fields when possible.
  • $.html(‘<a href=”/?foo=’ + myVar + ‘”>Link</a>’);
  • $(‘#description’).html(description);
  • $(‘<div class=”‘ + myVar + ‘”>’);
  • All of the above apply to a number of jQuery methods that construct HTML including append, insertBefore, $ itself and many others.
  • When using Backbone or similar, consider that someone may abuse your routing by sending users to a URL which routes to a view with side-effects. Views with side-effects should either not have routes, or should not be available to route to on initialization.
  • Window.postMessage / onmessage: if your site can be framed by others (no X-Frame-Options protection) or itself contains frames, any messages passed by other frames should be untrusted. What’s more, any data posted to other frames should be protected with a targetOrigin parameter.

Things to watch out for in application design:

  • Ensure your app URLs do not include sensitive data which may be passed to other sites by the Referer [sic] header. This is preferable to relying on browsers respecting rel=”noreferrer”
  • Make sure all requests with side-effects don’t just include, but require a CSRF token either in the request data or as a request header. Cookies do not count.
  • Avoid JSONP when designing APIs.

Ways to safely incorporate user data in the DOM:

  • For parameters or paths in links: $.html(‘<a href=”/?foo=’ + encodeURIComponent(myVar) + ‘”>Link</a>’);.
  • For domains in links where user data will be sent, whitelist the domains.
  • For text or attributes either use an existing escape function such as included in a JavaScript template library you may be using, jQuery’s text or attr functions or something along the lines of

    var $tmp = $('<div>');
    $tmp.text(unsanitizedData);
    return $tmp.html();
    . Do not construct HTML by just concatenating strings.

These are just a few quick things to look out for – this isn’t a black and white checklist, some of these things are safe in the right circumstances. And if you want to find more perverse flaws, more detailed analysis of the code is needed.

Links

There is a new version of gunicorn, 19.0 which has a couple of significant changes, including some interesting workers (gthread and gaiohttp) and actually responding to signals properly, which will make it work with Heroku.

The HTTP RFC, 2616, is now officially obsolete. It has been replaced by a bunch of RFCs from 7230 to 7235, covering different parts of the specification. The new RFCs look loads better, and it’s worth having a look through them to get familiar with them.

Some kind person has produced a recommended set of SSL directives for common webservers, which provide an A+ on the SSL Labs test, while still supporting older IEs. We’ve struggled to find a decent config for SSL that provides broad browser support, whilst also having the best levels of encryption, so this is very useful.

A few people are still struggling with Git.  There are lots of git tutorials around the Internet, but this one from Git Tower looks like it might be the best for the complete beginner. You know it’s for noobs, of course, because they make a client for the Mac :)

I haven’t seen a lot of noise about this, but the EU has outlawed pre-ticked checkboxes.  We have always recommended that these are not used, since they are evil UX, but now there’s an argument that might persuade everyone.

Here is a really nice post about splitting user stories. I think we are pretty good at this anyhow, but this is a nice way of describing the approach.

@monkchips gave a talk at IBM Impact about the effect of Mobile First. I think we’re on the right page with most of these things, but it’s interesting to see mobile called-out as one of the key drivers for these changes.

I’d not come across the REST Cookbook before, but here is a decent summary of how to treat PUT vs POST when designing RESTful APIs.

Fastly have produced a spectacularly detailed article about how to get tracking cookies working with Varnish.  This is very relevant to consumer facing projects.

This post from Thought Works is absolutely spot on, and I think accurately describes an important aspect of testing The Software Testing Cupcake.

As an example for how to make unit tests less fragile, this is a decent description of how to isolate tests, which is a key technique.

The examples are Ruby, but the principle is valid everywhere. Still on unit testing, Facebook have open sourced a Javascript unit testing framework called Jest. It looks really very good.

A nice implementation of “sudo mode” for Django. This ensures the user has recently entered their password, and is suitable for protecting particularly valuable assets in a web application like profile views or stored card payments.

If you are using Redis directly from Python, rather than through Django’s cache wrappers, then HOT Redis looks useful. This provides atomic operations for compound Python types stored within Redis.

Backbone history and IE9

This bit me the other day, so I thought I’d share the pain.

IE9 doesn’t support pushState as you probably know which meant everything was routing to root (as it were).

The following snippet checks and resorts to hash based routing if it can’t cut the mustard:

app.on('initialize:after', function() {
    if(Backbone.history && !Backbone.History.started) {
        if(!(window.history && history.pushState)) {
            Backbone.history.start({ pushState: false, silent: true });
            var fragment = window.location.pathname.substr(
                Backbone.history.options.root.length);
            Backbone.history.navigate(fragment, { trigger: true });
        }
        else {
            Backbone.history.start({ pushState: true });
        }
    }
});

Add it wherever you would initialize Backbone history – often the entry point of the app. Mine for instance has an app.js that is initialised by main.js

About us: Isotoma is a bespoke software development company based in York and London specialising in web apps, mobile apps and product design. If you’d like to know more you can review our work or get in touch.

Polite user interfaces know when to wait a little

Web page elements that appear or disappear on hover should almost always do so with a slight delay. Why?

  • To prevent distracting elements leaping out at you while your mouse is simply traversing the page.
  • To prevent you from accidentally clicking something that popped into view just as you were moving your cursor towards the target.
  • To prevent elements such as menus from unexpectedly disappearing when you just stray a pixel off, forcing you to re-invoke them.

Building in a small delay (say, 100ms) before elements appear or disappear is a hallmark of polite user interfaces, but is woefully rare. If you do a Google search for JavaScript plugins for menus, dropdowns, etc., you’ll find almost none that do this. This is also the biggest problem I have with using CSS :hover to show or hide elements (and why I think pure CSS dropdown menus are useless.)

On pretty much all projects with interactive JavaScript elements I’ve worked on in the past, I’ve specified this behaviour, which added considerable complexity for the developer. In most cases, they developed their solution from scratch.

So I was very happy to discover Brian Cherne’s hoverIntent jQuery plugin, a lightweight (4KB unminified) script which makes this effortless to do:

HoverIntent is similar to jQuery’s hover. However, instead of calling onMouseOver and onMouseOut functions immediately, this plugin tracks the user’s mouse onMouseOver and waits until it slows down before calling the onMouseOver function… and it will only call the onMouseOut function after an onMouseOver is called.

Please consider using it on your next project!

About us: Isotoma is a bespoke software development company based in York and London specialising in web apps, mobile apps and product design. If you’d like to know more you can review our work or get in touch.

Jonathan Stark – “Building mobile apps with HTML, CSS and JavaScript” workshop

Jonathan began with an overview of the current mobile landscape. In a nutshell it’s a minefield of platform fragmentation and device fragmentation.

“‘It’s brutal to develop natively across platforms”

Jonathan Stark
So what to do? Well, we then looked at three approaches; native, web and hybrid. All have their advantages but hybrid looked the way to go for most mobile use cases. In Jonathan’s words the advantages are:
  • Web files bundled in native executable
  • Access to device APIs
  • Sell in native app stores
  • Not perfect, but pretty damn good

Next we looked at some useful guidelines for the mobile use case. A nice summation would be “Do one thing and do it well”.

CSS3

We looked at various CSS3 features; advanced styling, transforms, transitions, animations etc. There’s a wealth of information out there on these so I won’t go into too much detail. Here are some interesting pointers:
  • android requires text-shadow to have a blur value.
  • box-shadow has performance issues, particularly on Blackberry (consider using media-query to turn off for Blackberry).
  • border-image is particularly interesting, as it does away with sliding doors css.
  • animations are hardware accelerated on iOS.
  • be careful of z-index on transforms. By default on it stacks above static positioned elements.
  • box-reflect works with form elements.
  • box-shadow takes up space like borders.
  • box-reflect works on <VIDEO> tag.
  • Talk of hardware acceleration for animation in the next version of Android  (Ice Cream Sandwich).
  • using lots of CSS3 gradients, animations, box-shadow will cause performance issues, tread careful and test often.
I liked the look of CSS animations. I have some reservations of how manageable they will become on medium to large projects. Having written code based animation sequences in Flash I know from experience it can get pretty complicated managing dependances and interactions. Using LESS, or SASS looks like an absolute must for using CSS animation on anything but the smallest projects.

HTML5

Next up was looking at some of the useful offline features of HTML5. Namely; web storage, web SQL database and application cache. My personal favourite was web storage – it looked easy to use and very useful for state maintenance. Application cache also looked great but I can see huge problems debugging, even Jonathan struggled to demo it properly.

Phonegap, jQTouch and Sencha Touch

Here we looked at the various frameworks for building hybrid apps. Again there is a ton of info out there. Jonathan was obviously keen on jQTouch but I actually think Phonegap looked the more promising platform for development. It seemed to have a good balance between ease of use, learning curve and features. Sencha Touch looked really powerful, whilst  jQTouch looked lightweight and quick to build with.

Debugging

In the final part of the day Jonathan showed some great debugging tools. If you’ve ever had to debug mobile CSS you know what a pain it is to debug on the device itself.  The stand out star for me was weinre, a really fantastic tool for debugging mobile apps.  The basic premise is it gives you access to the webkit inspector on your laptop which in turn is linked to the phone browser. It allows you to see exactly what is happening on the phone and allows live editing as well. Pretty amazing really.

Conclusion

Jonathan’s workshop has re-enforced my opinion that writing mobile apps for all formats is best done using HTML5 / CSS and JS.  The key benefits are:
  • write once*
  • The skills are already available and less costly than hiring native specific developers
  • one central code base
* They’ll always be some platform specific code needed.
All the slides and downloads from Jonathan’s talk are available here: http://jonathanstark.com/talks-wdx4.php

Javascript localization within Plone

For a while now, I’ve been trying to think of the best way to get localized strings into Javascript under Plone. Many existing packages and parts of Plone use one Javascript file per language, containing all translations. Whilst this is effective, it doesn’t “feel right” having all your translations distributed to the user on each page load, much less having them drawn from two different sources. Worse still (in my mind) is having translations dumped out into hidden HTML elements and then recalled by Javascript using getElementById or some such.

The solution I’ve eventually settled on is one that combines AJAX and client-side caching in cookies to minimize page load time. The implementation consists of a single view, a Javascript file and a bunch of XML & ZCML to tie it all together.

The powers that be here at Isotoma, have kindly agreed to let me open source this under the Apache 2.0 license.

Full documentation can be found for the code can be found in the README.txt, however I’ve included a quickstart guide below.

Continue reading

Best practices for speeding up your site

Best Practices for Speeding Up Your Web Site — part of an excellent series of articles on the Yahoo Developer Network.

They mainly focus on the front end, as that is where you’ll get most bang for your buck. They make frequent reference to YSlow, a Firebug extension that looks extremely useful.

The presentation here is also strongly recommended, as it makes succinct arguments for all the techniques. (View on Slideshare to see it full-screen.)

Some things were familiar to me (and most are already standard practice for my colleagues), many weren’t (e.g. avoid @import for CSS), and some I know of but haven’t been convinced of before (e.g. CSS sprites). I was quite intrigued by this example of CSS sprites in use on Google. (When using CSS sprites, remember to consider their accessibility to screen readers, and don’t use if you want them to print.)

I’d rather eat glass

A really interesting blog post from Brendan Eich regarding multithreading in Javascript.

It looks like JS might become the Next Big Language on the server as well as the client, so Brendan’s opinions really really matter. I am very glad to see he’s drinking the right kool aid. He doesn’t want explicit multithreading, and he namechecks Erlang, a language I am rather fond of.

Even more interestingly, he says:

In all the fast yet maintainable MT systems I’ve built or worked on, the key idea (which Will Clinger stated clearly to me over lunch last fall) is to separate the mutable unshared data from the immutable shared data. Do that well, with language and VM support, and threads become what they should be: not an abstraction violator from hell, but a scaling device that can be composed with existing abstractions.

Which is absolutely correct. I wonder if he’s thinking along the lines of Oz which uses these ideas explicitly to provide a quite different multiprogramming mechanism.

MochiKit.Animator

The release of MochiKit.Animator feels like quite a major thing here. You can read some comments by Kevin Dangoor and Ajaxian.

It’s a fantastic API, and it really looks and smells far better than Scriptaculous, the current king of the javascript animation space.

What’s really nice about MochiKit.Animation is it’s use of the functional style, which produces terse but expressive source that is often “obviously” right – much better than Object Orientation for a lot of tasks. Seeing this applied so neatly to animation is fantastic.