Category Archives: Javascript

Observations on the nature of time. And javascript.

In the course of working on one of our latest projects, I picked up an innocuous looking ticket that said: “Date pickers reset to empty on form submission”. “Easy”, I thought. It’s just the values being lost somewhere in form validation.And then I saw the ‘in Firefox and IE’ description. Shouldn’t be too hard, it’ll be a formatting issue or something, maybe even a placeholder, right?

Yeah, no.

Initial Investigations

Everything was fine in Chrome, but not in Firefox. I confirmed the fault also existed in IE (and then promptly ignored IE for now).

The responsible element looked like this:
<input class="form-control datepicker" data-date-format="{{ js_datepicker_format }}" type="date" name="departure_date" id="departure_date" value="{{ form.departure_date.value|default:'' }}">

This looks pretty innocent. It’s a date input, how wrong can that be?

Sit comfortably, there’s a rabbit hole coming up.

On Date Inputs

Date type inputs are a relatively new thing, they’re in the HTML5 Spec. Support for it is pretty mixed. This jumps out as being the cause of it working in Chrome, but nothing else. Onwards investigations (and flapping at colleagues) led to the fact that we use bootstrap-datepicker to provide a JS/CSS based implementation for the browsers that have no native support.

We have an isolated cause for the problem. It is obviously something to do with bootstrap-datepicker, clearly. Right?

On Wire Formats and Localisation

See that data-date-format="{{ js_datepicker_format }}" attribute of the input element. That’s setting the date format for bootstrap-datepicker. The HTML5 date element doesn’t have similar. I’m going to cite this stackoverflow answer rather than the appropriate sections of the documentation. The HTML5 element has the concept of a wire format and a presentation format. The wire format is YYYY-MM-DD (iso8601), the presentation format is whatever the user has the locale set to in their browser.

You have no control over this, it will do that and you can do nothing about it.

bootstrap-datepicker, meanwhile has the data-date-format element, which controls everything about the date that it displays and outputs. There’s only one option for this, the wire and presentation formats are not separated.

This leads to an issue. If you set the date in YYYY-MM-DD format for the html5 element value, then Chrome will work. If you set it to anything else, then Chrome will not work and bootstrap-datepicker might, depending on if the format matches what is expected.

There’s another issue. bootstrap-datepicker doesn’t do anything with the element value when you start it. So if you set the value to YYYY-MM-DD format (for Chrome), then a Firefox user will see 2015-06-24, until they select something, at which point it will change to whatever you specified in data-date-format. But a Chrome user will see it in their local format (24/06/2015 for me, GB format currently).

It’s all broken, Jim.

A sidetrack into Javascript date formats.

The usual answer for anything to do with dates in JS is ‘use moment.js’. But why? It’s a fairly large module, this is a small problem, surely we can just avoid it?

Give me a date:

>>> var d = new Date();
undefined

Lets make a date string!

>>> d.getYear() + d.getMonth() + d.getDay() + ""
"123"

Wat. (Yeah, I know that’s not how you do string formatting and therefore it’s my fault.)

>>> d.getDay()
3

It’s currently 2015-06-24. Why 3?.

Oh, that’s day of the week. Clearly.

>>> d.getDate()
24

The method that gets you the day of the month is called getDate(). It doesn’t, you know, RETURN A DATE.

>>> var d = new Date('10-06-2015')
undefined
>>> d
Tue Oct 06 2015 00:00:00 GMT+0100 (BST)

Oh. Default date format is US format (MM-DD-YYYY). Right. Wat.

>>> var d = new Date('31-06-2015')
undefined
>>> d
Invalid Date

That’s… reasonable, given the above. Except that’s a magic object that says Invalid Date. But at least I can compare against it.

>>> var d = new Date('31/06/2015')
undefined
>>> d
Invalid Date

Oh great, same behaviour if I give it UK date formats (/ rather than -). That’s okay.

>>> var d = new Date('31/06/2015')
undefined
>>> d
"Date 2017-07-05T23:00:00.000Z"

Wat.

What’s going on?

The difference here is that I’ve used Firefox, the previous examples are in Chrome. I tried to give an explanation of what that’s done, but I actually have no idea. I know it’s 31 months from something, as it’s parsed the 31 months and added it to something. But I can’t work out what, and I’ve spent too long on this already. Help. Stop.

So. Why you should use moment.js. Because otherwise the old great ones will be summoned and you will go mad.

Also:

ISO Date Format is not supported in Internet Explorer 8 standards mode and Quirks mode.

Yep.

The Actual Problem

Now I knew all of this, I could see the problem.

  1. The HTML5 widget expects YYYY-MM-DD
  2. The JS widget will set whatever you ask it to
  3. We were outputting GB formats into the form after submission
  4. This would then be an incorrect format for the HTML 5 widget
  5. The native widget would not change an existing date until a new one is selected, so changing the output format to YYYY-MM-DD meant that it changed when a user selected something.

A Solution In Two Parts

The solution is to standardise the behaviour and formats across both options. Since I have no control over the HTML5 widget, looks like it’s time to take a dive into bootstrap-datepicker and make that do the same thing.

Deep breath, and here we go…

Part 1

First job is to standardise the output date format in all the places. This means that the template needs to see a datetime object, not a preformatted date.

Once this is done, can feed the object into the date template tag, with the format filter. Which takes PHP date format strings. Okay, that’s helpful in 2015. Really.

Figured that out, changed the date parsing Date Input Formats and make sure it has the right ISO format in it.

That made the HTML5 element work consistently. Great.

Then, to the javascript widget.

bootstrap-datepicker does not do anything with the initial value of the element. To make it behave the same as the HTML5 widget, you need to:

1. Get the locale of the user

2. Get the date format for that locale

3. Set that as the format of the datepicker

4. Read the value

5. Convert the value into the right format

6. Call the setValue event of the datepicker with that value

This should be relatively straightforward, with a couple of complications.

  1. moment.js uses a different date format to bootstrap-datepicker
  2. There is no easy way to get a date format string, so a hardcoded list is the best solution.

// taken from bootstrap-datepicker.js
function parseFormat(format) {
    var separator = format.match(/[.\/\-\s].*?/),
        parts = format.split(/\W+/);
    if (!separator || !parts || parts.length === 0){
        throw new Error("Invalid date format.");
    }
    return {separator: separator, parts: parts};
}

var momentUserDateFormat = getLocaleDateString(true);
var datepickerUserDateFormat = getLocaleDateString(false);

$datepicker.each(function() {
    var $this = $(this);
    var presetData = $this.val();
    $this.data('datepicker').format = parseFormat(datepickerUserDateFormat);
    if (presetData) {
        $this.datepicker('setValue', moment(presetData).format(momentUserDateFormat));
    }
});

A bit of copy and paste code from the bootstrap-datepicker library, some jquery and moment.js and the problem is solved.

Part 3

Now we have the dates displaying in the right format on page load, we need to ensure they’re sent in the right format after the user has submitted the form. Should just be the reverse operation.

 function rewriteDateFormat(event) {
    var $this = $(event.data.input);
    if ($this.val()) {
        var momentUserDateFormat = getLocaleDateString(true);
        $this.val(moment($this.val(), [momentUserDateFormat, 'YYYY-MM-DD']).format('YYYY-MM-DD'));
    }
}

$datepicker.each(function() {
    var $this = $(this);
     // set the form handler for rewriting the format on submit
    var $form = $this.closest('form');
    $form.on('submit', {input: this}, rewriteDateFormat);
});

And we’re done.

Takeaways

Some final points that I’ve learnt.

  1. Always work in datetime objects until the last possible point. You don’t have to format them.
  2. Default to ISO format unless otherwise instructed
  3. Use parsing libraries

 

Links

There is a new version of gunicorn, 19.0 which has a couple of significant changes, including some interesting workers (gthread and gaiohttp) and actually responding to signals properly, which will make it work with Heroku.

The HTTP RFC, 2616, is now officially obsolete. It has been replaced by a bunch of RFCs from 7230 to 7235, covering different parts of the specification. The new RFCs look loads better, and it’s worth having a look through them to get familiar with them.

Some kind person has produced a recommended set of SSL directives for common webservers, which provide an A+ on the SSL Labs test, while still supporting older IEs. We’ve struggled to find a decent config for SSL that provides broad browser support, whilst also having the best levels of encryption, so this is very useful.

A few people are still struggling with Git.  There are lots of git tutorials around the Internet, but this one from Git Tower looks like it might be the best for the complete beginner. You know it’s for noobs, of course, because they make a client for the Mac :)

I haven’t seen a lot of noise about this, but the EU has outlawed pre-ticked checkboxes.  We have always recommended that these are not used, since they are evil UX, but now there’s an argument that might persuade everyone.

Here is a really nice post about splitting user stories. I think we are pretty good at this anyhow, but this is a nice way of describing the approach.

@monkchips gave a talk at IBM Impact about the effect of Mobile First. I think we’re on the right page with most of these things, but it’s interesting to see mobile called-out as one of the key drivers for these changes.

I’d not come across the REST Cookbook before, but here is a decent summary of how to treat PUT vs POST when designing RESTful APIs.

Fastly have produced a spectacularly detailed article about how to get tracking cookies working with Varnish.  This is very relevant to consumer facing projects.

This post from Thought Works is absolutely spot on, and I think accurately describes an important aspect of testing The Software Testing Cupcake.

As an example for how to make unit tests less fragile, this is a decent description of how to isolate tests, which is a key technique.

The examples are Ruby, but the principle is valid everywhere. Still on unit testing, Facebook have open sourced a Javascript unit testing framework called Jest. It looks really very good.

A nice implementation of “sudo mode” for Django. This ensures the user has recently entered their password, and is suitable for protecting particularly valuable assets in a web application like profile views or stored card payments.

If you are using Redis directly from Python, rather than through Django’s cache wrappers, then HOT Redis looks useful. This provides atomic operations for compound Python types stored within Redis.

Backbone history and IE9

This bit me the other day, so I thought I’d share the pain.

IE9 doesn’t support pushState as you probably know which meant everything was routing to root (as it were).

The following snippet checks and resorts to hash based routing if it can’t cut the mustard:

app.on('initialize:after', function() {
    if(Backbone.history && !Backbone.History.started) {
        if(!(window.history && history.pushState)) {
            Backbone.history.start({ pushState: false, silent: true });
            var fragment = window.location.pathname.substr(
                Backbone.history.options.root.length);
            Backbone.history.navigate(fragment, { trigger: true });
        }
        else {
            Backbone.history.start({ pushState: true });
        }
    }
});

Add it wherever you would initialize Backbone history – often the entry point of the app. Mine for instance has an app.js that is initialised by main.js

About us: Isotoma is a bespoke software development company based in York and London specialising in web apps, mobile apps and product design. If you’d like to know more you can review our work or get in touch.

Polite user interfaces know when to wait a little

Web page elements that appear or disappear on hover should almost always do so with a slight delay. Why?

  • To prevent distracting elements leaping out at you while your mouse is simply traversing the page.
  • To prevent you from accidentally clicking something that popped into view just as you were moving your cursor towards the target.
  • To prevent elements such as menus from unexpectedly disappearing when you just stray a pixel off, forcing you to re-invoke them.

Building in a small delay (say, 100ms) before elements appear or disappear is a hallmark of polite user interfaces, but is woefully rare. If you do a Google search for JavaScript plugins for menus, dropdowns, etc., you’ll find almost none that do this. This is also the biggest problem I have with using CSS :hover to show or hide elements (and why I think pure CSS dropdown menus are useless.)

On pretty much all projects with interactive JavaScript elements I’ve worked on in the past, I’ve specified this behaviour, which added considerable complexity for the developer. In most cases, they developed their solution from scratch.

So I was very happy to discover Brian Cherne’s hoverIntent jQuery plugin, a lightweight (4KB unminified) script which makes this effortless to do:

HoverIntent is similar to jQuery’s hover. However, instead of calling onMouseOver and onMouseOut functions immediately, this plugin tracks the user’s mouse onMouseOver and waits until it slows down before calling the onMouseOver function… and it will only call the onMouseOut function after an onMouseOver is called.

Please consider using it on your next project!

About us: Isotoma is a bespoke software development company based in York and London specialising in web apps, mobile apps and product design. If you’d like to know more you can review our work or get in touch.

Jonathan Stark – “Building mobile apps with HTML, CSS and JavaScript” workshop

Jonathan began with an overview of the current mobile landscape. In a nutshell it’s a minefield of platform fragmentation and device fragmentation.

“‘It’s brutal to develop natively across platforms”

Jonathan Stark
So what to do? Well, we then looked at three approaches; native, web and hybrid. All have their advantages but hybrid looked the way to go for most mobile use cases. In Jonathan’s words the advantages are:
  • Web files bundled in native executable
  • Access to device APIs
  • Sell in native app stores
  • Not perfect, but pretty damn good

Next we looked at some useful guidelines for the mobile use case. A nice summation would be “Do one thing and do it well”.

CSS3

We looked at various CSS3 features; advanced styling, transforms, transitions, animations etc. There’s a wealth of information out there on these so I won’t go into too much detail. Here are some interesting pointers:
  • android requires text-shadow to have a blur value.
  • box-shadow has performance issues, particularly on Blackberry (consider using media-query to turn off for Blackberry).
  • border-image is particularly interesting, as it does away with sliding doors css.
  • animations are hardware accelerated on iOS.
  • be careful of z-index on transforms. By default on it stacks above static positioned elements.
  • box-reflect works with form elements.
  • box-shadow takes up space like borders.
  • box-reflect works on <VIDEO> tag.
  • Talk of hardware acceleration for animation in the next version of Android  (Ice Cream Sandwich).
  • using lots of CSS3 gradients, animations, box-shadow will cause performance issues, tread careful and test often.
I liked the look of CSS animations. I have some reservations of how manageable they will become on medium to large projects. Having written code based animation sequences in Flash I know from experience it can get pretty complicated managing dependances and interactions. Using LESS, or SASS looks like an absolute must for using CSS animation on anything but the smallest projects.

HTML5

Next up was looking at some of the useful offline features of HTML5. Namely; web storage, web SQL database and application cache. My personal favourite was web storage – it looked easy to use and very useful for state maintenance. Application cache also looked great but I can see huge problems debugging, even Jonathan struggled to demo it properly.

Phonegap, jQTouch and Sencha Touch

Here we looked at the various frameworks for building hybrid apps. Again there is a ton of info out there. Jonathan was obviously keen on jQTouch but I actually think Phonegap looked the more promising platform for development. It seemed to have a good balance between ease of use, learning curve and features. Sencha Touch looked really powerful, whilst  jQTouch looked lightweight and quick to build with.

Debugging

In the final part of the day Jonathan showed some great debugging tools. If you’ve ever had to debug mobile CSS you know what a pain it is to debug on the device itself.  The stand out star for me was weinre, a really fantastic tool for debugging mobile apps.  The basic premise is it gives you access to the webkit inspector on your laptop which in turn is linked to the phone browser. It allows you to see exactly what is happening on the phone and allows live editing as well. Pretty amazing really.

Conclusion

Jonathan’s workshop has re-enforced my opinion that writing mobile apps for all formats is best done using HTML5 / CSS and JS.  The key benefits are:
  • write once*
  • The skills are already available and less costly than hiring native specific developers
  • one central code base
* They’ll always be some platform specific code needed.
All the slides and downloads from Jonathan’s talk are available here: http://jonathanstark.com/talks-wdx4.php

Javascript localization within Plone

For a while now, I’ve been trying to think of the best way to get localized strings into Javascript under Plone. Many existing packages and parts of Plone use one Javascript file per language, containing all translations. Whilst this is effective, it doesn’t “feel right” having all your translations distributed to the user on each page load, much less having them drawn from two different sources. Worse still (in my mind) is having translations dumped out into hidden HTML elements and then recalled by Javascript using getElementById or some such.

The solution I’ve eventually settled on is one that combines AJAX and client-side caching in cookies to minimize page load time. The implementation consists of a single view, a Javascript file and a bunch of XML & ZCML to tie it all together.

The powers that be here at Isotoma, have kindly agreed to let me open source this under the Apache 2.0 license.

Full documentation can be found for the code can be found in the README.txt, however I’ve included a quickstart guide below.

Continue reading

Best practices for speeding up your site

Best Practices for Speeding Up Your Web Site — part of an excellent series of articles on the Yahoo Developer Network.

They mainly focus on the front end, as that is where you’ll get most bang for your buck. They make frequent reference to YSlow, a Firebug extension that looks extremely useful.

The presentation here is also strongly recommended, as it makes succinct arguments for all the techniques. (View on Slideshare to see it full-screen.)

Some things were familiar to me (and most are already standard practice for my colleagues), many weren’t (e.g. avoid @import for CSS), and some I know of but haven’t been convinced of before (e.g. CSS sprites). I was quite intrigued by this example of CSS sprites in use on Google. (When using CSS sprites, remember to consider their accessibility to screen readers, and don’t use if you want them to print.)

I’d rather eat glass

A really interesting blog post from Brendan Eich regarding multithreading in Javascript.

It looks like JS might become the Next Big Language on the server as well as the client, so Brendan’s opinions really really matter. I am very glad to see he’s drinking the right kool aid. He doesn’t want explicit multithreading, and he namechecks Erlang, a language I am rather fond of.

Even more interestingly, he says:

In all the fast yet maintainable MT systems I’ve built or worked on, the key idea (which Will Clinger stated clearly to me over lunch last fall) is to separate the mutable unshared data from the immutable shared data. Do that well, with language and VM support, and threads become what they should be: not an abstraction violator from hell, but a scaling device that can be composed with existing abstractions.

Which is absolutely correct. I wonder if he’s thinking along the lines of Oz which uses these ideas explicitly to provide a quite different multiprogramming mechanism.

MochiKit.Animator

The release of MochiKit.Animator feels like quite a major thing here. You can read some comments by Kevin Dangoor and Ajaxian.

It’s a fantastic API, and it really looks and smells far better than Scriptaculous, the current king of the javascript animation space.

What’s really nice about MochiKit.Animation is it’s use of the functional style, which produces terse but expressive source that is often “obviously” right – much better than Object Orientation for a lot of tasks. Seeing this applied so neatly to animation is fantastic.