wheresrhys

I’m starting to find that when I ask javascript questions on forums similar to

I’ve been using such and such a method to achieve something, but there must be a better way. What is it?

Then the answer that comes back is quite frequently (though far from always)

Your method is probably the best way.

Which is a nice feeling. As I’m a self-taught programmer I’ve always been very aware that most people I work with have studied and worked as programmers for years and years, and inevitably* have picked up a lot of tricks along the way… and that try as I might I can’t just learn these from a book or a website (though I’ll give it my best shot); it’s just a matter of building up years of experience, banging up against this or that obstacle and learning from other programmers.

So, considering that a year ago I didn’t know javascript, or any programming language, I’m pretty pleased that I can confidently lay claim to not being a complete moron any more.

*though it’s surprising how many exceptions to this rule there are.

I’ve now written two jQuery plugins, and have a pretty good idea for a third. As you might expect, I’ve learned quite a few things along the way. When I started there was a lot I didn’t know about javascript and jQuery. There still is, but one thing that struck me is that there are a lot of great tutorials on writing jQuery plugins out there but none of them individually captures all the things that took my plugins from being nice ideas with mediocre implementations to being something I’m proud of.

So here is my definitive combination of them all, together with one or two of my own contributions (a bit scant on detail, but follow the links for more info).

1. Build a good test page

My test page for any given plugin has 3 instances of a type of element the plugin is suppose to effect. eg a list. They are marked up as follows:

<ul class="single" >...</ul>
<ul class="multiple" >...</ul>
<ul class="multiple" >...</ul>

And I run the following script in the document head

$(document).ready(function() {
  $('.single').myPlugin();
  $('.multiple').myPlugin();
});

This will make sure your plugin works when called more than once on a page, and when one instance affects more than one element. For testing your code for speed the same page with various bits of html copied lots of times does the trick.

2. Follow jQuery’s recommended tutorial

It really is very good, and covers most important points. The other items on the list are either little tweaks that I think make the code neater or, in just one or two cases (highlighted in red), things its missed out. Read the tutorial

3. Make sure your plugin won’t conflict with other javascript libraries

The most effective way to do this is to wrap your whole plugin in a function where $ is its parameter, and then call it on jQuery, like so:

(function($) {
  $.fn.pluginName = function(options) { ... };
})(jQuery);

More info

4. Make your plugin’s settings changeable, both when the plugin is called, and by tweaking the default settings

The way to do this is to set up your plugin’s defaults as a JSON which is a public property of the plugin object. So within your plugin add these lines (where options is the name of the JSON passed to the plugin as a parameter).

this.defaults = {parameter:'value', ...};
options = $.extend(defaults,options);

More info

5. Make sure your plugin keeps track of DOM elements effectively

Structure your code as follows in order to make sure your variables are accessible to the right instances of the plugin, but aren’t stored in memory longer than they need to be.

$.fn.pluginName = function(options) {
  var constructor1, constructor2, ...;//elements that are used only when the plugIn is constructing instances of itself at the start and play no role in handling later events;
  return this.each(function() {
     var tracker1, tracker2, ...;//elements that are important for events and other later changes. Also, if your plugin's constructor involves some AJAX it may be best to put all variables in here
     subFunction() {
	//this should always be able to call the variables it needs, in particular access the right DOM elements
     }
  });
};

6. Optimise, opimise, optimise

You never know how demanding the website using your plugin is going to be: how many times is it going to be called on one page; will the dom objects (eg sorting a list of items) it has to handle be a lot bigger than the ones you’ve tested it on. My crossSelect plugin was, I discovered, before version 4/5, really sluggish on lists larger than about 20 items.

So I found a great list of techniques to optimise your code, the most pertinent to writing plugins I have put below:

• Make sure your optimisations are optimisations

  To keep track of whether your optimisations are having the desired effect, run your plugin on a page full of huge elements for it to process and then use firebug to measure the time it takes.

• Don’t over-query the DOM

  jQuery is principallya tool for accessing and manipulating the DOM, but it doesn’t pay to do this too often as your code will be a lot quicker if you keep it to a minimum. Techniques for avoiding this include:

  • store DOM elements you know you will have to access a few times in variables (but be careful – see point 5)
  • Add new DOM elements as html, . So instead of

    $('selector').append('<ul>');
    for(i=0;i<limit;i++)
    {
      $('selector > ul').append('<li>'+i-th text+'<li>');
    }

    use

    string = '<ul>';
    for(i=0;i<limit;i++)
    {
      string+='<li>'+i-th text+'<li>';
    }
    $('selector').append(string);

    The improvement in speed of code is astounding.

  • Use contexts. A lot. It will both reduce code size (as shown below) and speed up the code (because jQuery only has to search a subset of the DOM). So instead of

    var theContainer = $('selector');
    $(theContainer).children('ul').children('li').aMethod();

    use

    var theContainer = $('selector');
    $('ul>li', theContainer).aMethod();

And that’s yer lot. Please let me know if you have any comments or suggested additions.

‘It’ being object oriented programming (OOP).

For a long time I got frustrated with reading tutorials telling you about inheritance and other useful features of using OOP’s class-based approach. The trouble is that in the example scenarios given (eg a ‘bird’ class which inherits from an ‘animal’ class) the set theory-like approach to writing code to represent things makes sense. But when it comes to building a website or something similar, and creating classes which do not display a nested structure, the analogy with set theory falls away and largely (at least in the websites I’ve worked on) you’re left with a convoluted structure which, at first glance, could just be done away with in favour of using normal variables and functions instead of properties and methods. I could see how perhaps the stronger structure might help organise code, and was gradually getting mor ecomfortable with the techniques, but still didn’t really get why using OOP was so important.

But now I think I do, courtesy of wikipedia, which has the best summary of the motivation behind OOP I’ve ever read:

The methodology focuses on data rather than processes, with programs composed of self-sufficient modules (objects) each containing all the information needed to manipulate its own data structure. This is in contrast to the existing modular programming which had been dominant for many years that focused on the function of a module, rather than specifically the data, but equally provided for code reuse, and self-sufficient reusable units of programming logic, enabling collaboration through the use of linked modules (subroutines). This more conventional approach, which still persists, tends to consider data and behavior separately.

After reading this it’s all become clear to me what the point is – creating a more concrete connection between a thing and what can be done to it. In a real world situation I would liken it to having the steering wheel in the car, always ready to be used to steer the car, as opposed to having a steering wheel which can be passed around and maybe tried on other things, eg a chicken, a whale, a front-bench tory… where the results would be unpredictable (apart from the tory, who would almost certainly find the experience arousing). And as long as you define the steering wheel at a high enough level (eg the ‘vehicle’ class has a steering wheel, which the ‘car’ class inherits) you don’t fall foul of code duplication either.

I won’t go into details, but I’ve had quite a tough month or so in the web programming side of my life. A lot of hard work and dedication leading to a very unsatisfactory ending – the first time in my adult life I’ve left a job with a bitter taste in my mouth.

So I thought I’d write this to say thank you to everyone who has, in the same time period, contributed to the small flood of comments relating to my two jQuery plugins – fullTextArea and crossSelect. Up until recently I had no idea anyone was even using them, so it really brightened the last 4 weeks to know that the free time I put into programming is appreciated (even if the same can’t be said for paid work all the time).

So thank you again for restoring my faith in the programming community. I’ll get on to incorporating the best suggestions as soon as I can.

In previous posts I’ve covered the inherent difficulty of geocoding addresses and postcodes in the UK, specifically in order use the geocodes with Google maps. I learned a lot about the limitations of Google’s various geocoding services. To sum up the situation:

• The Google maps API’s geocoding webservice is buggy: giving more address details can give a less accurate, and in many cases plain wrong, geocode. However, the GClientGeocoder class built into Google maps fares a lot better with addresses…
• … apart from when the address entered is a postcode. Then it geocodes to the central point of the postcode sector, rather than the precise postcode location, i.e if you search for E3 4GY or E3 4NT, it will return the geocode for the centre of E3 4 in both cases.
• But, as I discovered when trying an alternative route to postcode geocoding, the Google Ajax search API, the localSearch class in particular, gives very accurate geocodes for full postcodes.

Previously I’d been geocoding large batches of addresses/postcodes, but having got the geocodes in order to plot points on the map, I wanted to add search functionality to the map, so that it would zoom in on a given address/postcode. I wanted it to be accurate for both addresses and postcodes, so using what I’d learned, I wrote a javascript function, geocodeUKAddress, which always returns the best geocode that Google can offer, so your website can be as reliable as a London cabbie (again, I can’t embed it here as javascript has a habit of breaking wordpress, though there must be some way to make it safe – I will research).

You will need to include the following in ther head of your webpage too

<script src="http://maps.google.com/maps?file=api&amp;v=2&amp;key=your_api_key_here" type="text/javascript"></script>
<script src="http://www.google.com/jsapi" type="text/javascript"></script>

I can’t take credit for the regular expression that recognises UK postcodes, but apart from that, it’s all my own work, which anyone can feel free to use. Unless you maybe work for Rupert Murdoch.

The art of geocoding addresses in the UK, as I previously explained, is a soul-destroying process, frought with inaccuracy, bugs and convoluted workarounds. And for all that work you end up with a set of points of which a great deal are probably somewhat inaccurate and at least some of which are completely wrong. UK addresses (and probably those elsewhere in the world) are complicated creatures, which Google’s geocoding engine often interprets wrongly.

Postcodes, on the other hand, are rather easier; there is a well-defined relationship between a UK postcode and its corresponding (usually pretty small) piece of the British countryside. But google’s geocoding api will only return a geocode for the postcode sector (ie will give a geocode for LL12 5 when you searched for LL12 5TH). However, someone did figure out a way of using Google’s local search API combined with google maps to geocode UK postcodes. Since he blogged about it the API has changed, so below is an outline of how to geocode a batch of postcodes in the UK using just some simple php, the current google ajax search API and a little javascript (jQuery isn’t essential, but cuts down on coding a bit). The javascript is the crucial step.

Assuming you have a database full of postcodes and id numbers, and 2 empty columns to store latitude and longitude values, this is how it’s done. (Download source geocode.zip).

1. Create a html page geocode.html with the following content:

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" >

<head>
<title></title>
<meta name="description" content="" />
<meta http-equiv="content-type" content="text/html; charset=iso-8859-1" />
<link rel="stylesheet" href="" type="text/css" media="screen" />
<script type="text/javascript" src="jquery-1.3.2.js"></script>
<script src="http://www.google.com/jsapi" type="text/javascript"></script>
<script type="text/javascript" src="geocode.js"></script>
</head>
<body>
<div id="counter"></div>
</body>
</html>

(Make sure you specify the correct location for your local javascript files)

2. Create a php file (in the same directory), geocode.php, with the following rough structure (it will only be accessed via ajax, so is very stripped down):

<?php
require_once ('mysqlConnect.php'); //or other database connection details
if($_GET)
{
 //var_dump($_GET);
 update_record();
 send_new_data();
}

//gets the next record without a geocode and sends the id and postcode to the browser
function send_new_data() {
 $query = @mysql_query("SELECT id, postcode FROM geocode_table WHERE lat = '' AND postcode != '' ORDER BY id LIMIT 1");
 if(($query) &&mysql_num_rows($query)) {
  $row = mysql_fetch_array($query, MYSQL_ASSOC);
  echo $row['id'].','.$row['postcode'];
 } else {
  echo 'stop';
 }
}

//updates the last record with data sent from browser
function update_record() {
 $id = $_GET['id'];
 $lat = $_GET['lat'];
 $lng = $_GET['lng'];
 if($id > 0)
 {
  $update = "UPDATE geocode_table SET lat = '".$lat."', lng = '".$lng."' WHERE id = ".$id;
  $result = @mysql_query($update);
  if (!$result) {
   die('Invalid query: ' . mysql_error());
  }
 }
}
?>

3. Create a javascript file geocode.js, saved in the same directory again (I would paste it here but it keeps breaking wordpress)

4. Running the code

Once you’ve altered the database connection details, and SQL query to suit your setup, simply open geocode.html in your browser. A counter will tell you which record you’re on. To stop the code simply close your browser/browser tab.

How it all works

In a nutshell (ignoring the special case of starting off the loop) the code repeatedly performs the following process:

….in geocode.php, send_new_data() finds a record which has no latitude value and sends it’s id number and postcode as an ajax response to set_and_get_next(). This keeps track of the id in a global variable and sends the postcode to getPointFromPostcode(), which uses google’s local search to get a geocode. Once it’s found a geocode it passes it to set_and_get_next(), which sends it to geocode.php in an ajax request. There update_record()… well… updates the record, and send_new_data() finds a record which has no la….

Compared to my previous approach iterating a script over large sets of data, using ajax is very sleek. Similarly to a pure php script I can load from a browser, though with much of the resource intensive scripting taking place on my or google’s server. But with ajax there’s no problem with the browser timing out from time to time, or baulking at the number of times a page is requested. It’s a little harder to code, and probably less efficient… but I like it. And I’ll definitely be using my shiny new geocoded postcode data.

I haven’t blogged much about programming recently due to being too busy… too busy to blog, not too busy to program.

This is just a swift post to highlight an important distinction that I haven’t seen mentioned anywhere else, and which recently I realised the siginificance of.

In javascript, what’s the difference between

delete my_array[i];

and

my_array.splice(i, 1);

Superficially they’re the same – they each remove the ith element from an array. But the crucial difference is that delete leaves a gap in the array, so there will no longer be an ith element – the array jumps straight from the (i-1)th to the (i+1)th element. But splice moves the (i+1)th element into the ith place in the array, and so on, shortening the array by one but moving everything along to fill the gap.

Why is this important?

If your code relies often on iterating through an array thus:

for(i=0;i<my_array.length;i++)
{
    //some code
}

then any future iterations over the array will throw errors if you used delete to remove the ith element. But the iterations should still work if you used the splice method.

One note of caution, if you carry out

my_array.splice(i, 1);

within a loop over i, you should add the line

i--;

immediately after, to make sure you don’t miss out the next array element (which is now, of course, not in the (i+1)th place, but in the ith place).

No, this isn’t an artist’s impression of the UK trapped in the midst of a swine flu epidemic, but is in fact what all my geocoded points look like plotted on a map of the UK*. All 4,528 of them. Now I just have to figure out how to make this look presentable, i.e. feed the points to the map gradualy as the user zooms in. I bet this is where things will really start to get difficult.

*with apologies to Cornwall and northern Scotland

This damn economic crisis/swine flu outbreak isn’t quite that bad yet, but nevertheless there is a very limited sense where the UK is quite anarchic: geocoding addresses using Google Maps.

Having completed my download of addresses for my new Google Maps website the next stage was to geocode them so that I can plot them on the map. I had no idea how tricky it would be when I started out.

The most irritating and fundamental difficulty is that geocodes for UK postcodes are not available for free. The data is owned by the Royal Mail, and there is at least one website where you can buy access to this information (it has a free trial, but I discovered that this is just for about 10 or so geocodes). You can search by postcode on google maps, but if you put a postcode e.g. LL13 7YH into the geocoder API you’re given the geocode for LL13 7 – not accurate enough to be of any real use.

So you have to go for geocoding full addresses instead. The geocodes for these data isn’t owned by Royal Mail, but by the Ordnance Survey, and for some reason they are less restrictive about sharing the information. But there’s still a long hard slog before you can get the geocodes out of this.

Google offer a really useful turorial on geocoding addresses, and this, combined with my approach to iterating over a large number of records meant I was collecting the geocodes in no time. However, it wasn’t as peachy as it seemed.

For example, the address Llantysilio, Denbighshire, UK brings up a pretty accurate geocode for the village of Llantysilio in North Wales. However, the full address, including the postal town is Llantysilio, Llangollen, Denbighshire, UK, and this unexpectedly brings up the geocode for an address on Castle Street, right in the middle of Llangollen. So a more complete address leads to a far less accurate geocode. This is immensely problematic.

In general I was feeding in the longest possible address made up out of the data I had, so in my php script I had something like the following:

while(count($arr_address > 1) && !$str_lat)
 {
$str_address = implode(', ', $arr_address);
 attempt_geocode($url.$str_name.', '.$str_address.', '.$str_county.', UK');
 attempt_geocode($url.$str_address.', '.$str_county.', UK');
 attempt_geocode($url.$str_address.', UK');
 array_pop($arr_address);
 }

This starts with the longest, most detailed address string, and then gradually cuts the string down (possibly sacrificing accuracy in order to get a passable geocode), with attempt_geocode() exiting the loop on success.

But the fact that longer addresses can lead to incorrect geocodes meant I had to work in a way to start off with shorter addresses, and if that doesn’t get a geocode then gradually shorten them and keep trying to geocode. So I’ now have:

 attempt_geocode($url.$arr_address[0].', '.$str_county.', UK');
 attempt_geocode($url.$arr_address[0].', '.end($arr_address).', '.$str_county.', UK');
while(count($arr_address > 1) && !$str_lat)
{
 $str_address = implode(', ', $arr_address);
 attempt_geocode($url.$str_address.', '.$str_county.', UK');
 attempt_geocode($url.$str_address.', UK');
 array_pop($arr_address);
}

A long process in order to get a geocode that could still quite likely be wrong, and even if it’s basically correct might not be as accurate as a postcode; but nevertheless an improvement on what I had before.

A glimmer of hope though is that google maps itself doesn’t suffer from this issue – both address versions return the same accurate point on the map, and as someone pointed out to me on stackoverflow, google maps is in beta, so maybe teh geocoder API just hasn’t been updated to the newer, better address parser, and maybe one day reliable geocoding for free in the UK will be a reality. Also, somebody has found a way to geocode in the UK using postcodes, by hacking together the google maps and search APIs, and I may well try it, as this address geocoding malarchy leaves a lot to be desired. (*edit – turns out it’s heavily reliant on javascript so can’t be used for geocoding masses of pages without slowing down your browser.)

Finally, if this article wasn’t any help, there’s loads of geocoding links here.

While refining  jQuery.crossselect.js recently I was briefly faced with a problem which often rears its ugly head, though this time I found a solution.

Consider a function alters_item(), which can be applied to certain DOM elements. Further, consider that it can be triggered in two distinct ways:

1. By a click (or other event) on the item to be altered, so the DOM element is the context and can be accessed via the pseudo-variable this
2. Just applied like a normal javascript function, which means the DOM element needs to be passed in as a parameter, i.e. you need to call the function using alters_item(element)

So to have a function usable in both circumstances I would write some conditionals at the start which check if an argument has been passed, if it hasn’t then set var element = this, etc…

But there is another way.

For the second case instead of

alters_item(element)

we can write

$(element).each(alters_item)

because jQuery.each works even on jQueries that return only one object.

Doing this is a bit of a trade off – the second line of code I bet takes measurably longer to execute, but it does mean my functions get to be simpler, so it’s my weapon of choice at the moment.

But it does make me think that jQuery should have a call() method, that runs a function on an object, but also setting the object as the context.

Incidentally, if anyone knows of a better way of dealng with this isue than the one I’ve found, please let me know.