memcache.js for Google App Engine application front-ends?

The Google App Engine datastore is a pretty neat example of a key-value datastore.  Mapping this to JavaScript objects can be fairly straightforward, especially when you need to cache AJAX responses.

An App Engine Model

Let’s say you have a Person model defined in your GAE application:

# models.py
from google.appengine.ext import db
from google.appengine.api import memcache

class Person(db.Model):
    first_name = db.StringProperty()
    last_name = db.StringProperty()
    birthdate = db.DateTimeProperty()

    @classmethod
    def get_all(cls):
        cache_key = 'Person.get_all'
        cached_value = memcache.get(cache_key)
        if cached_value:
            return cached_value
        else:
            people = Person.all().fetch(20)
            memcache.set(cache_key, people, 120)  # cache for 2 minutes/120 seconds
            return people

Getting a list of 20 people (more specifically, Person objects) from the datastore is as easy as calling:

people = Person.get_all()

The Person.get_all() method uses memcache to temporarily cache a copy of the data in distributed memory to avoid hitting the datastore every time it is called.  Of course, as you can see, the data is only cached for a particular duration in memory and then cleared away.

What if you could use memcache on the client side to cache server responses?

You can’t really use the actual memcached daemon for this purpose, but you can surely emulate memcache behavior using a simple JavaScript object to cache values just like memcached would.  Quite naturally, since the code would be restricted to a single script runtime environment, you wouldn’t have distributed memcache either.  But, hey, something is better than nothing.

Caching server responses that result from AJAX calls can make people perceive that your application is pretty quick.  All you’re doing to achieve this is avoiding hitting the Web server repeatedly for the same information.  Let’s look at a code excerpt:

function get_person(key){
  var person = memcache.get(key);
  if (person){
    return person;
  } else {
    remote_api.get_person(key, function(person){
      memcache.set(key, person, 120000);  // Cache for 2 minutes (120 seconds).
    });
  }
}

Note that get_person(key) will not send a request to the server if the data is already available in the memcache store.  In the above case, the data is only cached for 2 minutes, after which any call to get_person() will send a request to the server.

Where’s the code?

/**
 * memcache.js - A simple memcache-like object implemented in JavaScript.
 * Copyright (c) 2009, happychickoo.
 * All rights reserved.
 *
 * Redistribution and use in source and binary forms, with or
 * without modification, are permitted provided that the following
 * conditions are met:
 *
 *   * Redistributions of source code must retain the above copyright
 *     notice, this list of conditions and the following disclaimer.
 *   * Redistributions in binary form must reproduce the above copyright
 *     notice, this list of conditions and the following disclaimer
 *     in the documentation and/or other materials provided with the
 *     distribution.
 *   * Neither the name of happychickoo nor the names of its
 *     contributors may be used to endorse or promote products derived
 *     from this software without specific prior written permission.
 *
 * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
 * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
 * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
 * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE
 * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
 * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
 * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
 * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
 * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
 * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
 * POSSIBILITY OF SUCH DAMAGE.
 */
this.memcache = {
  datastore: {},
  get: function(key){
    return this.datastore[key];
  },
  set: function(key, value, timeout /* milliseconds */){
    var store = this.datastore;
    if (typeof timeout === 'undefined'){
      timeout = 0;
    }
    store[key] = value;
    if (timeout){
      setTimeout(function(){
        delete store[key];
      }, timeout);
    }
  },
  remove: function(key){
    delete this.datastore[key];
  },
  clear: function(){
    this.datastore = {};
  }
};

There you go.

How does this help with GAE applications?

App Engine doesn’t use a conventional RDBMS to store data.  GAE uses a distributed key-value data store, where every object stored in the database is assigned a unique key that looks something like this:

agttaWxzLXNlY3VyZXIKCxIEVXNlchgdDA

This key is guaranteed to start with an alphabet, which implies you can use it as an identifier for a DOM element. Attaching behaviors to such DOM elements that fetch corresponding data then becomes as easy as doing this (example uses jQuery):

jQuery('ul#people > li').click(function(e){
  var key = jQuery(this).attr('id');
  get_person(key, function(person){
    show_information(person);
  });
});

Hope that helps clear out some air.  It should be noted that if you’re adding li elements dynamically to the ul#people unordered list, the above event handler will not be called for them.  Instead of using jQuery(…).click(handler), you should consider using jQuery(…).live(‘click’, handler).