validates_presence_of :purpose : Tag javascript, everything about javascript

JavaScript syntax highlighting in the LaTeX listings package

Thu, 20 May 2010 11:20:00 +0000

I’m currently writing my thesis, using LaTeX. As the programming part of my thesis was mostly done in Javascript, I have to include some bits of Javascript source code in the LaTeX document. For this I’m using the listings package. With this package you can “include the source code of any programming language within your document”. Unfortunately Javascript is not in the list of supported programming languages. (In return, the list contains languages like Oberon-2, Promela or RSL, which are probably being used way more often than Javascript … er.)

Luckily @awendt pointed me to these links: Javascript Quelltexte in LaTeX and a thread on a mailinglist. I changed those examples only a bit, I’m posting my result here mainly in the hope it will produce a Google hit for “javascript latex source code” :-)

Put this at the beginning of your document. It will define support for the Javascript language. I included the keywords that show up in my code most frequently.

\usepackage{listings}
\usepackage{color}
\definecolor{lightgray}{rgb}{.9,.9,.9}
\definecolor{darkgray}{rgb}{.4,.4,.4}
\definecolor{purple}{rgb}{0.65, 0.12, 0.82}

\lstdefinelanguage{JavaScript}{
  keywords={typeof, new, true, false, catch, function, return, null, catch, switch, var, if, in, while, do, else, case, break},
  keywordstyle=\color{blue}\bfseries,
  ndkeywords={class, export, boolean, throw, implements, import, this},
  ndkeywordstyle=\color{darkgray}\bfseries,
  identifierstyle=\color{black},
  sensitive=false,
  comment=[l]{//},
  morecomment=[s]{/*}{*/},
  commentstyle=\color{purple}\ttfamily,
  stringstyle=\color{red}\ttfamily,
  morestring=[b]',
  morestring=[b]"
}

Here you define the style for the newly defined language. This also belongs in the preamble of your LaTeX file.

\lstset{
   language=JavaScript,
   backgroundcolor=\color{lightgray},
   extendedchars=true,
   basicstyle=\footnotesize\ttfamily,
   showstringspaces=false,
   showspaces=false,
   numbers=left,
   numberstyle=\footnotesize,
   numbersep=9pt,
   tabsize=2,
   breaklines=true,
   showtabs=false,
   captionpos=b
}

The colours are just an example, read how to define more colours in the wikibook.

Now you can insert code like this. The example is nonsense, I tried to include as many keywords as possible. The medskip command before the listing inserts a linebreak, a not very nice way to have some distance between the text and the listing.

\medskip
\begin{lstlisting}[caption=My Javascript Example]
Name.prototype = {
  methodName: function(params){
    var doubleQuoteString = "some text";
    var singleQuoteString = 'some more text';
    // this is a comment
    if(this.confirmed != null && typeof(this.confirmed) == Boolean && this.confirmed == true){
      document.createElement('h3');
      $('#system').append("This looks great");
      return false;
    } else {
      throw new Error;
    }
  }
}
\end{lstlisting}

The result looks like this:

Update: @perkee pointed out you can also use the minted package for syntax highlighting.

Security in CouchDB: changing the authentication db

Thu, 29 Apr 2010 10:40:00 +0000

I recently wrote a testsuite for CouchDB’s Javascript HTTP API. Among the things I tested were the security methods. That’s those that deal with authentication - like signup, login, logout and so on. The challenge was to make sure my tests wouldn’t interfere with the CouchDB setup of the person running the tests.

Background

CouchDB stores the information about the database users in a special database, the authentication_db. The default authentication_db is called “_users”. So this is where your user data is saved when you sign up without taking care about the details, e.g. when you use the signup method provided by the JQuery HTTP API. Read more about the ways to interact with the authentication_db in the HTTP API documentation (JQuery methods), and read more about general CouchDB security in the CouchDB Wiki or in The Definitive Guide.

How to set the authentication_db

When you e.g. create database users from within your tests, how can you make sure they don’t mess up the actual authentication_db, the one the unsuspecting user might use in production? Of course you have to create a custom authentication_db for storing the user information, and make CouchDB use that one. And after your tests are finished, you want CouchDB to use the original authentication_db again, so that only your testsuite uses the custom users database.

So there is one function you call right at the beginning of the part of the testsuite that’s concerned with security features:

function useTestUserDb(){
  test_users_db = new CouchDB("test_users_db");
  var xhr = CouchDB.request("PUT", "/_config/couch_httpd_auth/authentication_db", {
    body: JSON.stringify("test_users_db")
  });
  if(typeof(old_auth_db) == 'undefined'){
    old_auth_db = xhr.responseText.replace(/\n/,'').replace(/"/g,'');
  }
}

We create a custom authentication_db (“test_users_db”) and write its name into the CouchDB configuration. If we don’t have the name of the old authentication_db stored yet (the one that’s called “_users” per default, but might be called differently), we strip this name from special characters and store it in a variable, so we can restore it later. This is the function we call at the very end of the testsuite:

function useOldUserDb(){
  CouchDB.request("PUT", "/_config/couch_httpd_auth/authentication_db", {
    body: JSON.stringify(old_auth_db)
  });
  test_users_db.deleteDb();
}

In the last line, we delete our custom authentication_db. Everything is back to normal, there is no trace of our tests left. You can have a look at how it looks like in production.

How to work with the authentication_db

You can create a new user like this:

var userDoc = CouchDB.prepareUserDoc({name: "Gaius Baltar", roles: ["president"]}, "secretpassword");
test_users_db.save(userDoc);

Then you are logged in and can do whatever you want. About working with users and roles, read the chapter in The Definitive Guide. When you’re done, delete the userDoc from the authentication_db, as the authentication_db has to be empty before it can be deleted:

test_users_db.deleteDoc({_id : userDoc.id, _rev : userDoc.rev})

JSpec: Slides of my talk at Berlin JS Usergroup

Mon, 22 Mar 2010 12:52:00 +0000

Last thursday I gave a talk at the monthly meeting of the Berlin JS Usergroup. I was talking about JSpec, a Javascript Unit Testing framework. Thanks to all who were there to listen and ask questions!

Here are the slides. When you have a look at them, make sure to also read the notes (next to the comments), so the slides make sense to you.

To find more information about JSpec, check out the README on github, or read the blog post I wrote about it recently.

Thanks also to @gisikw and @tjholowaychuk for this talk from which I borrowed a few grammar examples :)

Refactoring asynchronous code

Mon, 15 Feb 2010 16:17:00 +0000

How to refactor a long chunk of asynchronous code is one thing I learned during my Javascript & CouchDB project. It’s not a difficult thing, but I thought it was before I figured out how to do it, so I guess it might be interesting for Javascript newbies.

Asynchronous code

In the first couple of weeks I struggled with this new style of coding before I got the hang of it. If you’re fairly new to Javascript, you’re probably used to just assign a variable in one line and use it in the next line. In the asynchronous world you query for a value and do stuff with it in the callback function of the query.

To illustrate - “normal” code looks like this:

var result = db.query("select * from T");
// use result

Although writing web applications just screams for doing it like this instead:


db.query("select..", function (result) {
// use result
});

(Examples stolen from Ryan Dahl’s Presentation about Node.js.)

The Problem

I have a method that checks if I have to display a notification message. For this it makes a lot of requests to the database, the received data has to be compared with other data, there’s lots of if/else clauses to consider all kinds of conditions and edge cases. It started simple, but then more and more conditions came in, unless I had the much feared diagonal closing bracket line on my screen.

You don’t need to know the details, just look at the picture and you’ll get the impression.

Refactoring

By way of illustration I made up a much shorter example:


checkForConflicts: function(){
  //set arguments
  openDoc(id, function(result){
    // do stuff with arguments
    if(result.bar == whatever){
      // do more stuff with result
      openAnotherDoc(id2, function(result2){
        if(result2.baz == true){
          // show notification message with result2
        }
      });
    }
  });
}

It is time to refactor after you are sure this actually is the way to go and the code basically works (“make it work, then make it pretty”). First you group the code into methods: You split the lines by what they do, and then you name each section. I think around 8 LOC is a good length for a method.

If this was synchronous code, you could just call these methods after each other. But this doesn’t work here, because result and result2 are not available outside the function that retrieves them.

My first approach was to call the functions from within each other. This works - but to see what checkForConflicts does at its very heart, you have to scroll down through all the methods - not much gained:

checkForConflicts: function(){
  //set arguments
  doSomething(arguments);
}

doSomething(arguments){
  openDoc(id, function(result){
    // do stuff with arguments
    if(result.bar = whatever){
      doSomethingElse(result);
    }
  });
}

doSomethingElse(result){
  // do more stuff with result
  openAnotherDoc(id2, function(result2){
    if(result2.baz == true){
      showMessage(result2);
    }
  });
}

showMessage = function(result2){
  // show notification message with result2
}

The way to do it is to use callbacks. Each method gets an anonymous function as (additional) argument: in the method declaration this function has the name callback. This callback is called if all the conditions within the method are met. The callback gets its arguments from within the method that calls it. You have to specify these arguments again in checkForConflicts to have a valid function definition.

checkForConflicts: function(){
  //set arguments
  doSomething(arguments, function(result){
    doSomethingElse(result, function(result2){
      showMessage(result2); 
    });
  });
}

doSomething(arguments, callback){
  openDoc(id, function(result){
    // do stuff with arguments
    if(result.bar = whatever){
      callback(result);
    }
  });
}

doSomethingElse(result, callback){
  // do more stuff with result
  openAnotherDoc(id2, function(result2){
    if(result2.baz == true){
      callback(result2);
    }
  });
}

showMessage = function(result2){
  // show notification message with result2
}

When you look at checkForConflicts, you now immediately see what it does, without being bothered by the details. As a side effect, deciding which code needs access to which arguments helps you to understand your code better. In my case I was able to optimize the use of passed variables a lot.

You might argue that the code is much longer now. First, longer is not worse if it is more readable. Second, this is only an example - if there was code instead of //do stuff the few extra lines wouldn’t carry weight.

Finally here is a screenshot of my refactored code. All the methods don’t fit on one screen, but you can see what checkForConflicts does very quickly.

JSpec - JavaScript Unit testing how it should be

Mon, 04 Jan 2010 12:07:00 +0000

Selecting a unit test framework for a JavaScript-only application - not a task with an obvious answer! Coming from the Rails world, where most people work with either TestUnit or RSpec, I was a bit lost when I started researching. This table on wikipedia was quite helpful for a first overview. But: JsUnit, JSTest, jsUnitTest, jsUnity - these are actually different frameworks, and there’s even many more …

I narrowed the selection down to those being currently developed, or are in use in “big” projects. JsUnit seems to be one of the major players, but I found almost no documentation, and the code base hasn’t changed a lot in the last years. That’s not a negative thing in itself, but having an active community around is a good thing. QUnit is used for testing JQuery, it looked like you could work with it as well, but its syntax and setup didn’t make me go “wow” exactly. Same goes for JQUnit, about which I had read good stuff in several blog posts. It’s also what Sammy.js is tested with.

I like RSpec, so when I had my first look at ScrewUnit, I was delighted to find there’s BDD syntax for JavaScript too. ScrewUnit is also a part of Blue Ridge, which is sadly still only available as a Rails plugin. So, ScrewUnit looked nice.

And THEN … I found JSpec. I was thrilled! It’s a JavaScript framework, but it doesn’t look like JavaScript at all: no curly brackets, no semicolons, just plain domain language. It just looks like RSpec. Here is a random snippet from my test suite, just have a look:

describe 'NoteCollection'
  before_each
    note3 = new Note({id: "8c8", text: 'three'});
    note1 = new Note({id: "ae9", text: 'one', next_id: "107"});
    note2 = new Note({id: "107", text: 'two', next_id: "8c8"});
      
    notes = new NoteCollection([note3, note1, note2]);
  end
  
  describe 'firstNote'
    it 'should return a note'
      notes.firstNote().should.be_an_instance_of Note
    end
    
    it 'should return the first note'
      notes.firstNote().id.should.eql note1.id
    end

    it 'should throw an error if there is more than one note that could be the first note'
      note4 = new Note({id: "fff", rev: "2-420"});
      notes.notes.push(note4);
      -{notes.firstNote()}.should.throw_error 'More than one first note found'
    end  
  end
end

And that are just 3 out of over 45 core matchers. There are also matchers for JQuery, see the next example, where I put the DOM I wanted to test in a fixture called outline.html. In Rails I don’t use fixtures, but if you want to test an application where all the markup is generated out of many different sources, it makes sense to test little pieces of the DOM in isolation. In ScrewUnit you have only one file for your all your mocked DOM.

describe 'NoteElement'
  before_each
    outline = elements(fixture('outline'))
    notes = outline.find('li')
  
    parent_note = new NoteElement($(notes.get(0)))
    child_note = new NoteElement($(notes.get(1)))
  end
  
  describe 'focusNextTextarea'
    it 'should focus the first child note if there is one'
      parent_note.focusNextTextarea();
      parent_note.noteLi().should.not.have_attr("data-focus")
      child_note.noteLi().attr("data-focus").should.eql 'true'
    end
  end

  describe 'insertUpdateNotePointers'    
    it 'should call setNextPointerToNewlyInsertedNote'
      inserted_note = {id:'4', text: 'inserted!'};
      parent_note.should.receive('setNextPointerToNewlyInsertedNote').with_args(inserted_note)
      parent_note.insertUpdateNotePointers(inserted_note);
    end
  end
end

Note, if you don’t like the RSpec-like Syntax, you can also skip the JSpec DSL and just use plain Javascript syntax.

More JSpec goodness: You can either run the tests in the browser, or in the terminal, so you can include it in your CI. In the terminal you can specify the browsers it should be tested with: The browsers will be opened in the background, but you won’t see your stuff being run like in Selenium. You can mock timers to test asynchronous calls. You can stub methods, mock AJAX requests, specify shared behaviours, hook into JSpec to specify your own matchers or do whatever else, and … just see for yourself.

Installation is easy too: just add JSpec.js and JSpec.css to your application. If you need them, add the files for jquery, timers or xhr support too. In spec/index.html you include your application files and specify which tests you want to run. Then open it in the browser, wait a few milliseconds, and see the result. If you want to run JSpec tests in the terminal, just install the JSpec Ruby gem. For this you also have to install Rhino (a Java-based JavaScript interpreter) - when you are on OS X, this should get you started.

What I don’t like about JSpec … the BlueRidge browser output is much more clear and pretty. I thought really hard about more things but that’s it.

Make sure to read the documentation on the JSpec website as well as the github Readme. Both ressources look similar, but at the time of writing there were different things on the two pages.

Bulk deletion of documents in CouchDB

Tue, 22 Dec 2009 17:37:00 +0000

For the couchapp I'm writing, I recently wanted to do a bulk delete operation with CouchDB, but found very little about it on the web. I don't know if my approach is the best way to go, but well, it works - if you know a better way, I'm happy to learn about it.

In the file jquery.couch.js that comes with CouchDB there are a couple of convenience methods to help you dealing with the database, documents and views. There is a method for deletion too (removeDoc), but it only can handle one document at a time. In the style of bulkSave, I added a bulkRemove method:

bulkRemove: function(docs, options){
  docs.docs = $.each(docs.docs, function(i, doc){doc._deleted = true});
  $.extend(options, {successStatus: 201});
  ajax({
      type: "POST",
      contentType: "application/json",
      dataType: "json", 
      data: toJSON(docs),
      url: this.uri + "_bulk_docs" + encodeOptions(options)
    },
    options,
    "The documents could not be deleted"
  );
},

In here I add the attribute "_deleted = true" to every document I want to delete, and then I do a bulk update with the documents. That's it.

This is one possible use case: In the delete action of my controller I first fetch an array with the post and its comments from the couch (how to do this see below), and then I call the bulkRemove action on that array.

couchapp.design.view('post_with_comments', {
  startkey: [params['id']],
  endkey: [params['id'], {}],
  success: function(json) {
    if (json.rows.length > 0) { 
      var post_and_comments = json.rows.map(function(row) {return row.value}); 
      couchapp.db.bulkRemove({docs: post_and_comments}, {
        success: function() {
          flash = {message: 'Post deleted.', type: 'notice'}
          redirect('#/outlines', flash);
        },
        error: function(response_code, msg) {
          flash = {message: 'Error deleting post: ' + msg, type: 'error'};
        }
      });
    }      
  }
});

Note that the "flash" object I'm using is my own Sammy.js implementation of a Rails-like flash to show a message after a redirect. Maybe I'll share that in another blog post.

As a bonus, this is the view I'm using to retrieve a post and its comments in one request. I'm following the pattern Christopher Lenz is recommending in his article on CouchDB Joins.

function(doc) {
  if (doc.type == "Post") {
    emit([doc._id, 0], doc);
  } else if (doc.type == "Comment") {
    emit([doc.post_id, 1, Date.parse(doc.created_at)], doc);
  }
}

When you call this view with

startkey: [params['id']],
endkey: [params['id'], {}]

you get an array with the post as the first element, and then all the comments that have the post's ID as post_id.

This can of course also be used for things other than deleting: to show all the comments for a post, I get the array's first element and extract the title and content. Then I remove it from the array with

json.rows.splice(0,1);

You now have an array with only the post's comments and you can display them somehow.

Testing key events with Culerity

Sun, 22 Nov 2009 20:43:00 +0000

The application I'm writing depends heavily on key events. When you press enter in a specific text field, some other field opens, when you press tab, the text fields moves, etc. Of course I need to cover this in my cucumber integration tests. The Celerity API is a bit thin for this purpose: You can fire a, let's say, keydown event - but it doesn't pass through the specific key code.

Actually it's easy when you know how. You can execute any javascript directly on the Culerity RemoteBrowserProxy object. In order to fire a specific key event, you have to create a KeyboardEvent, initialize it with whatever you want, and dispatch it on the element.

Read up the details, especially the meaning of initKeyEvent's parameters in this Javascript API.

I ended up with this cucumber step:

When /^I hit "([^\"]*)" in a text_field with id "([^\"]*)"$/ do |key, id|
  key_code = case key
    when 'enter'
      13
    when 'up'
      38
    when 'down'
      40
    else
      0
    end
  $browser.execute_script('event = document.createEvent("KeyboardEvent");')
  $browser.execute_script('event.initKeyEvent("keydown", true, false, document.window, false, false, false, false, ' + key_code.to_s + ', 0)')
  $browser.execute_script("document.getElementById('#{id}').dispatchEvent(event)")
  When 'I wait for the AJAX call to finish'
end

Note that this is meant to work for Firefox only. For IE you can do a similar thing with createEventObject, see here how.

Couchapp / Culerity glitches, part 1

Wed, 04 Nov 2009 10:07:00 +0000

Do you know what's behind these buzzwords: CouchDB / Couchapp / Sammy.js? Not really? Then please look at this article now: Sammy.js, CouchDB, and the new web architecture. The title says it all.

This setup happens to be the field of my diploma thesis, friendly sponsored and supported by Upstream. During the next months I'll develop a super duper thing with these technologies, and then I'll write 80 scientific pages about it. Yees, I'm not that much looking forward to the second part! Until then, I'm planning to semi-regularly blog about my findings.

I'm not allowed to share the code before I have the diploma in my hands. But, because my workmate Frank Proessdorf is so jealous of what I do, he and I started working on an app with a similar setup on our Upstream Research Fridays, to try things out. Here it is. Don't expect anything yet.

Today I'm going to tell you about some details of my testing setup. My *cough* supervising tutor Alexander Lang already wrote down all there is to know yet about Test-Driven Development with Cucumber-Culerity-Celerity and Couchapps.

But there are a few things that will make your life hard, when the almost perfect Rails testing world has spoiled you as much as me. One thing is that we have to deal with a lot of asynchrony. Sammy renders the page, loads the data from the couch, and whenever the data is there it gets rendered into the page. The best way to do this is via callbacks. I'm going to write another blog post about that soon.

In the integration test the problem is that celerity doesn't behave as it claims it does:

$browser.wait

is supposed to make the test wait until everything is rendered completely. But for whatever reason it doesn't. The result is that your test fails because it searches for your expressions in an unfinished page.

Because of that, in culerity we need something like:

sleep 0.4

after every follow, press and go-to-path step. When using culerity in a Rails environment, this usually is enough. Sammy behaves a bit more unpredictable - it sometimes takes more than a whole second until everything is loaded.

Solution? We can either increase the sleep time a lot, but that's not very nice, the features need long enough to run as it is. Fortunately we can use another method that's build into celerity: wait_while.

I created a div "spinner" somewhere in index.html:

 <div id="spinner"></div>

You can also add an actual spinner gif here. Just take care to place the div outside of your sammy element selector ("#main" by default), because this div gets replaced all the time.

We need to show that div before every route that gets run. In application.js:

before(function() {
  $('#spinner').show();
});

And we need to hide it after your rendering is completed. I haven't found an ideal solution where to do that, it depends on the application you are writing. As a quick and dirty fix, you can add it as a last line in sammy's partial function, in sammy.js:

partial: function(path, data, callback) {
  //...
  $('#spinner').hide();
},

I prefer to have more control about when I hide it, as my partial calls sometimes have nested callbacks. So I put it in application.js, resp. in my controllers, after each render call or in the partial callbacks. I admit this still sucks a bit.

Finally, amend the cucumber step:

When /I wait for the AJAX call to finish/ do
  sleep 0.2
  puts 'Waiting for page to load ...' if $browser.div(:id, 'spinner').visible?
  $browser.wait_while { $browser.div(:id, 'spinner').visible?} 
end

The browser still needs to sleep a bit before it actually finds the spinner div. But with this step in my common_culerity_steps.rb, the features finally run smoothly.