A Full Javascript Architecture, Part One – NodeJS

2

This post is the first of a three-part series showing how to build a complete javascript architecture using :

  • NodeJS : For the server side. We will use Socket.IO to manage long-terms real-time connection.
  • Google Chrome Extension : For the client side. WebSocket, Notification and Local Storage will be used.
  • MongoDB : To store the datas.

To illustrate our architecture we are going to create a Node application that will track tweets about the “What’s Next” event and broadcast them in real time to the clients. The client will be a Google Chrome browser extension and will use two features of the HTML5 specification to display the tweets broadcasted by the server. In the last post I will introduce the MongoDB database, we will use it to store tweets and provide some statistics when the event ends.

 

Architecture Javascript

This first part is dedicated to the creation of a Node application. We will see step by step how to install Node on a Linux environment and setup a HTTP server with WebSocket support.

Introduction to Node

Node's Logo

What is Node ?

Node's Architecture

Node is an open source toolkit for developing server side applications based on the V8 JavaScript engine. Like Node, V8 is written in C++ and is mostly known for being used in Google Chrome.

Node is part of the Server Side JavaScript environnement and extend JavaScript API to offer usual server side functionalities. Node base API can be extended by using the CommonJS module system.

Node has be created by Ryan Dahl in February 2009 and has since become very popular. It’s spirit is similar to Twisted for Python and EventMachine for Ruby.

The project is run by Joyent (the company employing Ryan Dalh) who lunched a cloud hosting service for node applications (no.de).

Node’s Goal ?

It’s goal is to offer an easy and safe way to build high performance and scalable network applications in JavaScript.

Those goals are achieved thanks it’s architecture:

  • Single Threaded :

Node use a single thread to run instead of other server like Apache HTTP who spawn a thread per request, this approach result in avoiding CPU context switching and massive execution stacks in memory. This is also the method used by nginx and other servers developed to counter the C10K problem.

  • Event Loop :

Written in C++ using the Marc Lehman’s libev library, the event loop use epoll or kqueue for scalable event notification mechanism.

  • Non blocking I/O :

Node avoid CPU time loss usually made by waiting for an input or an output response (database, file system, web service, …) thanks to the full-featured asynchronous I/O provided by Marc Lehmann’s libeio library.

These characteristics allow Node to handle a large amount of traffic by handling as quickly as possible a request to free the thread for the next one.

Node has a built-in support for most important protocols like TCP, DNS, and HTTP (the one that we will focus on). The design goal of a Node application is that any function performing an I/O must use a callback. That’s why there is no blocking methods provided in Node’s API.

The HTTP implementation offered by Node is very complete and natively support chunked request and response (very useful since we are going to use the twitter streaming api) and hanging request for comet applications. The Node’s footprint for each http stream is only 36 bytes (source).

JavaScript

Being an Event Driven Language, Javascript is the most suited to develop on the Node’s “Event Loop” architecture. Node’s applications really use javascript’s strengths like anonymous functions and closures.

Community

Despite its young age, the community is really growing fast and the GitHub repository is the 3rd most followed one.

Installing Node

The first step for us to take is the installation of Node. I choose to run Node on a Linux Debian 6 system but Windows users can try this.

Prerequisites

We need some packages before installing Node itself, same prerequisites apply for Ubuntu.

Node

Now we just have to clone the git repository and install (make might take a while) :

When the installation is complete, try it with :

That should print out your node version.

Installing Node Package Manager :

Isaac Schlueter (another employee of Joyent) has created a really great Node package manager called NPM. We need it because our Node application will use the Socket.IO module.

Once installed you simply call this command line to install the Socket.IO package.

About Socket.IO

Socket.IO is a powerful Node module that bring the ability to simply manage long term connections with the clients. The module is used on the server side and on the client one. The client side will automatically and seamlessly use the best communication type (based on browser capabilities) to connect to a Node server.

Our architecture use a Google Chrome client, since this browser has WebSocket protocol capability, that’s the connection type Socket.IO will choose.

But what happen if we decided to create a Mozilla Firefox extension where WebSocket support is disable ? Well, by using Socket.IO, long polling connection will automatically be used and there is no need to change anything in our code on both client and server sides.

Ready to code

All the source of this post can be found on it’s dedicated github repository.

Hello World

To introduce the concept behind Node’s application development, we will start by the classic Hello World example.

First let’s create the folder tha
t will be our workspace:

Then copy the following content in a file named server.js

 

This is quiet simple, we just create a http server by requiring the HTTP module and calling the createServer method. The callback method will be executed each time a request comes in.

To start the server call :

We can now access to the server by using http://hostname:8124 and see the « Hello World ».

Adding Socket.IO

The next step is to use Socket.IO to handle long-terms connections, replace the server.js content by the following one:

Two more lines, that’s the only things we need to add to attach the Socket.IO module to the http server.

You might have noticed that the callback function of the http.createServer() has been removed, this is due to our use case where the server does not serve data to the client when they make a request but directly push data when an event is raised.

We can put a callback to the io.listen() method, it will be fired each time a request comes from the Socket.IO client side library.

Twitter tracker module

To retrieve the tweets about the “What’s Next” event we need to use the Twitter Streaming API. The API will return us in real time all the tweets mentioning “WsN_Paris” and « zenika ».

In order to do that, we are going to create a simple module that will connect to the Twitter API and emit an event each time it found a new tweet related to our topic.

Connection to the API require a valid twitter account. The module will encode your credentials and add them to the request header. We do not need to use OAuth to use the Twitter streaming API.

Create a new file called twitter.js and add the following content :

 

The only tricky part is the logic implemented in the « data » event callback. This is Twitter specific and other API are much more easy to deal with.

Using The Tracker

Now that we have a functional tweet tracker module, we include it in the server.js and start waiting for « tweet » events to broadcast them to the clients.

As you can see, we just need to add a listener on the « tweet » event and send the useful part of the tweet to the clients by calling the socket’s broadcast() method.

Conclusion

This close up the first part in the development of a full JavaScript architecture. I hope that reading this help you see how easy it is to build a simple but yet highly scalable application.

Node is very powerful and writting an app in JavaScript that is not only meant to wait for onclick events is really a nice experience.

You can quickly start developing with Node by using the Cloud9IDE. Just register or sign in with your github account and you will be ready to code within a minute.

In the next part we will see how to develop a Google Chrome extension that will display the tweets in real time by playing with HTML5 notifications.

Ressources

Partagez cet article.

A propos de l'auteur

2 commentaires

  1. Thanks for this great article.

    About Node’s Goal / Single Threaded :

    Node JS is natively single threaded so he doesn’t scale with multi core processor. It may change in future version. User should create a cluster of Node to scale in. It’s a flaw rather an advantage.

    About IO / NIO have a look at :
    http://mailinator.blogspot.com/2008

  2. fantastic article. Thanks a lot for sharing. one of my faviroate things about Node.js is that Javascript is the core programming language for the browser, and writing code in node.js allows us to share code between browser and server (validation code, is a perfect example) or choose whether to execute algorithms on the browser side or the client side – and if we change our mind, we can switch back easily as well.

Ajouter un commentaire