Node, exit and asynchronous cleanup

Node can exit for three main reasons:

• implicitly: when there is no more code to be executed AND the event loop runs dry (i.e. there is no outstanding timer and I/O listener).
• explicitly: when the code execute a process.exit().
• accidentally: an uncaught exception bubbles up to the event loop, and there is no uncaught exception listener on the process.

If you are explicitly closing the app, you can trigger whatever cleanup code you need, before calling process.exit().

However there are a lot of case where you can't predict at all when your program will exit: e.g. if there is simply no more job scheduled in the event loop, your program has done the job and will exit.

If some cleanup code has to be performed, we can listen for the exit event on the process object. However, the program will exit as soon as all listeners have returned: only synchronous tasks can be perfomed here!

But the very nature of Node.js is asynchronous: lot of stuff we would want to do cannot be done synchronously.

For example: we want to log the context and status when exiting, but one of our transport is async (e.g. it logs to a database or over the network). If we send data over the wire from that kind of listener, there are good chance it will never reach its destination.

Introducing async-kit

I will sound like I'm doing (again) my self-promotion here, but there is a great package (sic!) that do the job: async-kit. This is the first package I ever wrote. This is a toolbox that deals with whatever async stuff you have to do, and it has been tested for nearly two years now... and it still receives few improvements once in a while!

The last feature addition it received is the async.exit() method. This is a replacement for process.exit(), and its purpose is to exit asynchronously.

async.exit( code , timeout )

• code number the exit code
• timeout number the maximum time allowed for each underlying listener before aborting, default to 1000 (ms).

When you call async.exit(), it emits the asyncExit event on the process object.

There are two kinds of listeners:

• function( [code] , [timeout] ) listeners, that does not have a callback, are interested in the event but they don't need to perform critical tasks or can handle them synchronously. E.g.: a server that will not accept connection or data anymore after receiving this event.

• function( code , timeout , completionCallback ) listeners, that DO have a completion callback, have some critical asynchronous tasks to perform before exiting. E.g.: a server that needs to gracefully exit will not accept connection or data anymore, but it still has to wait for client request in progress to be done.

Note that the code and timeout arguments passed to listeners are actual values used by async.exit().

So async.exit() will simply wait for all listeners having a completionCallback to trigger it (or being timed out) before exiting, using process.exit() internally.

process.on( 'asyncExit' , function( code , timeout , callback ) {

    console.log( 'asyncExit event received - starting a short task' ) ;

    setTimeout( function() {
        console.log( 'Short task finished' ) ;
        callback() ;
    } , 100 ) ;
} ) ;

process.on( 'asyncExit' , function( code , timeout ) {

    console.log( 'asyncExit event received - non-critical task' ) ;

    setTimeout( function() {
        console.log( 'Non-critical task finished' ) ;
    } , 200 ) ;
} ) ;

async.exit( 5 , 500 ) ;  

After 100ms, it will produce:

asyncExit event received - starting a short task  
asyncExit event received - non-critical task  
Short task finished  

Note how the setTimeout()'s function is not executed in the second event handler: this handler does not accept a callback, hence the process will exit as soon as the first handler is done: after 100ms.

Nice, isn't it?

Now you will never use process.exit() anymore!

Happy coding!

The number one rule is consistency... cause nobody want to use a broken API like PHP... It's very clear and does not need further explanation.

Ok, so let's keep it consistent, but what order should we use anyway?

When designing a Javascript API, an important point one may consider is the binding possibility (i.e. using .bind() on the API methods).

Ask yourself what is the most constant parameter and what is the most variable parameter for a method.

Let's use a real-world example here.

I'm currently working on a new general purpose data validation lib (doormen). The main feature of the lib is a function that takes a schema and a data as arguments, and throws if the data does not validate according to the schema.

As of doormen v0.2.x, the syntax is now doormen( schema , data ):

// do not throw
doormen( { type: 'number' } , 1 ) ;

// throw: 'hello' is not a number
doormen( { type: 'number' } , 'hello' ) ;  

But before v0.2.x, the syntax was doormen( data , schema ). What a terrible design mistake I made back then!

As you can imagine, a single schema aims to validate multiple data, hence there will be a greater number of unique data than unique schema. The schema parameter is less variable than the data parameter.

Therefore, as of v0.2.x, one is able to create a specific validator easily using .bind(), e.g.:

var singleDigitNumberValidator = doormen.bind( null , { type: 'integer', min: 0, max: 9 } ) ;

singleDigitNumberValidator( 3 ) ; // OK  
singleDigitNumberValidator( 13 ) ; // Not OK  
singleDigitNumberValidator( 'bob' ) ; // Not OK  
...

Before v0.2.x, slightly more code was needed, and one more function call to achieve the same:

var singleDigitNumberValidator = function( data , schema ) {  
    return doormen( data , { type: 'integer', min: 0, max: 9 } ) ;
} ;

singleDigitNumberValidator( 3 ) ; // OK  
singleDigitNumberValidator( 13 ) ; // Not OK  
singleDigitNumberValidator( 'bob' ) ; // Not OK  
...

I think this rule can apply to language that do not have .bind() too, since it follows a “generic to specific” pattern.

Do not miss other articles on this subject:

• Part I: Prologue
• Part II: Performances
• Part III: Language comparison

Now it's time to compare the Javascript language to the PHP language. By the way, I will not talk about various browser's implementation: we will compare apple with apple here, so we will talk about Javascript inside of Node.js.

The PHP language

PHP is probably the most hated language. There are tons of PHP bashing articles in the world. But what exactly is PHP and why the hate?

It's hard to tell exactly what PHP is. PHP looks more or less like a random and extensive compilation of tools. This is probably what make it simultanously great and bad: everything exists in the core language: procedural, class-based OOP with lot of features, tons of functions and methods...

You want to manipulate arrays? There is probably a builtin function doing what you want already: you don't have to code it by yourself. So you can focus on your application business logic.

You like OOP? PHP features almost everything that a class-based programming language has: classes, abstract classes, final classes, inheritance, constructors, destructors, static methods, interfaces, scope resolution, late static binding... except a strong type hinting...

And that's what make people angry about PHP. There is always a "but" or "except" at the end of the phrase.

All design pattern can be implemented easily in PHP, since its OOP has been inspired by C++ and Java. You can even do more.

Because PHP behaves like a dynamic language too, you have magic methods that allow you to perform some cool but unpopular things. For example there are magic getters and magic setters that are called everytime you try to access a property that does not exist. Since PHP doesn't mix method's and member's namespace in an object, they cannot clash. There is also a magic method called everytime you try to access a method that does not exist.

There are some interesting use cases for this: e.g. one can build a simple xml lib where your object map exactly the XML tree... and there IS in fact a lib called SimpleXML that takes advantage of this. Back in my PHP's day I coded a lib inspired by SimpleXML but without its various gotcha. It was fun and entertaining, and I found the language really flexible. Combo that with the ArrayAccess interface, and the lib was allowing one to navigate easily from an XML node to another, in a concise manner, in a way that cannot be done in C++ or Java.

There is also the traits feature that is useful once in a while. It permits class aggregation. It's useful because PHP does not support multiple inheritance. It's not a bad thing, since multiple inheritance is broken by design... see the diamond problem.

So it's untrue to claim that PHP's OOP is badly designed.

PHP is great at copying and integrating every language features that its creators want. It even integrates goto, a feature that was introduced a very long time after the OOP's introduction: that one caused probably the greatest shitstorm in the entire PHP life! In fact, PHP can shamelessly absorb anything.

So that's turn out to be the main problem with PHP: its lack of identity. It was a procedural language, then becames an OOP language, so more than often, PHP is totally schizophrenic about that. Some core features are sometime functions, thus polluting massively the global namespace, sometime they are behind a class, and sometime they use BOTH! Yay, you read it: PHP core itself does not know who he/she/it is. Example? The date_* function collection and the DateTime class, you can use both DateTime::diff() and date_diff().

Okey, this is not critical, but that's what make PHP the Frankenstein of programming languages. From Wikipedia:

One criticism of PHP is that it was not originally designed, but instead it was developed organically.

That's the story of PHP. There is no consistency anywhere.
You have a function named htmlentities() to encode and html_entity_decode() to decode. Seriously, WTF??? Looks like people there cannot stick to a convention.

There is a core function named strcmp() WITHOUT underscore, and one named str_replace() WITH underscore. Speaking about that one, have a look to its prototype:

str_replace(search, replace, subject [, count ] )

... versus:

str_split(subject [, split_length ] )

Yes, that's another gotcha: the first one want the subject string to be its third argument, the second want it to be its first argument. Ugly. There are plenty of examples of inconsistencies in PHP, and after years of programming with it, it still happens that I forget the correct order for a particular function.

Also PHP does NOT support unicode natively, you have to rely on a very small subset of the string manipulation function prefixed by mb_*. When I started using PHP, most of those functions were missing, and it's still painful to do unicode aware code. This is probably the PHP greatest weakness. Almost all websites are using UTF-8 now.

There was a failed attempt to upgrade PHP's core with unicode, PHP 6, but it was miserably aborted.

Finally, PHP has errors AND exceptions. Silly.

Also I'm not going to list all of PHP's failures here, there are dozens of websites doing that. For example PHP: a fractal of bad design is a really well-documented article.

So there are two kinds of people: people that can cooperate with a weird but capable language, and perfectionist people that will hate it from day one.

A short story of modern Javascript

The Javascript language has its haters too.

However, most of the arguments against it exists because Javascript is the most misunderstood language. For decades, not a single book was accurate or written by someone who has a clue about the language very nature, except Douglas Crockford.

Worst: Internet Explorer totally distorted it for years, and we all know what the harm Microsoft can do when it has decided to totally break any living standard for the sake of its monopoly... Programmers exposed to Javascript back in the IE 5.5 and IE 6 days have been totally traumatized. And I was one of those. By the way, Microsoft's JScript is *NOT* Javascript, but sadly no one cares about that.

Then Javascript had a second live:

• JQuery helped to abstract away browser's incompatibilities.

• Google Chrome appears, rapidly conquers its share of market, and initiate a browser-war that Microsoft cannot win anymore, leading to the JS engine race we all know: suddenly Javascript becomes lightning fast! Also Internet Explorer was forced to follow the standards, or it would lost even more marketshare.

• Mozilla and Google has done a huge work to push the language forward: Google is usually the innovative guy and Mozilla the perfectionist that polish things and improve the spec before its standardization.

• Thanks to the V8 speed, Javascript becomes an option for server-side scripting: Node.js appears and the ecosystem is growing up really fast. Node.js enforces the RequireJS spec and the last major gotcha of Javascript is finally gone: global variable leaks. Now Javascript is modular, and is fully capable to handle big projects. Npm is probably the better package manager in the world and solves dependency hell in an elegant way.

The Javascript language

Javascript is a dynamic language, featuring a prototype-based OOP.

It has dynamic typing, so duck typing is really common here. Almost everything in Javascript is an object. Object literal are common too, and more than often they are used to pass named parameter to a function. Object are basically a hashmap, eventually a prototype chain (also useful for inheritance), and since functions are first-class citizens that make them methods. Finally, Javascript has closures.

Javascript is event-driven, so callbacks are used everywhere.

The beauty of Javascript is its simplicity and its expressiveness. With only few keywords, almost any constructs are possible.

Common criticisms

People often criticizes the Javascript object model. Some are claiming that it is a Procedural language, not an OOP language.

They are wrong. Usually, people are confusing OOP with Class-based OOP. Javascript is a Prototype-based OOP language. See the wikipedia page about programming paradigm and about prototype-based programming.

There is a very interesting comment written by @Plynx, explaining the benefits of the Javascript OOP, and the differences between class-based OOP. The comment is very long, detailed and well-written, this quote is just a small part of it:

I should probably say a few words about encapsulation, since it's often cited by object oriented programmers as being a very important aspect of programming.

The ability to hide data and functionality is present in Javascript in the form of closures, but they are rarely used that way, and the history of encapsulation should make it clear why. Earlier I mentioned namespace pollution as one of the key problems in early structured programming languages. Modern taboo against global variables flows in part from the problems programmers had when name collisions would occur after the merge of disparate codebases. Additionally the concept of the object as an API, and the need to subclass in order to extend functionality while preserving type compatibility, led to the creation of the access qualifiers in OOP. Initially it was thought that this method of disallowing unintended use of classes would be a boon to group development, but a number of studies have shown that it did not improve productivity.

The primary benefit of encapsulated classes was the fact that they created namespaces, now a headline feature in class-based languages. Namespaces produce the benefit of not having name collisions while also documenting the intended use of a class, without hamstringing future development by restricting the possibility of unforeseen useful code sharing by prematurely disallowing it. In Javascript, every object being a dynamic associative array makes every {} a namespace (even more name control is possible through module.exports in Node). Namespaces are generally a superior approach to encapsulation, preserving the benefits without dealing with the grueling problems of strict access control to portions of classes in class-based languages.

People previously exposed to class-based OOP will often found Javascript reluctant: no classes, no private members... Javascript is fully capable of that, but you have to do it by yourself: a constructor is just a mere function that can create object, and used in conjunction with prototypes achieve the same thing that classes. Private members are possible through closures. That doesn't means you need to code more to get that, and in fact you have to code less, that just means there is no class or private keyword.

However you can do more, if you want. A prototype is nothing more than a regular object: you can mutate it and all instances created by that prototype are mutated. No class-based language is able do to that at runtime.

Anyway, it's worth noting that there is no need for private members at all, in a dynamic language. Also there is no inherent reason to code a complex class hierarchy, that would lead to over-engineering, lasagna code or worst: spaghetti code. There is no need for such overhead in Javascript.

Good Javascript code embraces the KISS principle. KISS projects advance at a faster pace, they are robust, clean, healthy and easy to maintain.

Also, Javascript has some real gotcha:

• arguments is not an Array, and it's boring to type Array.prototype.slice.call( arguments ) everytime you want Array features...

• + operator can mix string and numbers: it performs concatenation or addition.

• you can't easily inherit from builtin objects (e.g. Array)

• typeof null === 'object' this one really sucks... it's the thing I hate the most about Javascript... The workaround is easy, still, I hate to type if ( myVar && typeof myVar === 'object' ) all days... And it won't be fixed :'(

• UCS-2 unicode encoding instead of UTF-16... In fact Javascript exposes characters as UCS-2 with surrogate pairs. E.g. a single chinese unicode character takes 2 characters in a Javascript string.

• automatic semicolon insertion: semicolon are not mandatory to end a statement, semicolon can be automatically inserted at the end of the line. A semicolon can be inserted by the parser where it's not intended by the coder, e.g. if the return keyword is followed by a new line.

Also people usually adds NaN to the list, because NaN stands for Not a Number and still typeof NaN === 'number', and because NaN !== NaN. However, there are wrong, actually NaN behaves just like it should. Really. Just read Javascript NaN Demystified if you don't believe me.

Let's compare them!

Where PHP wins:

• PHP is very classic: if you come from C/C++/Java & co, you will find similar concept.

• Easiest to grasp: partly in consequence of the previous point, you will be able to code in PHP in no time. You will be able to code in Javascript quickly too, but your code will be crappy even if you have some solid knowledge in other languages. You will have to realize that Javascript does not work the way you are used to, and you will have to change your habits.

• PHP has destructors, Javascript doesn't... sometime I'm missing them.

Where Javascript/Node.js wins

• Unicode support: UCS-2 is suboptimal, however it's still far better than no support at all.

• Isomorphic Javascript: you can share modules between your server-side Node.js codebase and your browser-side codebase. Browserify can help you in the process.

• Less boilerplate code.

• Faster: consequence of the previous point: it's possible to write application faster, and to achieve great things with a smaller codebase.

• Easiest to maintain: code above the average is easiest to maintain in Javascript than in PHP. Also please note that it is possible to code total crap in Javascript, so this does *NOT* apply to code below the average. PHP can be very crappy too, but its paradigm drive the programmer a bit more.

• Cleaner core: no global namespace pollution here, the language is rather consistent, and almost every core features are OO.

• Expressiveness and flexibility: Javascript is one of the most expressive language, and win this battle hands down.

• Javascript is event-driven, dealing with I/O is way faster and easier, internet is more and more about realtime and Node.js is a strong player in that area.

• Websocket: consequence of the previous point, Javascript is *THE* language of choice if you want to deal with websocket.

• Freedom: you build your own server or micro-service, so everything is possible, you are not tied to the PHP paradigm.

• Evolution: the Javascript language evolves quickly and far more openly. New features have been well weighted. Some quirks of the past will remain for the sake of backward compatibility, but at least new additions will be clean. On the contrary PHP is not really open, few peoples have total power on the future of the language and are not listening to criticism at all.

Conclusion

As you can see, Javascript scored more points than PHP. And that's not going to change anytime soon.

In fact PHP is evolving too. Some features were missing and they were added. But not in a sane way. The path of PHP is wrong.

So let's finish the talk with two angular stones of PHP 5 that allow it to be framework-compliant: autoloading and namespace.

When autoloading was included, I was like “Nice! This is really useful and solves a lot of trouble I had in the past”. And that's right: PHP is greater with autoloading than without. However, looking backward, it appears that autoloading was an infamous monstrosity. There was a problem, but it was fixed in the badest possible way. It is really hacky.

It was combined later with namespaces to avoid massive global scope pollution by third-parties. In a way, it opened the road to framework interoperabilities, but since it does not solve the whole thing, the frameworks had to do what PHP should had done by itself. It leads to a consortium of frameworks and those awkward PSR-x specs. But shitty PHP-frameworks will be explored in the next article of this series.

It's worth noting PHP's namespace syntax uses backslashes as separators. That's very unusual and I don't like it.

That does not fix the main issue with PHP: require() and include() (and therefore autoloading) in PHP works the same way the #include preprocessor directive works in C/C++: it's just a kind of runtime copy-paste of file to include into the parent file. Therefore, it is totally leaky by design!

Now that I'm a Node.js developper, it's even more evident that PHP was wrong. Node.js solves the dependency hell in the most elegant way: you require a lib (i.e. a module) into a variable. A Node.js module returns (i.e. exports) something into the caller's variable. Usually a module either exports a function or an object containing a collection of functions.

Okey, an example is worth thousand words, let's say we have a function called toto(). Later we found out a wonderful third party lib, but unfortunately its name is "toto" too. No problem, let's load the module into the "moduleToto" variable:

var toto = function() {  
    // My code
}

// Load toto into moduleToto
var moduleToto = require( 'toto' ) ;

// Use the third party lib
moduleToto.makeSomethingAwesome() ;  

Nothing leaks outside of a module.

No pollution.

Furthermore, in Node.js proper namespaces would be useless because they already exist the Javascript way: you just have to put submodules into properties of the main module.

So simple.

So KISS.

That's why I like Javascript.

Do not miss other articles on this subject:

• Part I: Prologue
• Part II: Performances
• Part III: Language comparison

Benchmarks: Truth and Lies

First, some raw benchmarks of PHP versus Javascript powered by V8, provided by The Benchmarks Game here:

According to those benchmarks, the median result give Javascript roughly 10 times faster than PHP.

Also we all know that benchmarks lies, but it's still pretty well known and well accepted that Javascript/V8 is faster than PHP. It's hard to tell how faster Javascript/V8 is for real world application, but expect +20% to +50% of raw speed in favor of it.

So we have a language+engine combination that, at its root, is faster. However we want to build website & webapp, so for this particular task, which one is faster?

Here it will be VERY difficult to come with a meaningful benchmark, because of the PHP paradigm's very nature. PHP's performance are tied with the actual web-server (Apache? NginX? Lighttpd?), how it interacts with it (mod_php? cgi? fast-cgi? php-fpm?), and the opcode cache (none? APC? OPcache?).

You can find benchmark all over the internet comparing PHP & Node.js, but be sure that there are all biased, if not deliberatly lying in favor of the author's language of choice.

You will find that most of time, there are in favor of Node.js... but truth should be told: an "Hello World" benchmark comparing Node.js vs Apache Prefork MPM + mod_php is just bottomless stupidity... And sadly that's what you will get if you google "php nodejs benchmark". At least give PHP a chance to shine!

Hey! If you are one of those awful benchmarks' author, here is a hint about how works a real world PHP stack: first a pool of NginX front server, then a pool of PHP-FPM server in the back-end. So at least for your glorious benchmark, create 2 VM, one with NginX and one with PHP-FPM... Seriously: Apache Prefork MPM *alone* (serving a static "Hello world" HTML) is already slower than Node.js! Do you even know what your benchmarks are measuring?

We should resign about benchmark here and accept that while it's easy to test Javascript/V8 vs PHP/Zend, we will never ever have a meaningful real-world web application benchmark.

Beyond Benchmarks: Some Rationale

Remember: those are just the fastest PHP and JavaScript V8 programs measured on this OS/machine.

-- The benchmark game

But that does not mean that we do not have any clue!

We said the very nature of PHP makes things hard to test, however, this is exactly this way of doing things that leads to a strong belief: the PHP paradigm itself is way slower than the Node.js paradigm.

A typical PHP app works this way:

• the webserver (Apache or NGinX or Lighttpd) receives a client HTTP request for some kind of dynamic content
• the webserver dispatches the request to an idle PHP process
• the PHP process loads the cached opcode of the script
• the PHP process runs the script
• the script runs its startup/init code:
  • if a framework is involved, the framework core is loaded and its startup code is run: this is usually the part that hurts performance the most
  • the PHP script may load session data attached to the requesting user/client, involving a memcached or a database round-trip
  • some application-specific startup may eventually run
• the script processes the request, and eventually send content back to the webserver
• the webserver forwards the response to the client
• the PHP process is recycled, waiting for the next request

A request performed by PHP is more or less like launching a program just for a request, then terminate the program when the client is served, launch again the program for the next request, terminate it, and so on.

The process itself can be recycled thanks to PHP-FPM, and most database drivers offer the persistent connection feature so connection get recycled too: that saves a lot of resources. But ultimately, your script get started and terminated for each single request.

It's a fact that the PHP Zend engine spend way more time executing startup code (mostly because of frameworks) than running the actual business logic of the application.

A typical Node.js app works this way:

• optionnal: NginX (used as a load balancer, HTTPS to HTTP bridge, or websocket bridge) receives a request or establish a connection with the client
  • then it dispatch the request to one of the Node.js service worker
• the Node.js service processes the request or accepts the client connection (by the way all startup code have already been executed when the service had been launched)

A Node.js service can run days!

In fact there is no particular limit, it all depends on the quality of your code.

If your code is full of bug, then probably the service will crash and will be resurrected again by forever every two or three minutes. If your code is clean enough, you can expect it to serve thousands or millions of clients before exiting cleanly.

So the time V8 processes your startup code is really nothing compared to the time it processes the actual business logic of your application.

On the contrary, for each line of your business logic the Zend engine processes, it processes 10, 20, 50 or 100 lines of your framework startup code.

With that in mind, do the math, and you will understand easily why Node.js is so powerful.

The PHP paradigm is defeated.

PHP is rarely the bottleneck?

Some will argue that PHP is rarely the bottleneck.

That's true, PHP, in the PHP's paradigm, is rarely the bottleneck. I mean, Zend engine is fast enough, not as fast as V8, but it is already pretty fast (you may know that PHP is actually faster than Python). However, that's not PHP/Zend itself that is slow, that's the PHP's way of doing things that is not relevant anymore.

The CGI way is doomed... You can evolve it into Fast-CGI/PHP-FPM, that will improve things, but the main issue remains unsolved.

Most probably, if you turn your PHP code into C++, and port all your framework to C++ as well, you will get slighly better performance, but you will not get the best your hardware can deliver anyway.

The truth is that most of the time, your PHP script is waiting for I/O, and I/O are the most time-consuming things. That IS the reasoning behind “PHP is rarely the bottleneck”.

However, it only applies to web application that are I/O bound (which is the case for at least 95% of the web), but if your application is CPU bound PHP will surely become the bottleneck. Web applications that need some complex data processing, Artificial Intelligence (e.g. board games), file-sharing with a complex layer of user access management, file-synchronization... are applications that are CPU-bound. I should mention here that Node.js has hard-time with CPU-bound tasks too, and some strategies should be employed or the event-loop will be drop dead.

However, being the bottleneck or not is one thing, but that's just one aspect of the performance that matters to us.

That suggests that the time to process a single request is not so much tied to the speed of PHP. But what happened when your application is literally besieged by your clients/users? How much time each request will take? How many request will be dropped?

There is indeed at least one bottleneck: the number of PHP process available.

The Power of Event-driven & non-Blocking I/O

Long-lived services capable of handling thousands of concurrent connections taking advantage of a non-blocking evented I/O system are *THE* way to do it now.

When your service is waiting for a response from the database, PHP is just lazily idling, while Node.js will simply accept another concurrent request, start to process it until a new database round-trip is needed... When the database response is available, the request processing is resumed, and so on. That's why a single PHP process can accept one request at a time while a single Node.js process can accept thousands of them.

Creating a process is pretty damn slow, so the PHP strategy is to create a pool of reusable processes.

However each PHP process has its own share of memory, so spawning thousands of them has a cost. Not to mention that CPU context-switching has a cost too.

With Node.js, you can save a lot of CPU. Just spawn as many Node.js services as CPU core available on your system. Evented non-blocking I/O will do the tricks for you: each client's request will trigger your request callback, it's easy and reliable.

Hanging requests waiting for I/O do not take much memory, it costs far less than spawning processes, there is almost no context switching since everything happens into the same Node.js process and it has a dedicated CPU core (remember: we spawn as many Node.js services as available core).

More interesting: your service can happily eat as much resources as available on your system. Your database back-end is engorged? No problem, Node.js can still accept client's request, while PHP would be limited to the number of process of the pool. With Node.js, there is always room for another request. Whenever a request get its precious I/O, your application can serve it to your client.

However like mentioned earlier, Node.js can have some trouble to be aware of, when your application has many CPU-bound task.

The event-loop MUST keep spinning fast, that's the key for blazing fast web application. If some CPU-bound task kick in without giving control back to the event-loop, the event-loop is blocked. That's it: other lightweight tasks waiting for their I/O will freeze, until the CPU-bound task ends (or at least give control back to the event-loop). Node.js is single-threaded, the tasks don't run in a parallel fashion, tasks are concurrents.

If you have a CPU-bound task, you should run it on another Node.js process. A pool of worker is what you need.

This article explains how to overcome those issues.

Final Words

There is no real metrics for that, but I think I made enough good points here, so let's cut it out now: Node.js is way faster than PHP.

In my experience, like I said at the begining of the article, you can expect from +20% to +50% of raw performance in favor of Node.js. By raw performance, I mean: code two CLI programs, one using Javascript and the other using PHP... the JS program will beat the PHP program, running from +20% to +50% faster. Let's be fair: Javascript without using asm.js. By the way Javascript taking advantage of asm.js is just in another league: it has 50% of the C speed, you can do 3D games with that.

But we don't want to code CLI programs or 3D games: that's out of the focus of PHP anyway.

So... for the back-end of a Software as a Service, powering a modern web app? Now that's involving the full PHP stack (Nginx, php-fpm, Memcached/Redis, databases, and eventually a PHP framework), not PHP alone.

In my experience, you can expect Node.js to be 5 times, 10 times or 20 times faster. Even more if the PHP team have chosen one of the full-featured frameworks (e.g. Zend or Symfony2) over a micro-framework. Even more if they have chosen one of the slowest framework (e.g. CakePHP, etc).

One may argue that frameworks should not be taken into consideration. However there are some facts: almost any PHP projects uses a framework. Frameworks are part of the PHP ecosystem, they are generally slow, and because of the PHP paradigm, a PHP framework has more impact on performance than a Node.js framework (mainly because of startup code).

Oh and just don't speak about CMS or blog... Can we even compare Wordpress to Ghost? Yay, Wordpress is the slowest piece of software ever made, I can't blame PHP for that.

Finally PHP can overcome its greatest weakness, I will discuss another day what the PHP of the futur should be, if it want to survive in the long run. But I keep that for one of the next articles on this subject!

Stay tuned!

This is a small tutorial to get started with Electron as a regular Node.js / Npm dependency, so you can build stand-alone app that can be published to npmjs.org.

You should be aware that Electron is the new name of the formerly known Atom-Shell. If you can't figure out what it is, please start reading First encounter with Atom-Shell first. This previous article contains valuable informations and may help you deciding which node/chromium runtime technologies you should pick... the other major contender being NW.js (formerly known as Node-Webkit, name dropped because it is now using io.js and chromium).

Let's get started!

Firstly, create your basic package structure, the usual way. Mine usually looks like this:

.
├── app.js
├── bin/
│   └── my-app
├── front/
│   ├── css
│   ├── html
│   ├── js
│   └── main.html
├── LICENSE
├── log/
├── lib/
├── Makefile
├── node_modules/
├── package.json
├── README.md
└── test/

Here, app.js is the entry point of the Electron application. When your application is not made the stand-alone way, you run with electron followed by the directory of your application, e.g. electron path/to/my/app (assuming Electron is installed globally). Electron needs a package.json, and fortunately, its package is fully compatible with node/npm package. So app.js is the entry point provided as the "main" property of your package.json, and it could be anything you want.

Have in mind that the root directory of your stand-alone application is the root directory of your package.

Also, I like app.js to be as simple and as short as possible. I choose to put Javascript code running in the browser context in the lib/ directory, and I choose to put Javascript code running in the renderer/webpage context into the front/js/ directory (the difference between both contexts has been explained here).

Any front-end code is put into the front/ directory. It could be useful to keep things separated, if one day we want an online website version of our app, and we have to bring back to life the good ol' client/server paradigm.

Secondly, add the electron-prebuilt dependency:

npm install electron-prebuilt --save

If you plan to use third-party package with native code, you should install electron-rebuild as well:

npm install electron-rebuild --save

... I will explain later how to use it properly.

Finally, you need to create your own executable file. For this step, I have simply copied, adapted and reformated the launcher that you can found at ./node_modules/electron-prebuilt/cli.js, and put it into the ./bin/ directory with the name my-app.

I do not append .js to the name, because it will be the real command your user will have to type in their console, if they installed your package globally (e.g. npm install -g my-app). In fact, you CAN provide a different name for the globally installed binaries (see below), but I don't like to confuse myself, I like to stay consistent...

Here is the ./bin/my-app file:

#!/usr/bin/env node

// It just returns a path
var electronPath = require( 'electron-prebuilt' ) ;

var childProcess = require( 'child_process' ) ;

// Adjust the command line arguments: remove the "node <code.js>" part
var args = process.argv.slice( 2 ) ;  
// ... and insert the root path of our application (it's the parent directory)
// as the first argument
args.unshift( __dirname + '/../' ) ;

// Run electron
childProcess.spawn( electronPath , args , { stdio: 'inherit' } ) ;  

Now you can run your application using the command ./bin/my-app, and by the way, if you installed your application globally (npm install -g my-app), you would run it simply with the command: my-app. You should just add that line into your package.json to let npm know about it:

"bin": {
  "my-app": "./bin/my-app"
},

But wait? We don't have an application right now, do we?

Okey, let's copy-paste the code from the quick start documentation of Electron!

Here is our ./app.js file, slightly modified:

var app = require('app');  // Module to control application life.  
var BrowserWindow = require('browser-window');  // Module to create native browser window.

// Report crashes to our server.
require('crash-reporter').start();

// Keep a global reference of the window object, if you don't, the window will
// be closed automatically when the javascript object is GCed.
var mainWindow = null;

// Quit when all windows are closed.
app.on('window-all-closed', function() {  
  if (process.platform != 'darwin')
    app.quit();
});

// This method will be called when Electron has done everything
// initialization and ready for creating browser windows.
app.on('ready', function() {  
  // Create the browser window.
  mainWindow = new BrowserWindow({width: 800, height: 600});

  // and load the index.html of the app.
  mainWindow.loadUrl('file://' + __dirname + '/front/main.html');

  // Open the devtools?
  //mainWindow.openDevTools();

  // Emitted when the window is closed.
  mainWindow.on('closed', function() {
    // Dereference the window object, usually you would store windows
    // in an array if your app supports multi windows, this is the time
    // when you should delete the corresponding element.
    mainWindow = null;
  });
});

And now our ./front/main.html file:

<!DOCTYPE html>  
<html>  
  <head>
    <title>Hello World!</title>
  </head>
  <body>
    <h1>Hello World!</h1>
    We are using io.js <script>document.write(process.version)</script>
    and Electron <script>document.write(process.versions['electron'])</script>.
  </body>
</html>  

Now you can run your wonderful app! Now just run ./bin/my-app and let it shine!

Using package featuring native code

Electron use a version of Chromium that is different from node.js. So native packages cannot work without being rebuilt.

Rebuilding can be painful, hopefully electron-rebuild is here to do the job for us.

You should have installed it using npm install electron-rebuild --save already.

Anytime you install some packages featuring native code, you should run electron-rebuild afterward. E.g. if we want to install the package whatever, we have to do it in two step:

npm install whatever --save  
./node_modules/.bin/electron-rebuild

I should warn you here:

“electron-rebuild is REALLY slow”

... and aside from that:

“electron-rebuild does NOT output ANYTHING”

It's really misleading...

The first time I used it, I was thinking that it was not working at all, and I Ctrl-C it multiple times... Then, I finally give it a last chance to shine, while I was making a tea. On my two year old laptop, it takes few minutes.

Also people installing your supa-ground-breaking package are not aware of electron-rebuild, so you have to add a line into your package.json, so it will kick-in at installation time. Here is the line:

"scripts": {
    "install": "electron-rebuild"
},

You should also warn your user that the installation process can take a while...

That's all folks!

Do not miss other articles on this subject:

• Part I: Prologue
• Part II: Performances
• Part III: Language comparison

In the blue corner, the defender, the world wide web superstar, having no less than 70% of website using it as their server-side solution, the good friend of all daddy's personnal homepages, the companion of some of the top-ten biggest websites in the world including Wikipedia and Wordpress.com, having a tons of fans and a tons of haters, here is the three-letters world wide famous P... H... P... !

In the red corner, the challenger, the rising star of web's server-side solution, the solution of choice for any real-time, modern & reactive applications, using the programming language the most used in the world, the interpreted language powered by the fastest engine in the world, a language with an ever-growing popularity and use cases that even gnome-shell used it to create apps, a language with "good parts"... and bad parts, the most misunderstood language, here is Node.js & Javascript!

Some backgrounds of mine

Since 2007, I was a big fan of PHP.

PHP has a friendly learning curve and open up a whole world of possibility.

At that time, it was crystal clear to me that I MUST master this technology. I'm not into Windows, so ASP, .NET & co was not fitting my usual dev environment and philosophy. I have always found that Java's technologies was bloated and cumbersome. Perl looks too esoteric for me, Python looked interesting, but PHP's ecosystem was already beating those two for web applications.

So, at that time, learning PHP was the good choice for me.

Later, I was hired to build a large application, a full-featured project management SaaS, with file-sharing and file-synchronization.

Then a new player appears: Node.js.

I first heard it when I was searching for a decent way of using websocket alongside with PHP. That was really a pain in the neck. However, the young Node.js was already the best solution for that task.

Later, I was searching for an alternative for my company's file-sharing/file-synchronization service.

PHP was not exactly the better tools for that.

I put a lot of love and engineering into my PHP algorithm, so it worked pretty well for years, but as people join and add thousands of files, it becomes more and more complicated to scale.

Meanwhile, internet has changed...

People were more and more used to real-time application. The old request-response server paradigm was begining to fall apart. Our file-synchronisation service used to receive a request for update every 2 minutes per client applications.

But then our customers wanted real-time.

They wanted that when a coworker add a new file, the file would appears immediately on all the company's computers.

Reducing the update delay was not an option: it would cost at least +25% of resources to support one request per minute per client, and still, the customers would argue that one minute is not real-time. Perhaps one request every 10 seconds would pleased them, but this service would cost us at least four times more resources, not to mention the scalability nightmare.

If our customers want real-time, we need a persistent connection: the client should be wired to the service, and Websocket is the solution. So again, I considered Node.js for that task.

I started to realize that the web was changing, and that the good old PHP doesn't fit the new deal very well.

So I started learning intensively Node.js in the middle of 2013.

And believe me or not: it's a true love story.

The Node.js paradigm was blowing my mind, this wonderful event-driven/asynchronous/non-blocking I/O thing can do wonders. This model is just what make NginX kick Apache's ass. Its natural affinity with real-time and cutting edge technologies was really appealing.

Finally, we can code both side with the same language (No! Do not insist! Java is NOT an option! Come on: nobody is using browser-side Java anymore, the Dark Age is over).

Also Node.js is capable of more than just webservices, it can be used as a general purpose language/runtime, for various applications, even desktop applications and games using node-webkit / nw.js or atom-shell / electron.

Also, I found out that Node.js has a really fine community.

Here people are enthusiast, people are curious, people are happy. People are humble here, something really rare nowadays, since the Java's spirit has spread the world with its arrogant and obscurantist mindset, and now the majority of coders are passionless, serious and pretentious people. As for PHP, I always found it has a bloated community.

So thanks to Node.js, I have finally decided to go out of the dark, and start publishing various tools of mine. Thanks to the wonderful NPM registry, it is really easy and friendly.

You can see few public lib of mine here.

Now I am a freelance and an entrepreneur, and I try to work less and less on PHP projects... I'm certain that the Node.js' path will be the successfull one.

Next time!

For the next article of this series, I will go further and detail closely the difference between Node.js and PHP, the pros & cons of each technologies.

See you soon!

This is the third part of a series of tutorials about terminal-friendly application. This one will focus on user inputs!

• Terminal application part. I: Styles & Colors
• Terminal application part. II: Moving & Editing
• Terminal application part. III: User Inputs

This tutorial will focus exclusively on the Terminal-kit lib for Node.js. Make sure to npm install terminal-kit before trying the given examples.

Keyboard inputs!

The .grabInput() method turns input grabbing on. When input grabbing is on, the terminal will switch to what is known as the raw mode.

From wikipedia:

Raw mode:

Raw mode is the other of the two character-at-a-time modes. The line discipline performs no line editing, and the control sequences for both line editing functions and the various special characters ("interrupt", "quit", and flow control) are treated as normal character input. Applications programs reading from the terminal receive characters immediately, and receive the entire character stream unaltered, just as it came from the terminal device itself.

The first thing you need to know: your program will not exit when you hit CTRL-C anymore. Instead, you will have to watch for CTRL-C and use process.exit() by yourself.

Turning input grabbing on make your terminal interface emit key events.

Here is a small example:

var term = require( 'terminal-kit' ).terminal ;

term.grabInput() ;

term.on( 'key' , function( name , matches , data ) {  
    console.log( "'key' event:" , name ) ;

    // Detect CTRL-C and exit 'manually'
    if ( key === 'CTRL_C' ) { process.exit() ; }
} ) ;

The key event is emitted with three arguments:

• name string the key name
• matches Array of matched key name
• data Object contains more informations, mostly useful for debugging purpose, where:
  • isCharacter boolean is true if this is a regular character, i.e. not a control character
  • codepoint number (optional) the utf-8 code point of the character, if relevant
  • code number or Buffer, for multibyte character it is the raw Buffer input, for single byte character it is a number between 0 and 255

Usually, if the name argument's length is 1, this is a regular character, if it is longer, it is a special key code, like CTRL_C, ENTER, DELETE, TAB, UP , HOME , F1, and so on... But be careful! A single asian character (Chinese, Japanese and Korean) always has a length of 2, so you should not rely on that and instead always data.isCharacter if you want to know if it is a true regular character or not.

The full list of special key's code can be found here

Note that there are few issues with the way keys produce inputs in a terminal application that you should be aware of.

That's not the lib that should be blamed for that, but the way terminals actually works. Have in mind that it's not a kind of keyboard driver that pass keys to our application, we are just reading from the Standard Input stream (STDIN). And that's your terminal that pushes bytes into that STDIN stream.

For that purpose, the matches argument contains all matched keys. This is because sometimes, the input stream produces code that matches many possibilities. E.g. ENTER, KP_ENTER and CTRL_M are all producing a 0x0d in STDIN. TAB and CTRL_I both produce 0x09, BACKSPACE usually produces a 0x08 like CTRL_H, ...

When multiple matches happens, Terminal-kit will pass as the name argument the most useful matches. By the way ENTER has a greater priority than CTRL_M, TAB has greater priority than CTRL_I and BACKSPACE greater priority than CTRL_H.

Actually, all Ctrl-letter combo key produce a control character, i.e. one of the 32 lower ASCII character. But most of those control characters are useless nowaday, so it is safe to use almost all Ctrl-letter except CTRL_M, CTRL_I and CTRL_H.

Finally, you should be aware that special keys produce input in STDIN that vary greatly from one terminal to another. E.g. there is rarely two terminals that produce the same escape sequence for all F1 to F12 keys. Terminal-kit try to abstract that away from you, but exotic terminals can still causes some detection troubles. That's because there isn't any standard for that.

Also, some terminals like Gnome-terminal will intercept function keys for their own stuffs, e.g. F1 will open the Gnome-terminal help window, F11 will go fullscreen, ALT_F4 will close the window, and your application will never get those intercepted keys. So, the best practice is to bind multiple keys for the same action in your application. If you are going to use function keys, try to bind a function key and it's shift or ctrl variant to the same action, e.g. F1, CTRL_F1 and SHIFT_F1: if the terminal intercepts F1, there are chances that SHIFT_F1 will work...

Even better, if it's relevant and you can afford it, allow your users to configure their own key binding.

When you are done with user input, you can turn input grabbing off with .grabInput( false ). The terminal will leave the raw mode and returns to the cooked mode.

A bit further: mouse handling!

Terminal-kit supports mouse handling. To turn mouse handling on, simply pass an object of options to .grabInput()!

Example:

var term = require( 'terminal-kit' ).terminal ;

term.grabInput( { mouse: 'button' } ) ;

term.on( 'mouse' , function( name , data ) {  
    console.log( "'mouse' event:" , name , data ) ;
} ) ;

The mouse option can take three values:

• 'button': report only button-event
• 'drag': report button-event and report motion-event only when a button is pressed (i.e. a mouse drag)
• 'motion': report button-event and all motion-event, use it only when needed, many escape sequences are sent from the terminal (e.g. you may consider it for script running over SSH)

The key event is emitted with two arguments:

• name string the name of the subtype of event
• data Object provide the mouse coordinates and keyboard modifiers status, where:
  
  • x number the row number where the mouse is
  • y number the column number where the mouse is
  • ctrl boolean true if the CTRL key is down or not
  • alt boolean true if the ALT key is down or not
  • shift boolean true if the SHIFT key is down or not

The argument 'name' can be:

• MOUSE_LEFT_BUTTON_PRESSED: well... it is emited when the left mouse button is pressed
• MOUSE_LEFT_BUTTON_RELEASED: when this button is released.
• MOUSE_RIGHT_BUTTON_PRESSED, MOUSE_RIGHT_BUTTON_RELEASED, MOUSE_MIDDLE_BUTTON_PRESSED, MOUSE_MIDDEL_BUTTON_RELEASED: self explanatory.
• MOUSE_WHEEL_UP, MOUSE_WHEEL_DOWN: self explanatory
• MOUSE_OTHER_BUTTON_PRESSED, MOUSE_OTHER_BUTTON_RELEASED: a fourth mouse button is sometime supported by terminals.
• MOUSE_BUTTON_RELEASED: a button were released, however the terminal does not tell us which one.
• MOUSE_MOTION: if the option { mouse: 'motion' } is passed to grabInput(), every moves of the mouse will fire this event, if { mouse: 'drag' } is given, it will be fired if the mouse move while a button is pressed.

Again, there are some issues to be aware of.

Firstly, do not expect all terminals to emit all *_RELEASED subtype. You should not rely on this, or you should at least have some fallbacks. E.g. Gnome-terminal emits MOUSE_LEFT_BUTTON_RELEASED and MOUSE_RIGHT_BUTTON_RELEASED, but does not emit MOUSE_MIDDEL_BUTTON_RELEASED... don't ask me why... -_-'

Secondly, do not expect all terminals to support the option { mouse: 'motion' }. E.g. the KDE Konsole will only report the MOUSE_MOTION event-subtype when a button is pressed, the same way it works with the { mouse: 'drag' } mode.

Thirdly, some terminals intercept right click to display a context menu. Gnome-terminal used to do that, but it seems that newer versions (at least on my Fedora at time of writing) don't do that anymore when the terminal has switched to raw mode, which was done with .grabInput().

By the way, the good old Xterm works perfectly fine! Outdated UI/UX, but extremely reliable when it comes to raw features support.

The Ultimate Geek Touch: Terminal-kit even supports the mouse in the Linux Console by talking directly with the GPM driver if it is installed on your box. Seriously, I'm quite proud of that, since I have almost done reverse engineering to provide that. Yay, there is no documentation for the GPM driver, so one have to: read the source code, watch inputs and outputs, guess how it works, repeat.

Putting it all together

Here a small sample code that allows one to write anywhere on the screen, using arrow keys to move while other keys are echoed:

var term = require( 'terminal-kit' ).terminal ;

term.grabInput( { mouse: 'button' } ) ;

term.on( 'key' , function( key , matches , data ) {

    switch ( key )
    {
        case 'UP' : term.up( 1 ) ; break ;
        case 'DOWN' : term.down( 1 ) ; break ;
        case 'LEFT' : term.left( 1 ) ; break ;
        case 'RIGHT' : term.right( 1 ) ; break ;
        case 'CTRL_C' : process.exit() ; break ;
        default:   
            // Echo anything else
            term.noFormat(
                Buffer.isBuffer( data.code ) ?
                    data.code :
                    String.fromCharCode( data.code )
            ) ;
            break ;
    }
} ) ;

term.on( 'mouse' , function( name , data ) {  
    term.moveTo( data.x , data.y ) ;
} ) ;

Misc inputs with the 'terminal' event

The terminal event is a general purpose event for all things coming from your terminal that are not key or mouse event.

The terminal event is emitted with two arguments:

• name string the name of the subtype of event
• data Object provide some data depending on the event's subtype

The SCREEN_RESIZE subtype is emited when the terminal is resized by the user. The data argument will contain the width and height property: the new size of the screen expressed in characters.

Finally, if .grabInput() was called with the { focus: true } option, a terminal event will be emited with FOCUS_IN or FOCUS_OUT subtype when the terminal gains or loses focus. Not all terminal supports that.

Next time?

So, we have learn many interesting things, but we have not explored all features Terminal-kit has.

Next time we will learn how to use higher level user-inputs methods, like .inputField().

I hope you enjoyed this tutorial!

• Terminal application part. I: Styles & Colors
• Terminal application part. II: Moving & Editing
• Terminal application part. III: User Inputs

This tutorial will focus exclusively on the terminal-kit lib for Node.js. Make sure to npm install terminal-kit before trying the given examples.

For those who have already done some C/C++ coding with Ncurses, you may know that there is a saying:

More than often, a programmer dealing with Ncurses will curse.

The goal of terminal-kit is to provide a simple and modern way to interact with the terminal. The design should be simple and intuitive, and yet give maximum power in the hand of the user. Hopefully, you will say goodbye to the good ol' Ncurses' days!

Moving the cursor

That cannot be easier! 

To move the cursor relative to its position:

• .up( n ): move the cursor up by n cells
• .down( n ): move it down by n cells…
• .left( n ): …
• .right( n ): …
• .nextLine(n): move the cursor to beginning of the line, n lines down
• .previousLine(n): move the cursor to beginning of the line, n lines up
• .column(x): move the cursor to column x

Example:

var term = require( 'terminal-kit' ).terminal ;  
term( "Hello" ) ;  
term.down( 1 ) ;  
term.left( 1 ) ;  
term( "^-- world!\n" ) ;  
/* It writes:
Hello  
    ^-- world!
*/

Also, you can one-line that, this way:

term( "Hello" ).down.left( 1 , 1 , "^-- world!\n" ) ;  

Also you can use move() and moveTo() to achieve relative and absolute positioning:

• .moveTo(x,y): move the cursor to the (x,y) coordinate (1,1 is the upper-left corner)
• .move(x,y): relative move of the cursor

With those methods, you can move it move it anywhere. Creating an app with a nice layout is almost at our finger...

One last couple of function that can be useful:

• .saveCursor(): save cursor position
• .restoreCursor(): restore a previously saved cursor position

This is extremely useful in many case, for example i you want to update status bar:

var term = require( 'terminal-kit' ).terminal ;

term( "Mike says: " ) ;

term.saveCursor() ;  
term.moveTo.bgWhite.black( 1 , 1 ).eraseLine() ;  
term( "This is a status bar update!" ) ;  
term.white.bgBlack() ;  
term.restoreCursor() ;

term( '"Hey hey hey!"\n' ) ;  
// Cursor is back to its previous position
// Thus producing: Mike says: "Hey hey hey!"

This creates a status bar with a white background and black text color, on the top of the screen. The update code saves the cursor's position and restores it, so it doesn't disturb the main text flow, that's why it still writes Mike says: "Hey hey hey!" as if no cursor movement happened.

Editing the screen

Now that we've learn how to move the cursor, let's see how to edit what is already displayed.

Let's see what we can do:

• .clear(): clear the screen and move the cursor to the upper-left corner
• .eraseDisplayBelow(): erase everything below the cursor
• .eraseDisplayAbove(): erase everything above the cursor
• .eraseDisplay(): erase everything
• .eraseLineAfter(): erase current line after the cursor
• .eraseLineBefore(): erase current line before the cursor
• .eraseLine(): erase current line
• .insertLine(n): insert n lines
• .deleteLine(n): delete n lines
• .insert(n): insert n char after (works like INSERT on the keyboard)
• .delete(n): delete n char after (works like DELETE on the keyboard)
• .backDelete(): delete one char backward (works like BACKSPACE on the keyboard), shorthand composed by a .left(1) followed by a .delete(1)

E.g., move the cursor and then erase anything below it to clean the area:

term.moveTo( 1 , 5 ) ;  
term.eraseDisplayBelow() ;  

For all erase-like methods, note that the screen is erased using the current background color! Hence, if we wanted to erase the screen, and paint anything below the cursor with a red background, we can modify the above code this way:

term.moveTo( 1 , 5 ) ;  
term.bgRed() ;  
term.eraseDisplayBelow() ;  

This apply to .clear() as well, so term.bgBlue.clear() will erase the whole screen, paint it blue, and move the cursor to the top-left corner.

Also there is the advanced method .fullscreen( true ) (not chainable) that clears the screen, moves the cursor to the top-left corner, and if the terminal supports it, it turns the alternate screen buffer on.

If alternate screen buffer is available, when you invoke .fullscreen( false ), the screen will be restored into the state it was before calling .fullscreen( true ).

This is really a key feature for writing cool terminal application! Now you can code something that behaves like the htop linux command, using the whole terminal display area, and when users quit your app, the command history of their shell will be restored. Neat!

If alternate screen buffer is not supported by your terminal, it will fail gracefully.

Last words

Now the cool factor: you can change your terminal window's title with the method .windowTitle():

term.windowTitle( "My wonderful app" )` ;    // set the title of the window to "My wonderful app"  

Hehe ;)

Next time we will focus on input: keyboard and mouse!

I hope you have found this tutorial useful, and will be happy if you drop me a line or two!

Have a nice day!

• Terminal application part. I: Styles & Colors
• Terminal application part. II: Moving & Editing
• Terminal application part. III: User Inputs

Prologue

Have you ever wanted to make your CLI script shine? While cool scripts expose colors and styles, you are still stuck with black & white boring text? Hey! I've got something for you!

In this series of article, we will explore how to build a great terminal application: how to move the cursor, edit the screen, handle keyboard input, handle mouse input, and even more goodies!

But let's start with something simpler... Life is really bland without colors, doesn't it?... Let's add some spicy colors to our terminal application!

The hard way

You wonder how colors and styles are achieved? You guess that one have do deal with an obscur underlying driver? Actually, you're wrong: you can do that with a simple console.log().

So let's write a red "Hello world!":

console.log( '\x1b[31mHello world!' ) ;  

Yes, you guessed it: \x1b[31m is an escape sequence that will be intercepted by your terminal and instructs it to switch to the red color. In fact, \x1b is the code for the non-printable control character escape... you know, that key on the top-left corner of your keyboard?

Eventually, if we want to produce a notice of this kind:
Warning: the error #105 just happened!
... you will have to code this:

console.log( '\x1b[31m\x1b[1mWarning:\x1b[22m \x1b[93mthe error \x1b[4m#105\x1b[24m just happened!\x1b[0m' ) ;  

Just copy-paste it to the node's REPL console, to try it out!

Okey, now if you want the full list of what you can achieve with control sequences, have a look at the official xterm's control sequences guide.

Isn't that nice? We don't have to deal with an obscur underlying driver! We just have to deal with obscur escape sequences!

But that's not all: complex sequences are not compatible between all terminals.

Ok, ok... Let's hunt for a good lib then...

Popular libs

Since the goal of this part is just to put some colors and styles to our terminal application, there are two popular packages with a very high download count in the npm's registry: colors and chalk.

Escape sequences dealing only with colors & styles are known as ANSI escape code, they are standardized and therefore they work everywhere. That's the focus of those two packages.

The first package, colors, is probably the oldest package around dealing with colors and styles. At time of writing, it has no less than 3.5 millions of downloads for the last month.

Some example of the syntax, taken directly from the package's documentation:

var colors = require('colors');

console.log('hello'.green); // outputs green text  
console.log('i like cake and pies'.underline.red) // outputs red underlined text  
console.log('inverse the color'.inverse); // inverses the color  
console.log('OMG Rainbows!'.rainbow); // rainbow  
console.log('Run the trap'.trap); // Drops the bass  

This is the old syntax. While it looks really easy to use, note that it extends the native String object, which is a very bad thing.

Due to popular pressure, some of them directly coming from the author of the chalk package itself, now colors can be required without extending native object... Of course, the syntax differs:

var colors = require('colors/safe');

console.log(colors.green('hello')); // outputs green text  
console.log(colors.red.underline('i like cake and pies')) // outputs red underlined text  
console.log(colors.inverse('inverse the color')); // inverses the color  
console.log(colors.rainbow('OMG Rainbows!')); // rainbow  
console.log(colors.trap('Run the trap')); // Drops the bass  

The chalk package has recently surpassed colors, at time of writing its download count reached 5.7 millions.

Again, taken directly from the chalk's documentation:

var chalk = require('chalk');

// style a string 
console.log( chalk.blue('Hello world!') );

// combine styled and normal strings 
console.log( chalk.blue('Hello') + 'World' + chalk.red('!') );

// compose multiple styles using the chainable API 
console.log( chalk.blue.bgRed.bold('Hello world!') );

// pass in multiple arguments 
console.log( chalk.blue('Hello', 'World!', 'Foo', 'bar', 'biz', 'baz') );

// nest styles 
console.log( chalk.red('Hello', chalk.underline.bgBlue('world') + '!') );

// nest styles of the same type even (color, underline, background) 
console.log( chalk.green(  
    'I am a green line ' +
    chalk.blue.underline.bold('with a blue substring') +
    ' that becomes green again!'
) );

The outsider: terminal-kit

It's time to introduce my little gem: terminal-kit.

The basic part of terminal-kit, the part dealing with styles and colors was inspired by chalk. It's easy, you can combine styles and colors, and furthermore, you don't have to use console.log() anymore. Why? Because terminal-kit IS a terminal lib, not just an ANSI's style helper.

Just compare the chalk's way:

console.log( chalk.blue.bold( 'Hello world!' ) ) ;  

... and the terminal-kit's way:

term.blue.bold( 'Hello world!' ) ;  

Less cumbersome, right?

This is the spaceship demo! After doing an npm install terminal-kit, go to node_modules/terminal-kit/demo/ and run ./spaceship.js to see it alive!

Let's have a look to some of the terminal-kit's primer, from the documentation:

// Require the lib, get a working terminal 
var term = require( 'terminal-kit' ).terminal ;

// The term() function simply output a string to stdout, using current style 
// output "Hello world!" in default terminal's colors 
term( 'Hello world!\n' ) ;

// This output 'red' in red 
term.red( 'red' ) ;

// This output 'bold' in bold 
term.bold( 'bold' ) ;

// output 'mixed' using bold, underlined & red, exposing the style-mixing syntax 
term.bold.underline.red( 'mixed' ) ; 

// printf() style formating everywhere: this will output 'My name is Jack, I'm 32.' in green 
term.green( "My name is %s, I'm %d.\n" , 'Jack' , 32 ) ;

// Width and height of the terminal 
term( 'The terminal size is %dx%d' , term.width , term.height ) ;

// Move the cursor at the upper-left corner 
term.moveTo( 1 , 1 ) ;

// We can always pass additionnal arguments that will be displayed... 
term.moveTo( 1 , 1 , 'Upper-left corner' ) ;

// ... and formated 
term.moveTo( 1 , 1 , "My name is %s, I'm %d.\n" , 'Jack' , 32 ) ;

// ... or even combined with other styles 
term.moveTo.cyan( 1 , 1 , "My name is %s, I'm %d.\n" , 'Jack' , 32  ) ;  

Okey, what if you don't want to output a string directly on the terminal, but just put it into a variable like colors and chalk does? Easy: just add the str property into your chainable combo:

var myString = term.red.str( 'Hello world!\n' ) ;  

It happens that at least 95% of time, we deal with color only when we are about to output something directly into the terminal, and don't care about storing a string containing obscur escape sequences into some variable... That's why the default behaviour of the lib is to print, not to return.

If you want to output to stderr instead, just add the error property into your chainable combo:

term.red.error( 'Some errors occurs...\n' ) ;  

Here is the list of styles the lib support:

• styleReset: reset all styles and go back to default colors
• bold: bold text
• dim: faint color
• italic: italic
• underline: underline
• blink: blink text, not widely supported
• inverse: foreground and background color
• hidden: invisible, but can be copy/paste'd
• strike: strike through

Note that your terminal may not support all of those features.

And now, here is the list of supported colors: black, red, green, yellow (actually brown or orange, most of the time), blue, magenta, cyan, white, brightBlack (grey, it sounds dumb, but in fact it's the combo of black and bright escape code), brightRed, brightGreen, brightYellow (true yellow), brightBlue, brightMagenta, brightCyan and brightWhite.

If you want to change the background color of characters about to be written, just prefix the above colors with 'bg' (and don't forget the camelCase rule, e.g. cyan becomes bgCyan).

For example, if you want to output 'Hello world!', 'Hello' in bold red over cyan and 'world!' in italic green over magenta for maximum contrast and bleeding eyes, just do that:

term.red.bgCyan.bold( 'Hello ' ).green.bgMagenta.italic('world!\n') ;  

Hello world!

Ok, ok... Let's stop making that kind of insane example now...

Finally, if we are not bothered by very old terminals, we can use all of the 256 colors. Most terminal support them.

• .color256(register): it chooses between one of the 256 colors directly
• .colorRgb(r,g,b): pick the closest match for an RGB value (from a 16 or 256 colors palette or even the exact color if the terminal support 24 bits colors), r,g,b are in the 0..255 range
• .colorGrayscale(l): pick the closest match for a grayscale value (from a 16 or 256 colors palette or even the exact color if the terminal support 24 bits colors), l is in the 0..255 range

Example:

term.colorRgb( 0x33 , 0xff , 0x88 , "This is teal!" )  

They all have their bg* counter-part, e.g.:

term.bgColorRgb( 0x33 , 0xff , 0x88 , "The background color is teal!" )  

It's worth noting that providing some text to a style method will apply the color *ONLY* for the given text, while calling a style method without text will turn the current style on, until another style contradict that.

E.g:

term.red( "Red!\n" ) ;  
term( "This is white\n" ) ;

term.red()( "Red!\n" ) ;  // Note the parenthesis  
term( "This is red\n" ) ;  

In practice, it's always good to define some handy shortcuts. For example, one may want to define a standard way to output errors. Putting it all together:

var logError = require( 'terminal-kit' ).terminal.error.bold.red ;

// Later...

logError( 'Oh my! A bad thing happens!\n' ) ;  

I hope you liked this little tutorial, next time we will learn how to move the cursor and edit the screen.

See ya! ;)

Truthy and Falsy

First, we need to understand the truthy and falsy concept, because it will be widely used in the second part.

Truthy and falsy is a common concept among dynamic language featuring loose typing.

Any value, whatever the type, belong to the truthy or falsy group. When an operator need a boolean value, truthy will be true and falsy will be false.

Those values are falsy:

undefined  
null  
false  
0  
NaN  
''  

Those values are truthy:

true  
1  
123    // any numbers that are not 0 are truthy  
'some random string'    // any non-empty string are truthy  
[]    // any array are truthy, even empty arrays
{}    // any object are truthy, even empty objects, except null (yay, null is an object...)

An if statement will execute its block of code if the expression is truthy.

if ( expression )  
{
    // if expression is truthy, the following lines are executed
    ...
}
else  
{
    // if expression is falsy, the following lines are executed
    ...
}

Syntactic Sugar

Here an unordered list of shortcuts that eases life and helps maintaining a neat and elegant codebase.

• !! expression: a very elegant way to cast a variable or an expression into a boolean, much more pleasant than the cumbersome expression ? true : false. It is also known as the double not. How it works? ! returns the inverse boolean value of an expression. Apply it twice and you are back to the original logical value of the expression, casted as a boolean.

• + stringExpression: it converts a string expression into a number, e.g. typeof + '42' === 'number' is true. It works if + is used as an unary operator, when there are no left member.

• + new Date(): it converts the Date value into a number, same mechanics than the previous one.

• optionalArgument || defaultValue: it returns optionalArgument if it is truthy, else it returns defaultValue. || is sometime refered as the default operator. In Javascript, the || and && operator are a bit like flow controle, they don't produce booleans, but instead they return the last evaluated operand. Still, the returned operand is considerated truthy or falsy according to the logical rules of AND and OR operators.

  Also, the first variable was named optionalArgument on purpose. Because it is an elegant way to define default parameter for functions when the type of the parameter is expected to be always truthy (object, array). Do not use when falsy values are expected.

function fn( optionalObject )  
{
  optionalObject = optionalObject || {} ;
  ...
}

• object && object.child && object.child.property: this is the guard operator, allowing us to safely access the property of nested objects. Without that, accessing directly object.child.property would raise an error if object was not an object, or if object has no child property.

• !a !== !b: this is the easiest way to achieve a logical XOR (eXclusive OR) in Javascript. This expression is true if a is truthy or b is truthy, but not both of them. The not operator cast a and b to boolean (while reversing their logical value), then if they are different, one and only one of them is true, therefore the whole expression is true.

Feel free to comment and share your own favorite syntactic sugar with us!

Every JavaScript's bashers will troll you with this thing: the NaN === NaN statement is false.

At first glance, this looks like an awful misconception. It's very misleading, and it is always temptful to compare some variable with NaN. I used to think that it was a bad design choice that lead to this.

But I was wrong.

That's true, NaN is an unruly child. But the fact that “NaN is not NaN” is an absolute necessity.

I will try to explain you the reason.

But what is NaN?

NaN stands for “Not A Number”.

Still, typeof NaN is 'number'. More lulz for bashers... ;)

But again, there are some rationale behind that fact.

NaN is produced by some arithmetic operation, when the result is unknown and cannot be represented, even by Infinity.

Some examples of expression and their values:

1 / 0    // Infinity  
-1 / 0    // -Infinity
0 / 0    // NaN  
Infinity / Infinity // NaN  
Infinity * 0    // NaN  
Infinity - Infinity    // NaN  
Math.sqrt( -1 )    // NaN  
Math.log( -1 )    // NaN  

NaN is used when the result is undefined (e.g. 0/0), or is not a real number (the square root of all negative numbers are imaginary numbers).

Therefore, NaN, as a language feature, IS programmatically a 'number'. That's why typeof NaN === 'number'. But as a mathematical entity, it IS NOT a number.

It MUST be of number type, so it can be autoboxed as an instance of Number, and thus have access to all methods of the Number's prototype. It is clear that any arithmetic operator or methods of the Math object should return a Number.

NaN is Not A Number from a mathematical point of view, not from a programming language point of view. Many programming language have a NaN concept, it is not a particularity or a fantasy of the JavaScript language. In fact, NaN is part of the floating-point spec, quoting Wikipedia:

Systematic use of NaNs was introduced by the IEEE 754 floating-point standard in 1985, along with the representation of other non-finite quantities like infinities.

Actually NaN exists in your bare-metal CPU's data representation, it is a special case of the float and of the double type. You have read it: it IS a special case of the float and the double type, that's why JavaScript tells you that typeof NaN === 'number'. Trust JavaScript, it is right!

Why NaN can't compare with itself?

Okey, again that's a question of point of view.

The main trouble comes from the fact that those operators == === != !== have two meanings. The first and immediate meaning is the mathematical equality. However, almost all programming language use the same operator to test equality or identity of any kind of non-numeric instance, e.g. string comparison, object identity, and so on.

NaN !== NaN is choking you, because you read it as an object's instance comparison. But NaN is of number type, so it is the mathematical version of those operators that kicks in there.

Ok, so you are now asking yourself why on earth NaN is different from NaN from a mathematical point of view?

That's pretty simple.

NaN is not a number, it represents something unknown.

Can you tell me how Infinity - Infinity and 0 / 0 compare? Is it really the same value to you? I hope your answer is NO or you need to study the math a bit more.

Moreover, compare Math.sqrt( -1 ) to Math.sqrt( -4 )... Both value are NaN because the square root of a negative number is an imaginary number. In the first case, the Complex Number value is i, in the second case it is 2i. Now I'm sure that you are glad that the JavaScript tells you that Math.sqrt( -1 ) !== Math.sqrt( -4 ) is true.

But then why Math.sqrt( -1 ) !== Math.sqrt( -1 )?

Again, that's logical. We all know how a programming language breaks down big expressions into smaller pieces.

First the left Math.sqrt( -1 ) is evalued and replaced by NaN, then the right Math.sqrt( -1 ) is evalued and replaced by NaN too. Finally the two NaN are compared with each other. At that point, JavaScript cannot tell if the first NaN represent i, 2i or whatever, since like most language, it doesn't have a native type for Imaginary Number. All it has is a comparison of two floating-point numbers, both saying that there are not floating-point numbers anymore.

If you are still confused, understand NaN as an Unknown Number.

NaN is not greater than anything, neither lesser than anything

We know for sure that NaN !== 0 is false.

However, NaN < 0 is false too, as well as NaN > 0.

You can try that for any numbers or Infinity (or even NaN itself), it will be false anyway!

How to deal with NaN

Short answer: use the global isNaN() function.

Better: if you don't have to deal with outdated browsers, use Number.isNaN(). It is more reliable. It is part of ECMAScript 6, a.k.a. Harmony.

You can learn the difference between those two on MDN.

Be aware: in many other programming language, things like a division by zero will throw an exception. JavaScript DO NOT throw any exception for that. The resulting value is simply NaN.

So you have to either check your input or your output, depending on what your code is doing.

You have to check with isNaN() everytime NaN could be expected, because:

NaN is viral!

Any computation with NaN produces NaN.

Let's look at that method, having in mind that this.a and value both equals zero:

MyClass.prototype.unsafe = function( value )  
{
    this.b = this.a / value ;   // Now this.b is NaN
    this.c += this.b ;  // Now this.c is NaN
    this.d = 15 + this.c + this.e * this.g ;    // Now this.d is NaN
    ...
}

As you can see, your whole object will be filled with NaN in no time!

So be careful!

Finally: the good news!

Yes, I have a good news for you: NaN is a falsy value.

So a simple check with if ( myNumber ) { blah() ; } will avoid it, just like 0 null undefined false '' (...).

To be safe, just turn the previous code into that:

MyClass.prototype.safe = function( value )  
{
    if ( ! value || typeof value !== 'number' )
    {
        // no division by zero or division by NaN anymore
        throw new Error( "Bad argument #0" ) ;
    }

    this.b = this.a / value ;
    this.c += this.b ;
    this.d = 15 + this.c + this.e * this.g ;
    ...
}

On the contrary, Infinity is a truthy value. Hopefully! ;)

Update: Atom-shell is now knwon as Electron, and has a brand new website: electron.atom.io.

Atom-shell vs other node/chromium runtime technologies

Atom-shell is a framework based on node.js that include Chromium for rendering web page as well as for desktop integration. We can roughly state that it combines both the server-side and the client-side to build great multi-purpose desktop application.

From Atom-shell creators:

You could also see it as a minimal Chromium browser, controlled by JavaScript.

It is backed by Github, and looks promising. It all begins as a specific framework for the Atom IDE, but it has quickly proven that its goal was to provide a great framework for a large variety of desktop application.

It is one of the three frameworks doing that, namely nw.js, formerly known as node-webkit, and brackets-shell by Adobe.

I have discarded brackets-shell early in the process, because Adobe seems to develop it only for the brackets IDE purpose. There was no guarantee that it will be improved in the long run, neither that it will be developed as a multi-purpose framework in mind. Also in the past, I have been highly frustrated with the Adobe's way of documenting things... I remember Adobe AIR, and frankly: I don't want to experience such a pain again...

Also from their own overview:

Note: The brackets-shell is only maintained for use by the Brackets project. Although some people have definitely had success using it as an app shell for other projects, we don't provide any official support for that and we haven't done a ton of work to make the app shell easily reusable. Many people will likely find it easier to use a project like node-webkit, which is more generic by design.

This notice alone convinced me to stay away from that, forever.

But here is the serious challenger: node-webkit a.k.a. nw.js.

This framework already have a dozen of real life application, and there is even a game in the Steam store.

Node-webkit was built on top of node.js and webkit, hence the name. However it doesn't use any of them anymore: now it's based on io.js, the node.js' fork featuring Harmony (ECMA-Script 6), and Chromium. So they decided to rename it nw.js to avoid confusion.

The governance of nw.js looks more open than it used to be, which is a very good news. On the other hand Github is THE reference of open source collaboration: so we can expect a lot from the atom-shell's community in the future.

Technically, both projects are awesome.

It is not clear which one is better.

But I tend to prefer Atom-shell because its paradigm is more natural to me, particularly those points:

• The entry point of the application is a Javascript file, not an HTML file: in my opinion this is cleaner

• We can roughly state that it is the back-end that controle the entire application and I like that (I can be biased here, since I'm a back-end developer)

• Consequently the back-end can create as many windows as it is needed... If I'm not wrong, nw.js has a main window that cannot be closed without closing the whole application, and new windows should be created by this main window. I found that a bit hacky...

• The way node.js and Chromium are integrated together is easier and the KISS principle is something I care a lot in computer science... See the details of the integration here.

• Multi-context, see the Atom-shell documentation:

  If you are an experienced Node-Webkit user, you should be familiar with the concept of Node context and web context, these concepts were invented because of how the Node-Webkit was implemented.

  By using the multi-context feature of Node, atom-shell doesn't introduce a new JavaScript context in web pages.

• I found the API more cleaner and straightforward, typically the back-end side (named browser-side in the doc) controle a browser, can create windows, close them, define how they look, it can make it interact with the user's desktop... Everything look less hacky and natural. With nw.js it's like a webpage having eventually access to node.js' stuff, with Atom-shell we code like we will do in any other technologies (like for example the C++/GTK combo): it's more like coding a node.js application that can create windows with webpage. I prefer that coding paradigm, the other way is like looking upside-down.

For all those reasons, despite the popularity of nw.js, I decided to go with Atom-shell, after experimenting both of them few hours. There is no large application using Atom-shell at the moment, except Atom itself, but I bet on it...

Atom-shell: the browser-side and the renderer-side concept

The key to understand Atom-shell is the concept of browser-side (our good ol' back-end/server side) and renderer-side (also called web page, our good ol' front-end/client side).

The first deals with the system and controle the application globally, the second renders a web application in the window created by the first.

Both of them have access to node.js nonetheless, however it is possible to forbid that for a web page, for security reason, if our application is designed to browse some arbitrary website on the internet, rather than run local web content.

For browser to client interaction, we have to use either the IPC module (Inter-Process Communication) which is asynchronous, or the remote module for synchronous RPC (Remote Procedure Call).

The IPC module is familiar to any web developer: it's more or less the same paradigm as the client/server one. Asynchronous content is sent back and forth.

The remote module is easier, we can typically call any method of the browser-side from the renderer-side, just like if both side was the same entity... but be careful: there are some drawbacks!

Let's get our hands dirty!!!

After a week with Atom-shell, playing and coding seriously, there are few things worth mentioning here.

The remote module is really handy, however the only way to gain access to an object of the browser-side is to use remote.getGlobal(name) which bind a global browser-side variable to its renderer-side counterpart. I don't like to pollute the global scope just to provide some objects to the renderer's remote module...

I would prefer passing those objects at the BrowserWindow's instanciation, or as an extra parameter of .loadUrl().

I have created an issue for that, if you like the idea, please contribute to the discussion.

Also there is the problem of callbacks passed to the browser, see here for details.

While I understand what happen and how to avoid it, this forbid the usage of my lovely node.js' EventEmitter. I came up with this kind of design, before reading why it should be avoided:

// Renderer-side code:
var bridge = remote.getGlobal( 'bridge' ) ;  
bridge.on( 'update' , function() {  
    // do something
} ) ;

That kind of design is really neat, but forbidden... I wonder if it's worth trying to fix the potential exceptions issues with some boilerplated try-catch block, or if it's a definitive no-go...

Trouble with jQuery and workaround

In Atom-shell, we cannot include jQuery like we do usually, i.e. putting <script src="../js/jquery.js"></script> into the main .html file of the renderer-side.

At first glance, it was a bit frustrating, since the renderer/web page side claims to behave like any normal browser. However, it is not an Atom-shell issue. Actually jQuery detect a module and a module.exports global, and therefore expect a CommonJS context, thus decide to not install itself into window.$ as usual.

After a bit of googling, the workaround is easy, we have to require jQuery in the main .js file of the web page, this way:
window.$ = require( '../js/jquery.js' ) ;

For the record, here is the related issue on Github, giving all the details.

Conclusions?

I cannot say I'm an experienced Atom-shell developer at the moment. There are many things to explore, and a great app to build.

But I can say I enjoy it: it just does the job, and that's what we like. There is no fuss, I didn't find anything opinionated, or anything.

Just node.js and Chromium, together.

... to be continued...

Do not miss the other post on Atom-Shell / Electron:

• Building a Package Featuring Electron as a Stand-Alone Application

io.js is born!

According to the website:

Bringing ES6 to the Node Community!

io.js is an npm compatible platform originally based on node.js™.

Also, from the baseline of the io.js repository:

“A spork of Node.js with an open governance model”.

All that sounds very promising!

Things was moving really slowly lately, but now I'm sure that development will speed up.

Among the team members there are some of the finest Node.js superstars, including Ben Noordhuis (also famous for libuv), Isaac Schlueter (also famous for npm) and Fedor Indutny.

All the team members are here.

Some people are feeling concerned, that it will induce fragmentation in the community. IMHO there are wrong.

Firstly, io.js and Node.js will continue to share the same package registry. So fragmentation will only affect core development, not modules you install using npm.

Secondly, most of core developers have moved to io.js. That's some kind of signs. Something is wrong with the Joyent's governance. Without the drag of that corporation, devs gonna dev, and that's a good news.

We don't want a slow and ill governance like the PHP one, those issues should be settled as soon as possible.

So, I see only two options:

• the happy end: reconciliation between Node.js & io.js will happen soon, the two projects will merge, dissidents will gain the power to put the governance model on the right track
• the putsch: let's face it, io.js is better than Node.js in any aspect and the governance is open... so it will simply overtake Node.js, and everyone will make the switch very quickly (just like the switch from MySQL to MariaDB)

In any case, no fragmentation will occur... and server-side Javascript will enjoy a good nitro-boost.

Harmony

Harmony is the upgrade Javascript needed. It implements Ecma-Script 6 and it really moves the language forward.

Keyed collections (Map, Set, WeakMap, WeakSet) are very very interesting things.

On my previous post, detecting acyclic/cyclic graph in object data structure was done using an array of object, searching multiple references of the same object using .indexOf(). No doubt about it, on big graph this can easily become the bottleneck. Using a WeakMap could be so much faster: the object to check is the key.

Speaking of algorithmic complexity, searching a value is a O(n). Searching a key should be O(1) as far as I can tell (if it is a hashmap), or at least O(log n) if WeakMap use some kind of binary tree. Way faster...

Some cool new features in io.js:

• default parameters
• Promise
• let, const
• generators
• template strings

Cool Harmony features that are not in io.js at the moment:

• Proxy
• Spread operator

Performances

No benchmark at the moment, but I ran the spaceship demo of the terminal-kit lib on my laptop... The script consumes 12-14% of CPU with Node.js, and consumes only 5-7% of CPU using io.js.

Incredible!

This is the spaceship demo!

I don't know exactly the reason of such spectacular improvements, and the very nature of the demo should not be considered as a common use case.

When I will get some time, I will test performances more seriously...

So what's up now?

You should have downloaded dat io.js already!

Test it! Hack it! Enjoy it!

• Object cloning part. I
• Object cloning part. II

Shallow copy vs deep copy

A shallow copy will clone the top-level object, but nested object are shared between the original and the clone. That's it: an object reference is copied, not cloned. So if the original object contains nested object, then the clone will not be a distinct entity.

A deep copy will recursively clone every objects it encounters. The clone and the original object will not share anything, so the clone will be a fully distinct entity.

Shallow copies are faster than deep copies.

When it is ok to share some data, you may use shallow copy. There are even use case where it is the best way to do the job. But whenever you need to clone a deep and complex data structure, a tree, you will have to perform deep copy. Have in mind that on really big tree, it can be an expensive operation.

How to perform a deep copy of an object in Javascript

Okey, so let's modify the shallowCopy() function of the previous article.

We need to detect properties containing objects, and recursively call the deepCopy() function again.

Here is the result:

function naiveDeepCopy( original )  
{
    // First create an empty object with
    // same prototype of our original source
    var clone = Object.create( Object.getPrototypeOf( original ) ) ;

    var i , descriptor , keys = Object.getOwnPropertyNames( original ) ;

    for ( i = 0 ; i < keys.length ; i ++ )
    {
        // Save the source's descriptor
        descriptor = Object.getOwnPropertyDescriptor( original , keys[ i ] ) ;

        if ( descriptor.value && typeof descriptor.value === 'object' )        
        {
            // If the value is an object, recursively deepCopy() it
            descriptor.value = naiveDeepCopy( descriptor.value ) ;
        }

        Object.defineProperty( clone , keys[ i ] , descriptor ) ;
    }

    return clone ;
}

By the way, if the property is a getter/setter, then descriptor.value will be undefined, so we won't perform recursion on them, and that's what we want. We actually don't care if the getter return an object or not.

There are still unsolved issues:

• Circular references will produce a stack overflow
• Some native objects like Date or Array do not work properly
• Design pattern emulating private members using a closure's scope cannot be truly cloned (e.g. the revealing pattern)

What is this circular reference thing?

Let's look at that object:

var o = {  
    a: 'a',
    sub: {
        b: 'b'
    },
    sub2: {
        c: 'c'
    }
} ;

o.loop = o ;  
o.sub.loop = o ;  
o.subcopy = o.sub ;  
o.sub.link = o.sub2 ;  
o.sub2.link = o.sub ;  

This object self-references itself.

That means that o.loop.a = 'Ha! implies that console.log( o.a ) outputs "Ha!" rather than "a". You remember how object assignment works? o and o.loop simply point to the same object.

However, the naiveDeepCopy() method above does not check that fact and therefore is doomed, iterating o.loop.loop.loop.loop.loop... forever.

That what is called a circular reference.

Even without the loop property, it happens that the original object want that the subcopy and sub properties point to the same object. Here, again, the naiveDeepCopy() method would produce two differents and independent clone.

A good clone method should be able to overcome that.

The closure's scope hell

Okey, let's examine this code:

function myConstructor()  
{
    var myPrivateVar = 'secret' ;

    return {
        myPublicVar: 'public!' ,
        getMyPrivateVar: function() {
            return myPrivateVar ;
        } ,
        setMyPrivateVar( value ) {
            myPrivateVar = value.toString() ;
        }
    } ;
}

var o = myContructor() ;  

So... o is an object containing three properties, the first is a string, the two others are methods.

The methods are currently using a variable of the parent scope, in the scope of myConstructor(). That variable (named myPrivateVariable) is created when the constructor is called, however while it is not part of the contructed object in any way, it still remains used by those methods.

Therefore, if we try to clone the object, methods of both the original and the clone will still refer to the same parent's scope variable.

It would not be a problem if this was not a common Javascript's pattern to simulate private members...

As far as I know, there is no way to alter the scope of a closure, so this is a dead-end: pattern using the parent scope cannot be cloned correctly.

Next step: using a library

Okey, so far, we have done a good job hacking Javascript, and it was fun.

Now how about using a ready to use library?

The tree-kit library has a great clone() method, that works in most use case.

It happens that I'm actually the author of this lib, probably some kind of coincidence! ;)

clone( original , [circular] )

• original Object the source object to clone
• circular boolean (default to false) if true then circular references are checked and each identical objects are reconnected (referenced), if false then nested object are blindly cloned

It returns a clone of the original object.

How to use it? That's pretty straightforward:

• first run the command npm install tree-kit --save into your project directory
• then use it like this:

var tree = require( 'tree-kit' ) ;  
var myClone = tree.clone( myOriginal ) ;  

... where myOriginal is the object you want to clone.

Some optimization work have been done, so tree.clone() should be able to clone large structure efficiently.

One big step in optimization: removing recursivity in the algorithm – it's all taking place in a loop. It avoids stack-overflow and function's call overhead. As a side-effect, depth-first search has been replaced by a breadth-first search algorithm.

Great news: this method is able to detect circular references and reconnect them if the circular option is set to true! Oooh yeah!

If you are interested, you can visit the related source code on github.

If you are that kind of lazy guy, here is the code as of version 0.4.1 (MIT license):

exports.clone = function clone( originalObject , circular )  
{
    // First create an empty object with
    // same prototype of our original source

    var propertyIndex ,
        descriptor ,
        keys ,
        current ,
        nextSource ,
        indexOf ,
        copies = [ {
            source: originalObject ,
            target: Object.create( Object.getPrototypeOf( originalObject ) )
        } ] ,
        cloneObject = copies[ 0 ].target ,
        sourceReferences = [ originalObject ] ,
        targetReferences = [ cloneObject ] ;

    // First in, first out
    while ( current = copies.shift() )
    {
        keys = Object.getOwnPropertyNames( current.source ) ;

        for ( propertyIndex = 0 ; propertyIndex < keys.length ; propertyIndex ++ )
        {
            // Save the source's descriptor
            descriptor = Object.getOwnPropertyDescriptor( current.source , keys[ propertyIndex ] ) ;

            if ( ! descriptor.value || typeof descriptor.value !== 'object' )
            {
                Object.defineProperty( current.target , keys[ propertyIndex ] , descriptor ) ;
                continue ;
            }

            nextSource = descriptor.value ;
            descriptor.value = Array.isArray( nextSource ) ?
                [] :
                Object.create( Object.getPrototypeOf( nextSource ) ) ;

            if ( circular )
            {
                indexOf = sourceReferences.indexOf( nextSource ) ;

                if ( indexOf !== -1 )
                {
                    // The source is already referenced, just assign reference
                    descriptor.value = targetReferences[ indexOf ] ;
                    Object.defineProperty( current.target , keys[ propertyIndex ] , descriptor ) ;
                    continue ;
                }

                sourceReferences.push( nextSource ) ;
                targetReferences.push( descriptor.value ) ;
            }

            Object.defineProperty( current.target , keys[ propertyIndex ] , descriptor ) ;

            copies.push( { source: nextSource , target: descriptor.value } ) ;
        }
    }

    return cloneObject ;
} ;

Happy hacking!

• Object cloning part. I
• Object cloning part. II

Prerequisite: Understanding objects assignment in Javascript

As you know, the assignment does not copy an object, it only assign a reference to it, therefore the following code:

var object = { a: 1, b: 2 } ;  
var copy = object ;  
object.a = 3 ;  
console.log( copy.a ) ;  

... will output 3 rather than 1.

The two variables object & copy reference the same object, so whatever the variable used to modify it, you will get the same result.

If you come from a C/C++ background, you should understand that object.a in Javascript should be translated into object->a in C/C++, it will help understand how copy = object works.

When it comes to object, a Javascript variable behaves more like a kind of automatic pointer.

Also there is a misleading saying commonly used in javascript, one may say that “Object are passed as reference”.

That's totally wrong.

If it was true, then the following code:

var object = { a: 1, b: 2 } ;

function fn( ob )  
{
    ob = { c: 3, d: 4 } ;
}

fn( object ) ;  
console.log( object ) ;  

... would output { c: 3, d: 4 }, but actually object still reference { a: 1, b: 2 }.

So what happened really at function call?

Nothing unusual, each caller's argument are assigned to a callee's argument just like it would if you had manually used the = operator. There are no special case for object.

When you pass a variable by reference in a language that supports this pass by reference feature, the caller & callee variable are identical, as if they were each others aliases, so mutating one mutates the other.

Here in Javascript, we have two distinct variables, that happen to point to the same object... ... ... until re-assignment happens.

That's why I prefer to say that a variable, after an object assignment, behaves like a pointer to that object. In a C/C++ fashion, object = { a: 1, b: 2 } should be understood as object = &( { a: 1, b: 2 } ).

How to perform a shallow copy of an object in Javascript

Javascript does not have built-in object-cloning facilities.

A quick and dirty way to clone an object would be to create a new empty object, then iterate over the original to copy properties one by one.

This naive function will do the trick:

function naiveShallowCopy( original )  
{
    // First create an empty object
    // that will receive copies of properties
    var clone = {} ;

    var key ;

    for ( key in original )
    {
        // copy each property into the clone
        clone[ key ] = original[ key ] ;
    }

    return clone ;
}

However, there are few issues with this code:

1. The clone produced doesn't have the same prototype than the original, it is simply an instance of Object... the prototype of the clone is not the same than the prototype of the original.

2. However, inherited properties of the original (inherited from its prototype) are copied into the clone as regular owned properties.

3. Only enumerable properties are copied.

4. Properties' descriptor are not copied, e.g. a read-only property in the original will be writable in the clone.

5. Finally: if a property is an object, then it will be shared between the clone and the original, their respective properties will point to the same object.

The 5th point is what make it a shallow copy: only the surface of the object is cloned, deeper objects are shared.

A variant using Object.keys() can be used if we want to copy only owned and enumerable properties:

function shallowCopyOfEnumerableOwnProperties( original )  
{
    // First create an empty object
    // that will receive copies of properties
    var clone = {} ;

    var i , keys = Object.keys( original ) ;

    for ( i = 0 ; i < keys.length ; i ++ )
    {
        // copy each property into the clone
        clone[ keys[ i ] ] = original[ keys[ i ] ] ;
    }

    return clone ;
}

If you want to copy non-enumerable properties as well, you can replace Object.keys() with Object.getOwnPropertyNames():

function shallowCopyOfOwnProperties( original )  
{
    // First create an empty object
    // that will receive copies of properties
    var clone = {} ;

    var i , keys = Object.getOwnPropertyNames( original ) ;

    for ( i = 0 ; i < keys.length ; i ++ )
    {
        // copy each property into the clone
        clone[ keys[ i ] ] = original[ keys[ i ] ] ;
    }

    return clone ;
}

Still, non-enumerable properties will be enumerable properties in the clone...

We can improve this function using Object.getOwnPropertyDescriptor() & Object.defineProperty(), so descriptors will be cloned properly.

And finally, if we create the clone with Object.create() and use the result of Object.getPrototypeOf( original ) as its argument, we can ensure that the clone will have the correct prototype.

function shallowCopy( original )  
{
    // First create an empty object with
    // same prototype of our original source
    var clone = Object.create( Object.getPrototypeOf( original ) ) ;

    var i , keys = Object.getOwnPropertyNames( original ) ;

    for ( i = 0 ; i < keys.length ; i ++ )
    {
        // copy each property into the clone
        Object.defineProperty( clone , keys[ i ] ,
            Object.getOwnPropertyDescriptor( original , keys[ i ] )
        ) ;
    }

    return clone ;
}

Okey, this is far better.

Next time we will go further, we will see how to perform deep copy, and inspect issues that cannot be overcome easily.