Node.js allows you to create apps fast and easily. But due to its asynchronous nature, it may be hard to write readable and manageable code. In this article I’ll show you a few tips on how to achieve that.

Callback Hell or the Pyramid of Doom

Node.js is built in a way that forces you to use asynchronous functions. That means callbacks, callbacks and even more callbacks. You’ve probably seen or even written yourself pieces of code like this:

app.get('/login', function (req, res) {
    	sql.query('SELECT 1 FROM users WHERE name = ?;', [ req.param('username') ], function (error, rows) {
    		if (error) {
    			res.writeHead(500);
    			return res.end();
    		}
    		if (rows.length < 1) {
    			res.end('Wrong username!');
    		} else {
    			sql.query('SELECT 1 FROM users WHERE name = ? && password = MD5(?);', [ req.param('username'), req.param('password') ], function (error, rows) {
    				if (error) {
    					res.writeHead(500);
    					return res.end();
    				}
    				if (rows.length < 1) {
    					res.end('Wrong password!');
    				} else {
    					sql.query('SELECT * FROM userdata WHERE name = ?;', [ req.param('username') ], function (error, rows) {
    						if (error) {
    							res.writeHead(500);
    							return res.end();
    						}
    						req.session.username = req.param('username');
    						req.session.data = rows[0];
    						res.rediect('/userarea');
    					});
    				}
    			});
    		}
    	});
    });
    

This is actually a snippet straight from one of my first Node.js apps. If you’ve done something more advanced in Node.js you probably understand everything, but the problem here is that the code is moving to the right every time you use some asynchronous function. It becomes harder to read and harder to debug. Luckily, there are a few solutions for this mess, so you can pick the right one for your project.

Solution 1: Callback Naming and Modularization

The simplest approach would be to name every callback (which will help you debug the code) and split all of your code into modules. The login example above can be turned into a module in a few simple steps.

The Structure

Let’s start with a simple module structure. To avoid the above situation, when you just split the mess into smaller messes, let’s have it be a class:

var util = require('util');
    
    function Login(username, password) {
    	function _checkForErrors(error, rows, reason) {
    		
    	}
    	
    	function _checkUsername(error, rows) {
    		
    	}
    	
    	function _checkPassword(error, rows) {
    		
    	}
    	
    	function _getData(error, rows) {
    		
    	}
    	
    	function perform() {
    		
    	}
    	
    	this.perform = perform;
    }
    
    util.inherits(Login, EventEmitter);
    

The class is constructed with two parameters: username and password. Looking at the sample code, we need three functions: one to check if the username is correct (_checkUsername), another to check the password (_checkPassword) and one more to return the user-related data (_getData) and notify the app that the login was successful. There is also a _checkForErrors helper, which will handle all errors. Finally, there is a perform function, which will start the login procedure (and is the only public function in the class). Finally, we inherit from EventEmitter to simplify the usage of this class.

The Helper

The _checkForErrors function will check if any error occurred or if the SQL query returns no rows, and emit the appropriate error (with the reason that was supplied):

function _checkForErrors(error, rows, reason) {
    	if (error) {
    		this.emit('error', error);
    		return true;
    	}
    	
    	if (rows.length < 1) {
    		this.emit('failure', reason);
    		return true;
    	}
    	
    	return false;
    }
    

It also returns true or false, depending on whether an error occurred or not.

Performing the Login

The perform function will have to do only one operation: perform the first SQL query (to check if the username exists) and assign the appropriate callback:

function perform() {
    	sql.query('SELECT 1 FROM users WHERE name = ?;', [ username ], _checkUsername);
    }
    

I assume you have your SQL connection accessible globally in the sql variable (just to simplify, discussing if this is a good practice is beyond the scope of this article). And that’s it for this function.

Checking the Username

The next step is to check if the username is correct, and if so fire the second query – to check the password:

function _checkUsername(error, rows) {
    	if (_checkForErrors(error, rows, 'username')) {
    		return false;
    	} else {
    		sql.query('SELECT 1 FROM users WHERE name = ? && password = MD5(?);', [ username, password ], _checkPassword);
    	}
    }
    

Pretty much the same code as in the messy sample, with the exception of error handling.

Checking the Password

This function is almost exactly the same as the previous one, the only difference being the query called:

function _checkPassword(error, rows) {
    	if (_checkForErrors(error, rows, 'password')) {
    		return false;
    	} else {
    		sql.query('SELECT * FROM userdata WHERE name = ?;', [ username ], _getData);
    	}
    }
    

Getting the User-Related Data

The last function in this class will get the data related to the user (the optional step) and fire a success event with it:

function _getData(error, rows) {
    	if (_checkForErrors(error, rows)) {
    		return false;
    	} else {
    		this.emit('success', rows[0]);
    	}
    }
    

Final Touches and Usage

The last thing to do is to export the class. Add this line after all of the code:

module.exports = Login;
    

This will make the Login class the only thing that the module will export. It can be later used like this (assuming that you’ve named the module file login.js and it’s in the same directory as the main script):

var Login = require('./login.js');
    
    ...
    
    app.get('/login', function (req, res) {
    	var login = new Login(req.param('username'), req.param('password));
    	login.on('error', function (error) {
    		res.writeHead(500);
    		res.end();
    	});
    	login.on('failure', function (reason) {
    		if (reason == 'username') {
    			res.end('Wrong username!');
    		} else if (reason == 'password') {
    			res.end('Wrong password!');
    		}
    	});
    	login.on('success', function (data) {
    		req.session.username = req.param('username');
    		req.session.data = data;
    		res.redirect('/userarea');
    	});
    	login.perform();
    });
    

Here’s a few more lines of code, but the readability of the code has increased, quite noticeably. Also, this solution does not use any external libraries, which makes it perfect if someone new comes to your project.

That was the first approach, let’s proceed to the second one.

Solution 2: Promises

Using promises is another way of solving this problem. A promise (as you can read in the link provided) “represents the eventual value returned from the single completion of an operation”. In practice, it means that you can chain the calls to flatten the pyramid and make the code easier to read.

We will use the Q module, available in the NPM repository.

Q in the Nutshell

Before we start, let me introduce you to the Q. For static classes (modules), we will primarily use the Q.nfcall function. It helps us in the conversion of every function following the Node.js’s callback pattern (where the parameters of the callback are the error and the result) to a promise. It’s used like this:

Q.nfcall(http.get, options);
    

It’s pretty much like Object.prototype.call. You can also use the Q.nfapply which resembles Object.prototype.apply:

Q.nfapply(fs.readFile, [ 'filename.txt', 'utf-8' ]);
    

Also, when we create the promise, we add each step with the then(stepCallback) method, catch the errors with catch(errorCallback) and finish with done().

In this case, since the sql object is an instance, not a static class, we have to use Q.ninvoke or Q.npost, which are similar to the above. The difference is that we pass the methods’ name as a string in the first argument, and the instance of the class that we want to work with as a second one, to avoid the method being unbinded from the instance.

Preparing the Promise

The first thing to do is to execute the first step, using Q.nfcall or Q.nfapply (use the one that you like more, there is no difference underneath):

var Q = require('q');
    
    ...
    app.get('/login', function (req, res) {
    	Q.ninvoke('query', sql, 'SELECT 1 FROM users WHERE name = ?;', [ req.param('username') ])
    });
    

Notice the lack of a semicolon at the end of the line – the function-calls will be chained so it cannot be there. We are just calling the sql.query as in the messy example, but we omit the callback parameter – it’s handled by the promise.

Checking the Username

Now we can create the callback for the SQL query, it will be almost identical to the one in the “pyramid of doom” example. Add this after the Q.ninvoke call:

.then(function (rows) {
    	if (rows.length < 1) {
    		res.end('Wrong username!');
    	} else {
    		return Q.ninvoke('query', sql, 'SELECT 1 FROM users WHERE name = ? && password = MD5(?);', [ req.param('username'), req.param('password') ]);
    	}
    })
    

As you can see we are attaching the callback (the next step) using the then method. Also, in the callback we omit the error parameter, because we will catch all of the errors later. We are manually checking, if the query returned something, and if so we are returning the next promise to be executed (again, no semicolon because of the chaining).

Checking the Password

As with the modularization example, checking the password is almost identical to checking the username. This should go right after the last then call:

.then(function (rows) {
    	if (rows.length < 1) {
    		res.end('Wrong password!');
    	} else {
    		return Q.ninvoke('query', sql, 'SELECT * FROM userdata WHERE name = ?;', [ req.param('username') ]);
    	}
    })
    

Getting the User-Related Data

The last step will be the one where we’re putting the users’ data in the session. Once more, the callback is not much different from the messy example:

.then(function (rows) {
    	req.session.username = req.param('username');
    	req.session.data = rows[0];
    	res.rediect('/userarea');
    })
    

Checking for Errors

When using promises and the Q library, all of the errors are handled by the callback set using the catch method. Here, we are only sending the HTTP 500 no matter what the error is, like in the examples above:

.catch(function (error) {
    	res.writeHead(500);
    	res.end();
    })
    .done();
    

After that, we must call the done method to “make sure that, if an error doesn’t get handled before the end, it will get rethrown and reported” (from the library’s README). Now our beautifully flattened code should look like this (and behave just like the messy one):

var Q = require('q');
    
    ...
    app.get('/login', function (req, res) {
    	Q.ninvoke('query', sql, 'SELECT 1 FROM users WHERE name = ?;', [ req.param('username') ])
    	.then(function (rows) {
    		if (rows.length < 1) {
    			res.end('Wrong username!');
    		} else {
    			return Q.ninvoke('query', sql, 'SELECT 1 FROM users WHERE name = ? && password = MD5(?);', [ req.param('username'), req.param('password') ]);
    		}
    	})
    	.then(function (rows) {
    		if (rows.length < 1) {
    			res.end('Wrong password!');
    		} else {
    			return Q.ninvoke('query', sql, 'SELECT * FROM userdata WHERE name = ?;', [ req.param('username') ]);
    		}
    	})
    	.then(function (rows) {
    		req.session.username = req.param('username');
    		req.session.data = rows[0];
    		res.rediect('/userarea');
    	})
    	.catch(function (error) {
    		res.writeHead(500);
    		res.end();
    	})
    	.done();
    });
    

The code is much cleaner, and it involved less rewriting than the modularization approach.

Solution 3: Step Library

This solution is similar to the previous one, but it’s simpler. Q is a bit heavy, because it implements the whole promises idea. The Step library is there only for the purpose of flattening the callback hell. It’s also a bit simpler to use, because you just call the only function that is exported from the module, pass all your callbacks as the parameters and use this in place of every callback. So the messy example can be converted into this, using the Step module:

var step = require('step');
    
    ...
    
    app.get('/login', function (req, res) {
    	step(
    		function start() {
    			sql.query('SELECT 1 FROM users WHERE name = ?;', [ req.param('username') ], this);
    		},
    		function checkUsername(error, rows) {
    			if (error) {
    				res.writeHead(500);
    				return res.end();
    			}
    			if (rows.length < 1) {
    				res.end('Wrong username!');
    			} else {
    				sql.query('SELECT 1 FROM users WHERE name = ? && password = MD5(?);', [ req.param('username'), req.param('password') ], this);
    			}
    		},
    		function checkPassword(error, rows) {
    			if (error) {
    				res.writeHead(500);
    				return res.end();
    			}
    			if (rows.length < 1) {
    				res.end('Wrong password!');
    			} else {
    				sql.query('SELECT * FROM userdata WHERE name = ?;', [ req.param('username') ], this);
    			}
    		},
    		function (error, rows) {
    			if (error) {
    				res.writeHead(500);
    				return res.end();
    			}
    			req.session.username = req.param('username');
    			req.session.data = rows[0];
    			res.rediect('/userarea');
    		}
    	);
    });
    

The drawback here is that there is no common error handler. Although any exceptions thrown in one callback are passed to the next one as the first parameter (so the script won’t go down because of the uncaught exception), having one handler for all errors is convenient most of the time.

Which One to Choose?

That’s pretty much a personal choice, but to help you pick the right one, here is a list of pros and cons of each approach:

Modularization:

Pros:

• No external libraries
• Helps to make the code more reusable

Cons:

• More code
• A lot of rewriting if you’re converting an existing project

Promises (Q):

Pros:

• Less code
• Only a little rewriting if applied to an existing project

Cons:

• You have to use an external library
• Requires a bit of learning

Step Library:

Pros:

• Easy to use, no learning required
• Pretty much copy-and-paste if converting an existing project

Cons:

• No common error handler
• A bit harder to indent that step function properly

Conclusion

As you can see, the asynchronous nature of Node.js can be managed and the callback hell can be avoided. I’m personally using the modularization approach, because I like to have my code well structured. I hope these tips will help you to write your code more readable and debug your scripts easier.