JavaScript with Promises

Chapter 1. Asynchronous JavaScript

The number of asynchronous JavaScript APIs is rapidly growing. Web applications asynchronously fetch data and load scripts in the browser. Node.js and its derivatives provide a host of APIs for asynchronous I/O. And new web specifications for Streams, Service Workers, and Font Loading all include asynchronous calls. These advancements broaden the capabilities of JavaScript applications, but using them without understanding how the async part works can result in unpredictable code that is difficult to maintain. Things may work as expected in development or test environments but fail when deployed to end users because of variables such as network speed or hardware performance.

This chapter explains how async JavaScript works. We’ll cover callbacks, the event loop, and threading. Most of the information is not specific to Promises but provides the foundation you need to get the most out of Promises and out of the rest of this book.

Let’s start with a code snippet that frequently surprises people. The code makes an HTTP request using the XMLHttpRequest (XHR) object and uses a while loop that runs for three seconds. Although it is generally bad practice to implement a delay with the while loop, it’s a good way to illustrate how JavaScript runs. Read the code in Example 1-1 and decide whether the listener callback for the XHR object will ever be triggered.

Example 1-1. Async XHR

// Make an async HTTP request
var async = true;
var xhr = new XMLHttpRequest();
xhr.open('get', 'data.json', async);
xhr.send();

// Create a three second delay (don't do this in real life)
var timestamp = Date.now() + 3000;
while (Date.now() < timestamp);

// Now that three seconds have passed,
// add a listener to the xhr.load and xhr.error events
function listener() {
    console.log('greetings from listener');
}
xhr.addEventListener('load', listener);
xhr.addEventListener('error', listener);

Here are some common opinions on whether listener is called:

1. Yes, listener is always called

2. Not a chance, the addEventListener calls must run before xhr.send()

3. Sometimes, depending on whether the request takes more than three seconds

The correct assessment is that listener is always called. Although the second and third answers are common, they are incorrect because of the event loop model and run-to-completion semantics in JavaScript. If you thought otherwise or would like a refresher on these concepts, this chapter is for you.

Callbacks

Callbacks are the cornerstone of asynchronous JavaScript programming. As a JavaScript developer you are probably familiar with callbacks, but just to be sure, Example 1-2 presents a quick case of a callback that prints each of the elements in an array.

Example 1-2. Example callback

var cities = ['Tokyo', 'London', 'Boston', 'Berlin', 'Chicago', 'New York'];

cities.forEach(function callback(city) {
    console.log(city);
});

// Console output:
// Tokyo
// London
// Boston
// Berlin
// Chicago
// New York

In short, a callback is a function provided to other code for invocation. Example 1-2 uses an inline function to define the callback. That is a commonly used style in JavaScript applications, but callbacks do not have to be declared inline. Example 1-3 shows the equivalent code with the function declared in advance.

Example 1-3. Passing a callback as a predefined function

function callback(city) {
    console.log(city);
}

cities.forEach(callback);

Whether your callbacks are inline functions or predefined is a matter of choice. As long as you have a reference to a function, you can use it as a callback.

Asynchronous JavaScript

Callbacks can be invoked synchronously or asynchronously (i.e., before or after the function they are passed to returns.) The array.forEach() method used in the previous section invokes the callback it receives synchronously. An example of a function that invokes its callback asynchronously is window.requestAnimationFrame(). Its callback is invoked between browser repaint intervals, as shown in Example 1-4.

Example 1-4. A callback being invoked asynchronously

function repositionElement() {
    console.log('repositioning!');
    // ...
}

window.requestAnimationFrame(repositionElement);
console.log('I am the last line of the script');

// Console output:
// I am the last line of the script
// repositioning!

In this example, “I am the last line of the script” is written to the console before “repositioning!” because requestAnimationFrame returns immediately and invokes the repositionElement callback at a later time.

Synchronous code can be easier to understand because it executes in the order it is written. A good comparison can be made using the synchronous and asynchronous file APIs in Node.js. Example 1-5 is a script that writes to a file and reads back the contents synchronously. The numbered comments indicate the relative order in which some of the lines of code are executed.

Example 1-5. Using synchronous code to write and read a file in Node.js

var fs = require('fs');
var timestamp = new Date().toString();
var contents;

fs.writeFileSync('date.txt', timestamp);
contents = fs.readFileSync('date.txt');
console.log('Checking the contents');            // 1
console.assert(contents == timestamp);           // 2

console.log('I am the last line of the script'); // 3

// Console output:
// Checking the contents
// I am the last line of the script

The script uses the writeFileSync and readFileSync functions of the fs module to write a timestamp to a file and read it back. After the contents of the file are read back, they are compared to the timestamp that was originally written to see if the two values match. The console.assert() displays an error if the values differ. In this example they always match so the only output is from the console.log() statements before and after the assertion.

The script shown in Example 1-6 does the same job using the async functions fs.writeFile() and fs.readFile(). Both functions take a callback as their last parameter. The numbered comments are used again to show the relative execution order, which differs from the previous script.

Example 1-6. Using asynchronous code to write and read a file in Node.js

var fs = require('fs');
var timestamp = new Date().toString();

fs.writeFile('date.txt', timestamp, function (err) {
    if (err) throw err;

    fs.readFile('date.txt', function (err, contents) {
        if (err) throw err;
        console.log('Checking the contents');          // 2
        console.assert(contents == timestamp);         // 3
    });
});

console.log('I am the last line of the script');       // 1

// Console output:
// I am the last line of the script
// Checking the contents

Comparing this code to the previous example, you’ll see that the console output appears in reverse order. Similar to the requestAnimationFrame example, the call to fs.writeFile() returns immediately so the last line of the script runs before the file contents are read and compared to what was written.

Although synchronous code can be easier to follow, it is also limiting. Programmers need the ability to write async code so long-running tasks such as network requests do not block other parts of the program while incomplete. Without that ability, you couldn’t type in an editor at the same time your document was being autosaved or scroll through a web page while the browser was still downloading images. This is where callbacks come in. In JavaScript, callbacks are used to manage the execution order of any code that depends on an async task.

When programmers are new to asynchronous programming, it’s easy for them to incorrectly expect an async script to run as if it were synchronous. Putting code that relies on the completion of an async task outside the appropriate callback creates problems. Example 1-7 shows some code that expects the callback given to readFile to be invoked before readFile returns, but when that doesn’t happen the content comparison fails.

Example 1-7. Naive asynchronous code. This doesn’t work!

var fs = require('fs');
var timestamp = new Date().toString();
var contents;

fs.writeFile('date.txt', timestamp);

fs.readFile('date.txt', function (err, data) {
    if (err) throw err;
    contents = data;                           // 3
});

console.log('Comparing the contents');         // 1
console.assert(timestamp == contents);         // 2 - FAIL!

Suppose the file only took a fraction of a millisecond to read. Does the example contain a race condition where the contents of the file are always ready for comparison when you test the code on your machine but fail every time you demo the application? The answer is that there isn’t a race condition because the callback to readFile is always invoked asynchronously, so readFile is guaranteed to return before invoking the callback. Once that happens, the callback never runs before the log or assert statements on the next two lines because of the run-to-completion semantics explained in the next section. But before we get to that, a word of caution about writing functions that accept callbacks.

When you pass a callback to a function it’s important to know whether the callback will be invoked synchronously or asynchronously. You don’t want a series of steps that build on one another to run out of order. This is generally straightforward to determine because the function’s implementation, documentation, and purpose indicate how your callback is handled. However, a function can have mixed behavior where it invokes a callback synchronously or asynchronously depending on some condition. Example 1-8 shows the jQuery ready function used to run code after the Document Object Model (DOM) is ready. If the DOM has finished loading before ready is invoked, the callback is invoked synchronously. Otherwise the callback is invoked once the DOM has loaded.

Example 1-8. The jQuery ready function can be synchronous or asynchronous

jQuery(document).ready(function () {
    // jQuery calls this function after the DOM is loaded and ready to use
    console.log('DOM is ready');
});

console.log('I am the last line of the script');

// Console output may appear in either order depending on when the DOM is ready

Functions that are not consistently synchronous or asynchronous create a fork in the execution path. The jQuery ready function creates a fork with two paths. If a function containing the same style of mixed behavior invoked ready, there would be four possible paths. The explosion in execution branches makes explaining and testing this approach difficult, and reliable behavior in a production environment more challenging. Isaac Schlueter has written a popular blog post about this titled “Designing APIs for Asynchrony,” in which he refers to the inconsistent behavior as “releasing Zalgo.”

Warning

Functions that invoke a callback synchronously in some cases and asynchronously in others create forks in the execution path that make your code less predictable.

Run to Completion and the Event Loop

The JavaScript you write runs on a single thread, which avoids complications found in other languages that share memory between threads. But if JavaScript is single-threaded, where are the async tasks and callbacks run? To explain, let’s start in Example 1-9 with a simple HTTP request in Node.

Example 1-9. HTTP request in Node.js

var http = require('http');
http.get('http://www.google.com', function (res) {
    console.log('got a response');
});

The call to http.get() triggers a network request that a separate thread handles. But wait—you were just told that JavaScript is single-threaded. Here’s the distinction: the JavaScript code you write all runs on a single thread, but the code that implements the async tasks (the http.get() implementation in Example 1-9) is not part of that JavaScript and is free to run in a separate thread.

Once the task completes the result needs to be provided to the JavaScript thread. At this point the callback is placed in a queue. A multithreaded language might interrupt whatever code was currently executing to provide the results, but in JavaScript these interruptions are forbidden. Instead there is a run-to-completion rule, which means that your code runs without interruption until it passes control back to the host environment by returning from the function that the host initially called. At that point the callback can be removed from the queue and invoked.

All other threads communicate with your code by placing items on the queue. They are not permitted to manipulate any other memory accessible to JavaScript. In the previous example the callback accesses the response from the async HTTP request.

After the callback is added to the queue, there is no guarantee how long it will have to wait. How long it takes the current code to run to completion and what else is in the queue controls the time. The queue can contain things such as mouse clicks, keystrokes, and callbacks for other async tasks. The JavaScript runtime simply continues in an endless cycle of pulling an item off the queue if one is available, running the code that the item triggers, and then checking the queue again. This cycle is known as the event loop.

Figure 1-1 shows how the queue is populated and Figure 1-2 shows how the event loop processes items from the queue. All the JavaScript you write executes in the box labeled Run JS Event Handler in Figure 1-2. The JavaScript engine performs the rest of the activity in both diagrams behind the scenes.

Figure 1-1. Filling the queue

Figure 1-2. The JavaScript event loop

Using setTimeout to trigger another function after a given amount of time is a simple way to watch the event loop in action, as shown in Example 1-10. The setTimeout function accepts two arguments: a function to call and the minimum number of milliseconds to wait before calling the function.

Example 1-10. Using setTimeout to demonstrate the event loop

function marco() {
    console.log('polo');
}

setTimeout(marco, 0); // zero delay
console.log('Ready when you are');

// Console output:
// Ready when you are
// polo

The marco function is immediately placed in the queue. After the console displays “Ready when you are,” the event loop turns and marco can be pulled off the queue. Notice the second parameter for setTimeout specifies the minimum amount of time that will lapse before the callback is run as opposed to the exact amount of time. It is impossible to know exactly when the callback will run because other JavaScript could be executing at that time and the machine has to let that finish before returning to the queue to invoke your callback.

Keeping in mind the run-to-completion and event loop concepts, let’s revisit the XHR example given at the beginning of the chapter, which is repeated in Example 1-11 for convenience.

Example 1-11. Async XHR (repeated from earlier)

// Make an async HTTP request
var async = true;
var xhr = new XMLHttpRequest();
xhr.open('get', 'data.json', async);
xhr.send();

// Create a three second delay (don't do this in real life)
var timestamp = Date.now() + 3000;
while (Date.now() < timestamp);

// Now that three seconds have passed,
// add a listener to the xhr.load and xhr.error events
function listener() {
    console.log('greetings from listener');
}
xhr.addEventListener('load', listener);
xhr.addEventListener('error', listener);

The question was whether the listener function will ever be triggered. The code plays out similarly to the previous example with setTimeout. The listeners are registered after invoking the send function, but this is safe to do until the event loop turns because the runtime cannot trigger the load or error events before then.

Allowing the event loop to turn before registering the event listeners would create a race condition. Example 1-12 demonstrates that by using setTimeout.

Example 1-12. Race condition

var async = true;
var xhr = new XMLHttpRequest();
xhr.open('get', 'data.json', async);
xhr.send();

setTimeout(function delayed() { // Creates race condition!
    function listener() {
        console.log('greetings from listener');
    }
    xhr.addEventListener('load', listener);
    xhr.addEventListener('error', listener);
}, 3000);

Performing the event listener registration inside a callback given to setTimeout causes a delay. Now the only way the listener function will be called is if the delayed function is pulled off the queue and run before the HTTP request completes and the load or error event is triggered. Experimenting with different values for the delay parameter of setTimeout shows listener being invoked sometimes but not always.

Summary

This chapter covered the underlying concepts of asynchronous JavaScript programming. Knowing how JavaScript handles callbacks allows you to control the order in which your code runs instead of writing things that work by coincidence. If the order in which your code is executed surprises you or you find yourself unsure of what will happen next, refer back to this chapter. Not only does it prepare you for using Promises, but it will make you a better JavaScript developer overall.