Web API Cookbook

Chapter 4. Network Requests

Introduction

You’d have a tough time finding a web application today that doesn’t send any network requests. Since the dawn of Web 2.0 and the novel approach known as Ajax (Asynchronous JavaScript and XML), web apps have been sending asynchronous requests to get new data without reloading the entire page. The XMLHttpRequest API started a new era of interactive JavaScript apps. Despite the name, XMLHttpRequest (or XHR, as it is sometimes known) can also work with JSON and form data payloads.

XMLHttpRequest was a game changer, but the API can be painful to work with. Eventually, third-party libraries such as Axios and jQuery added more streamlined APIs that wrapped the core XHR API.

In 2015, a newer Promise-based API called Fetch became a new standard, and browsers gradually started adding support for it. Today, Fetch is the standard way to make asynchronous requests from your web apps.

This chapter explores XHR and Fetch as well as some other APIs for network communication:

Beacons

A simple one-way POST request ideal for sending analytics data

Server-sent events

A one-way persistent connection with a server to receive real-time events

WebSockets

A two-way persistent connection for bidirectional communication

Sending a Request with XMLHttpRequest

Problem

You want to send a GET request to a public API, and you want to support older browsers that don’t implement the Fetch API.

Solution

Use the XMLHttpRequest API. XMLHttpRequest is an asynchronous, event-based API for making network requests. The general usage of XMLHttpRequest is this:

1. Create a new XMLHttpRequest object.

2. Add a listener for the load event, which receives the response data.

3. Call open on the request, passing the HTTP method and URL.

4. Finally, call send on the request. This triggers the HTTP request to be sent.

Example 4-1 shows a simple example of how to work with JSON data using an XHR.

Example 4-1. Making a GET request with XMLHttpRequest

/**
 * Loads user data from the URL /api/users, then prints them
 * to the console
 */
function getUsers() {
  const request = new XMLHttpRequest();

  request.addEventListener('load', event => {
    // The event target is the XHR itself; it contains a
    // responseText property that we can use to create a JavaScript object from
    // the JSON text.
    const users = JSON.parse(event.target.responseText);
    console.log('Got users:', users);
  });

  // Handle any potential errors with the request.
  // This only handles network errors. If the request
  // returns an error status like 404, the 'load' event still fires
  // where you can inspect the status code.
  request.addEventListener('error', err => {
    console.log('Error!', err);
  });

  request.open('GET', '/api/users');
  request.send();
}

Discussion

The XMLHttpRequest API is an event-based API. When the response is received, a load event is triggered. In Example 4-1, the load event handler passes the raw response text to JSON.parse. It expects the response body to be JSON and uses JSON.parse to turn the JSON string into an object.

If an error occurs while loading the data, the error event is triggered. This handles connection or network errors, but an HTTP status code that’s considered an “error,” like 404 or 500, does not trigger this event. Instead, it also triggers the load event.

To protect against such errors, you need to examine the response’s status property to determine if such an error situation exists. This can be accessed by referencing event.target.status.

Fetch has been supported for a long time now, so unless you have to support really old browsers you most likely won’t need to use XMLHttpRequest. Most—if not all—of the time, you’ll be using the Fetch API.

Sending a GET Request with the Fetch API

Problem

You want to send a GET request to a public API using a modern browser.

Solution

Use the Fetch API. Fetch is a newer request API that uses Promises. It’s very flexible and can send all kinds of data, but Example 4-2 sends a basic GET request to an API.

Example 4-2. Sending a GET request with the Fetch API

/**
 * Loads users by calling the /api/users API, and parses the
 * response JSON.
 * @returns a Promise that resolves to an array of users returned by the API
 */
function loadUsers() {
  // Make the request.
  return fetch('/api/users')
    // Parse the response body to an object.
    .then(response => response.json())
    // Handle errors, including network and JSON parsing errors.
    .catch(error => console.error('Unable to fetch:', error.message));
}

loadUsers().then(users => {
  console.log('Got users:', users);
});

Discussion

The Fetch API is more concise. It returns a Promise that resolves to an object representing the HTTP response. The response object contains data such as the status code, headers, and body.

To get the JSON response body, you need to call the response’s json method. This method reads the body from the stream and returns a Promise that resolves to the JSON body parsed as an object. If the response body is not valid JSON, the Promise is rejected.

The response also has methods to read the body in other formats such as FormData or a plain text string.

Because Fetch works with Promises, you can also use await, as shown in Example 4-3.

Example 4-3. Using Fetch with async/await

async function loadUsers() {
  try {
    const response = await fetch('/api/users');
    return response.json();
  } catch (error) {
    console.error('Error loading users:', error);
  }
}

async function printUsers() {
  const users = await loadUsers();
  console.log('Got users:', users);
}

Note

Remember that before using await in a function, that function must have the async keyword.

Sending a POST Request with the Fetch API

Problem

You want to send a POST request to an API that expects a JSON request body.

Solution

Use the Fetch API, specifying the method (POST), and the JSON body and content type (see Example 4-4).

Example 4-4. Sending JSON payload via POST with the Fetch API

/**
 * Creates a new user by sending a POST request to /api/users.
 * @param firstName The user's first name
 * @param lastName The user's last name
 * @param department The user's department
 * @returns a Promise that resolves to the API response body
 */
function createUser(firstName, lastName, department) {
  return fetch('/api/users', {
    method: 'POST',
    body: JSON.stringify({ firstName, lastName, department }),
    headers: {
      'Content-Type': 'application/json'
    }
  })
    .then(response => response.json());
}

createUser('John', 'Doe', 'Engineering')
  .then(() => console.log('Created user!'))
  .catch(error => console.error('Error creating user:', error));

Discussion

Example 4-4 sends some JSON data in a POST request. Calling JSON.stringify on the user object turns it into a JSON string, which is required to send it as the body with fetch. You also need to set the Content-Type header so the server knows how to interpret the body.

Fetch also allows you to send other content types as the body. Example 4-5 shows how you would send a POST request with some form data.

Example 4-5. Sending form data in a POST request

fetch('/login', {
  method: 'POST',
  body: 'username=sysadmin&password=password',
  headers: {
    'Content-Type': 'application/x-www-form-urlencoded;charset=UTF-8'
  }
})
  .then(response => response.json())
  .then(data => console.log('Logged in!', data))
  .catch(error => console.error('Request failed:', error));

Uploading a File with the Fetch API

Problem

You want to upload file data with a POST request, using the Fetch API.

Solution

Use an <input type="file"> element, and send the file content as the request body (see Example 4-6).

Example 4-6. Sending file data with the Fetch API

/**
 * Given a form with a 'file' input, sends a POST request containing
 * the file data in its body.
 * @param form the form object (should have a file input with the name 'file')
 * @returns a Promise that resolves when the response JSON is received
 */
function uploadFile(form) {
  const formData = new FormData(form);
  const fileData = formData.get('file');
  return fetch('https://httpbin.org/post', {
    method: 'POST',
    body: fileData
  })
    .then(response => response.json());
}

Discussion

There aren’t many steps involved to upload a file using modern browser APIs. The <input type="file"> provides the file data through the FormData API and is included in the body of the POST request. The browser takes care of the rest.

Sending a Beacon

Problem

You want to send a quick request without waiting for a response, for example, to send analytics data.

Solution

Use the Beacon API to send data in a POST request. A regular POST request with the Fetch API may not complete in time before the page unloads. Using a beacon is more likely to succeed (see Example 4-7). The browser doesn’t wait for a response, and the request is more likely to succeed when sent as the user is leaving your site.

Example 4-7. Sending a beacon

const currentUser = {
  username: 'sysadmin'
};

// Some analytics data we want to capture
const data = {
  user: currentUser.username,
  lastVisited: new Date()
};

// Send the data before unload.
document.addEventListener('visibilitychange', () => {
  // If the visibility state is 'hidden', that means the page just became hidden.
  if (document.visibilityState === 'hidden') {
    navigator.sendBeacon('/api/analytics', data);
  }
});

A Note on Beacon Reliability

In the past, the recommendation was to use the beforeunload or unload events to send analytics beacons, but this can be unreliable in many cases. Many sites such as MDN now recommend using the visibilitychange event instead.

Discussion

With an XMLHttpRequest or fetch call, the browser waits for the response and returns it (with an event or Promise). In general, you don’t need to wait for the response for one-way requests, such as sending analytics data.

Instead of a Promise, navigator.sendBeacon returns a boolean value that indicates if the send operation was scheduled. There are no further events or notifications.

navigator.sendBeacon always sends a POST request. If you want to send multiple sets of analytics data, such as a collection of UI interactions, you can collect them in an array as the user interacts with your page, then send the array as the POST body with the beacon.

Listening for Remote Events with Server-Sent Events

Problem

You want to receive notifications from your backend server without repeated polling.

Solution

Use the EventSource API to receive server-sent events (SSE).

To start listening for SSE, create a new instance of EventSource, passing the URL as the first argument (see Example 4-8).

Example 4-8. Opening an SSE connection

const events = new EventSource('https://example.com/events');

// Fired once connected
events.addEventListener('open', () => {
  console.log('Connection is open');
});

// Fired if a connection error occurs
events.addEventListener('error', event => {
  console.log('An error occurred:', event);
});

// Fired when receiving an event with a type of 'heartbeat'
events.addEventListener('heartbeat', event => {
  console.log('got heartbeat:', event.data);
});

// Fired when receiving an event with a type of 'notice'
events.addEventListener('notice', event => {
  console.log('got notice:', event.data);
})

// The EventSource leaves the connection open. If we want to close the connection,
// we need to call close on the EventSource object.
function cleanup() {
  events.close();
}

Discussion

An EventSource must connect to a special HTTP endpoint that leaves the connection open with a Content-Type header of text/event-stream. Whenever an event occurs, the server can send a new message across the open connection.

Note

As pointed out by MDN, It’s highly recommended to use HTTP/2 with SSE. Otherwise, browsers impose a strict limit on the number of EventSource connections per domain. In this case, there can only be up to six connections.

This limit is not per tab; it is imposed across all tabs in the browser on a given domain.

When EventSource receives an event over a persistent connection, it is plain text. You can access the event text from the received event object’s data property. Here’s an example of an event of type notice:

event: notice
data: Connection established at 10:51 PM, 2023-04-22
id: 3

To listen for this event, call addEventListener('notice') on the EventSource object. The event object has a data property, whose value is whatever string value is prefixed with data: in the event.

If an event does not have an event type, you can listen for the generic message event to receive it.

Exchanging Data in Real Time with WebSockets

Problem

You want to send and receive data in real time without having to repeatedly poll the server with Fetch requests.

Solution

Use the WebSocket API to open a persistent connection to your backend server (see Example 4-9).

Example 4-9. Creating a WebSocket connection

// Open the WebSocket connection (the URL scheme should be ws: or wss:).
const socket = new WebSocket(url);

socket.addEventListener('open', onSocketOpened);
socket.addEventListener('message', handleMessage);
socket.addEventListener('error', handleError);
socket.addEventListener('close', onSocketClosed);

function onSocketOpened() {
  console.log('Socket ready for messages');
}

function handleMessage(event) {
  console.log('Received message:', event.data);
}

function handleError(event) {
  console.log('Socket error:', event);
}

function onSocketClosed() {
  console.log('Connection was closed');
}

Note

To use WebSockets, your server must have a WebSocket-enabled endpoint you can connect to. MDN has a nice deep dive on creating a WebSocket server.

Once the socket fires the open event, you can begin sending messages, as shown in Example 4-10.

Example 4-10. Sending WebSocket messages

// Messages are simple strings.
socket.send('Hello');

// The socket needs the data as a string, so you can use
// JSON.stringify to serialize objects to be sent.
socket.send(JSON.stringify({
  username: 'sysadmin',
  password: 'password'
}));

A WebSocket connection is a bidirectional connection. Received data from the server fires a message event. You can handle these as needed or even send a response (see Example 4-11).

Example 4-11. Responding to a WebSocket message

socket.addEventListener('message', event => {
  socket.send('ACKNOWLEDGED');
});

Finally, to clean up when you’re done, you can close the connection by calling close on the WebSocket object.

Discussion

WebSockets are well suited for apps requiring real-time capabilities such as a chat system or event monitoring. WebSocket endpoints have a ws:// or wss:// scheme. These are analogous to http:// and https://—one is insecure and one uses encryption.

To initiate a WebSocket connection, the browser first sends a GET request to the WebSocket endpoint. The request payload for the URL wss://example.com/websocket looks like this:

GET /websocket HTTP/1.1
Host: example.com
Sec-WebSocket-Key: aSBjYW4gaGFzIHdzIHBsej8/
Sec-WebSocket-Version: 13
Connection: Upgrade
Upgrade: websocket

This initiates a WebSocket handshake. If it’s successful, the server responds with a status code of 101 (Switching Protocols):

HTTP/1.1 101 Switching Protocols
Connection: Upgrade
Upgrade: websocket
Sec-WebSocket-Accept: bm8gcGVla2luZywgcGxlYXNlIQ==

The WebSocket protocol specifies an algorithm to generate a Sec-Websocket-Accept header based on the request’s Sec-WebSocket-Key. The client verifies this value, and at that point the two-way WebSocket connection is active and the socket fires the open event.

Once the connection is open, you can listen for messages with the message event and send messages by calling send on the socket object. Later, you can terminate the WebSocket session by calling close on the socket object.