WebAssembly: The Definitive Guide

Chapter 4. WebAssembly Memory

Perhaps one day this too will be pleasant to remember.

Virgil

If WebAssembly is going to behave like a regular runtime environment, it needs a way to allocate and free memory for its data-handling activities. In this chapter, we will introduce you to how it emulates this behavior for efficiency but without the risk of typical memory manipulation problems seen with languages like C and C++ (even if that is what we are running). As we are potentially downloading arbitrary code over the internet, this is an important safety consideration.

The entire concept of computation usually involves some form of data processing. Whether we are spell-checking a document, manipulating an image, doing machine learning, sequencing proteins, playing video games, watching movies, or simply crunching numbers in a spreadsheet, we are generally interacting with arbitrary blocks of data. One of the most crucial performance considerations in these systems is how to get the data where it needs to be in order to interrogate or transform it somehow.

Central Processing Units (CPUs) work the fastest when data is available in a register or an on-chip cache.¹ Obviously these are very small containers, so large data sets are never going to be loaded onto the CPU in their entirety. We have to spend some effort moving data into and out of memory. The cost of waiting for the data to be loaded to one of these locations is an eternity in CPU clock time. This is one of the reasons they have gotten so complex. Modern chips have all manner of multipipeline, predictive branching, and instruction rewriting available to keep the chip busy while we are reading from a network into main memory, from there into multilevel caches, and finally to where it needs to be used.

Traditional programs have usually had stack memory to manage short-term variables of small or fixed sizes. They use heap-based memory for longer-term, arbitrarily sized blocks of data. These are generally just different areas of the memory allocated to a program that are treated differently. Stack memory gets overwritten frequently by the ebb and flow of functions being called during execution. Heap memory is used and cleaned up when it is no longer needed. If a program runs out of memory, it can ask for more, but it must be reasonably judicious about how it uses it.² These days virtual paging systems and cheaper memory make it entirely likely that a typical computer might have tens of gigabytes of memory. Being able to quickly and efficiently access individual bytes of potentially large data sets is a major key to decent software runtime performance.

WebAssembly programs need a way to simulate these blocks of memory without actually giving unfettered access to the privacy of our computer’s memory. Fortunately, there is a good story to tell here that balances convenience, speed, and safety. It starts with making it possible for JavaScript to access individual bytes in memory, but will expand beyond JavaScript to be a generic way of sharing memory between host environments and WebAssembly modules.

TypedArrays

JavaScript has traditionally not been able to provide convenient access to individual bytes in memory. This is why time-sensitive, low-level functionality is often provided by the browser or some kind of plug-in. Even Node.js applications often have to implement some functionality in a language that handles memory manipulation better than JavaScript can. This complicates the situation, as JavaScript is an interpreted language and you would need an efficient mechanism for switching control flow back and forth between interpreted, portable code, and fast compiled code. This also makes deployments trickier because one part of the application is inherently portable and one needs native library support on different operating systems.

There is usually a trade-off in software development: languages are either fast or they are safe. When you need raw speed, you might choose C or C++ as they provide very few runtime checks in the use and manipulation of data in memory. Consequently, they are very fast. When you want safety, you might pick a language with runtime boundary checks on array references. The downside of the speed trade-off is that things are either slow or the burden of memory management falls to the programmer. Unfortunately, it is extremely easy to mess up by forgetting to allocate space, reusing freed memory, or failing to deallocate the space when you are done. This is one of the reasons applications written in these fast languages are often buggy, crash easily, and serve as the source for many security vulnerabilities.³

Garbage-collected languages such as Java and JavaScript free developers from many of the burdens of managing memory, but often incur a performance burden at runtime as a trade-off. A piece of the runtime must constantly look for unused memory and release it. The performance overhead makes many such applications unpredictable and therefore unsuitable for embedded applications, financial systems, or other time-sensitive use cases.

Allocating memory is not a huge issue as long as what is created is a suitable size for what you want to put in it. The tricky part is knowing when to clean up. Obviously, freeing memory before a program is done with it is bad, but failing to do so when it is no longer needed is inefficient and you might run out of memory. Languages such as Rust strike a nice balance of convenience and safety. The compiler forces you to communicate your intentions more clearly, but when you do, it can be more effective in cleaning up after you.

How this is all managed at runtime is often one of the defining characteristics of a language and its runtime. As such, not every language requires the same level of support. This is one of the reasons WebAssembly’s designers did not overspecify features such as garbage collection in the MVP.

JavaScript is a flexible and dynamic language, but it has not historically made it easy or efficient to deal with individual bytes of large data sets. This complicates the use of low-level libraries, as the data has to be copied into and out of JavaScript-native formats, which is inefficient. The Array class stores JavaScript objects, which means it has to be prepared to deal with arbitrary types. Many of Python’s flexible containers are also similarly flexible and bloated.⁴ Fast traversal and manipulation of memory through pointers is a product of the uniformity of the data types in contiguous blocks. Bytes are the minimum addressable unit, particularly when dealing with images, videos, and sound files.

Numerical data requires more effort. A 16-bit integer takes up two bytes. A 32-bit integer, four. Location 0 in a byte array might represent the first such number in an array of data, but the second one will start at location 4.

JavaScript added TypedArray interfaces to address these issues, initially in the context of improving WebGL performance. These are portions of memory available through ArrayBuffer instances that can be treated as homogenous blocks of particular data types. The memory available is constrained to the ArrayBuffer instance, but it can be stored internally in a format that is convenient to pass to native libraries.

In Example 4-1, we see the basic functionality of creating a typed array of 32-bit unsigned integers.

Example 4-1. Ten 32-bit integers created in a Uint32Array

var u32arr = new Uint32Array(10);
u32arr[0] = 257;
console.log(u32arr);
console.log("u32arr length: " + u32arr.length);

The output of the invocation should look like this:

Uint32Array(10) [ 257, 0, 0, 0, 0, 0, 0, 0, 0, 0 ]
u32arr length: 10

As you can see, this works as you would expect an array of integers to. Keep in mind that these are 4-byte integers (thus the 32 in the type name). In Example 4-2, we retrieve the underlying ArrayBuffer from the Uint32Array and print it out. This shows us that its length is 40. Next we wrap the buffer with a Uint8Array representing an array of unsigned bytes and print out its contents and length.

Example 4-2. Accessing the 32-bit integers as a buffer of 8-bit bytes

var u32buf = u32arr.buffer;
var u8arr = new Uint8Array(u32buf);
console.log(u8arr);
console.log("u8arr length: " + u8arr.length);

The code produces the following output:

ArrayBuffer { byteLength: 40 }
Uint8Array(40) [ 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, … ]
u8arr length: 40

The ArrayBuffer represents the raw underlying bytes. The TypedArray is an interpreted view of those bytes based upon the specified type size. So when we initialized the Uint32Array with a length of 10, that meant ten 32-bit integers, which requires 40 bytes to represent. The detached buffer is set to be this big so it can hold all 10 integers. The Uint8Array treats each byte as an individual element due to its size definition.

If you check out Figure 4-1, you will hopefully see what is going on. The first element (position 0) of the Uint32Array is simply the value 257. This is an interpreted view of the underlying bytes in the ArrayBuffer. The Uint8Array directly reflects the underlying bytes of the buffer. The bit patterns at the bottom of the diagram reflect the bits per byte for the first two bytes.

Figure 4-1. Representing the value 257

It may surprise you that there are 1s in the first two bytes. This is due to a confusing notion called endianess that shows up when we store numbers in memory.⁵ In this case, a little endian system stores the least significant bytes first (the 1s). A big endian system would store the 0s first. In the grand scheme of things, it does not matter how they are stored, but different systems and protocols will pick one or the other. You just need to keep track of which format you are seeing.

As indicated earlier, TypedArray classes were introduced initially for WebGL, but since then, they have been adopted by other APIs including Canvas2D, XMLHttpRequest2, File, Binary WebSockets, and more. Notice these are all lower-level, performance-oriented I/O and visualization APIs that have to interface with native libraries. The underlying memory representation can be passed between these layers efficiently. It is for these reasons that they are useful for WebAssembly Memory instances as well.

WebAssembly Memory Instances

A WebAssembly Memory is an underlying ArrayBuffer (or SharedArrayBuffer, as we will see later) associated with a module. The MVP limits a module to having a single instance at the moment, but this is likely to change before long. A module may create its own Memory instance, or it may be given one from its host environment. These instances can be imported or exported just like we have done with functions so far. There is also an associated Memory section in the module structure that we skipped over in Chapter 3 because we had not covered the concept yet. We will fix that omission now.

In Example 4-3, we have a Wat file that defines a Memory instance and exports it as the name "memory". This represents a contiguous block of memory constrained to a particular ArrayBuffer instance. It is the beginning of our ability to emulate C/C++-like homogenous arrays of bytes in memory. Each instance is made up of one or more 64-kilobyte blocks of memory pages. In the example, we initialize it to a single page but allow it to grow up to 10 pages for a total of 640 kilobytes, which ought to be enough for anyone.⁶ You will see how to increase the available memory momentarily. For now, we are just going to write the bytes 1, 1, 0, and 0 to the beginning of the buffer. The i32.const instruction loads a constant value onto the stack. We want to write to the beginning of our buffer, so we use the value 0x0. The data instruction is a convenience for initializing portions of our Memory instance.

Example 4-3. Creating and exporting a Memory instance in a WebAssembly module

(module
  (memory (export "memory") 1 10)
  (data (i32.const 0x0) "\01\01\00\00")
)

If we compile this file to its binary representation with wat2wasm and then invoke wasm-objdump, we see some new details we have not yet encountered:

brian@tweezer ~/g/w/s/ch04> wasm-objdump -x memory.wasm

memory.wasm:	file format wasm 0x1

Section Details:

Memory[1]:
 - memory[0] pages: initial=1 max=10
Export[1]:
 - memory[0] -> "memory"
Data[1]:
 - segment[0] memory=0 size=4 - init i32=0
  - 0000000: 0101 0000

There is a configured Memory instance in the Memory section reflecting our initial size of one page and maximum size of 10 pages. We see that it is exported as "memory" in the Export section. We also see the fact that the Data section has initialized our memory instance with the four bytes we wrote into it.

Now we can use our exported memory by importing it into some JavaScript in the browser. For this example, we are going to load the module and fetch the Memory instance. We then display the buffer size in bytes, the number of pages, and what is currently in the memory buffer.

The basic structure of our HTML file is shown in Example 4-4. We have a series of <span> elements that will be populated with the details via a function called show​De⁠tails(), which will take a reference to our memory instance.

Example 4-4. Display Memory details in the browser

<!doctype html>
<html lang="en">
  <head>
    <meta charset="utf-8">
    <link rel="stylesheet" href="bootstrap.min.css">
    <title>Memory</title>
    <script src="utils.js"></script>
  </head>
  <body>
    <div class="container">
      <h1>Memory</h1>
      <div>Your memory instance is <span id="mem"></span> bytes.</div>
      <div>It has this many pages: <span id="pages"></span>.</div>
      <div>Uint32Buffer[0] = <span id="firstint"></span>.</div>
      <div>Uint8Buffer[0-4] = <span id="firstbytes"></span>.</div>
    </div>

    <button id="expand">Expand</button>

    <script>
       <!-- Shown below -->
    </script>
  </body>
</html>

In Example 4-5, we see the JavaScript for our <script> element. First look at the fetchAndInstantiate() call. It behaves in the same way we have seen before in terms of loading the module. Here we get a reference to the Memory instance through the exports section. We attach an onClick() function for our button that we will address momentarily.

Example 4-5. The JavaScript code for our example

function showDetails(mem) {
  var buf = mem.buffer;
  var memEl = document.getElementById('mem');
  var pagesEl = document.getElementById('pages');
  var firstIntEl = document.getElementById('firstint');
  var firstBytesEl = document.getElementById('firstbytes');

  memEl.innerText=buf.byteLength;
  pagesEl.innerText=buf.byteLength / 65536;

  var i32 = new Uint32Array(buf);
  var u8 = new Uint8Array(buf);

  firstIntEl.innerText=i32[0];
  firstBytesEl.innerText= "[" + u8[0] + "," + u8[1] + "," +
                                u8[2] + "," + u8[3] + "]";
};

fetchAndInstantiate('memory.wasm').then(function(instance) {
  var mem = instance.exports.memory;

  var button = document.getElementById("expand");
  button.onclick = function() {
    try {
       mem.grow(1);
       showDetails(mem);
    } catch(re) {
       alert("You cannot grow the Memory any more!");
    };
  };
  showDetails(mem);
});

Finally, we call the showDetails() function and pass in our mem variable. This function will retrieve the underlying ArrayBuffer and references to our various <span> elements to display the details. The buffer’s length is stored in the innerText field of our first <span>. The number of pages is this length divided by 64 KB to indicate the number of pages. We then wrap the ArrayBuffer with a Uint32Array, which allows us to fetch our memory values as 4-byte integers. The first element of this is shown in the next <span>. We also wrap our ArrayBuffer in Uint8Array and show the first four bytes. After our discussion earlier, the details shown in Figure 4-2 should not surprise you.

Figure 4-2. Showing the details of our Memory

The onClick() function calls a method on the Memory instance to grow the allocated size by one page of memory. This causes the original ArrayBuffer to become detached from the instance, and the existing data is copied over. If we are successful, we reinvoke the showDetails() function and extract the new ArrayBuffer. If the button is pressed once, you should see that the instance now represents two pages of memory representing 128 KB of memory. The data at the beginning should not have changed.

If you press the button too many times, the number of allocated pages will exceed the maximum specified amount of 10 pages. At this point, it is no longer possible to expand the memory and a RangeError will be thrown. Our example will pop up an alert window when this happens.

Using the WebAssembly Memory API

The grow() method we used in the previous example is part of the WebAssembly JavaScript API that the MVP expects all host environments to provide. We can expand our use of this API and go in the other direction. that is, we can create a Memory instance in JavaScript and then make it available to a module. Keep in mind the current limit of one instance per module.

In subsequent chapters, we will see more elaborate uses of memory, but we will want to use a higher-level language than Wat to do anything serious. For now, we will keep our example on the simpler side but still try to expand beyond what we have seen.

We will start with the HTML so you can see the whole workflow, and then we will dive into the details of the new module. In Example 4-6, you can see that we are using a similar HTML structure to what we have used so far. There is a <div> element with the ID of container into which we will place a series of Fibonacci numbers. If you are not familiar with these numbers, they are very important in a lot of natural systems, and you are encouraged to investigate them on your own. The first two numbers are defined to be 0 and 1. The subsequent numbers are set to the sum of the previous two. So the third number will be 1 (0 + 1). The fourth number will be "2" (1 + 1). The fifth number will be 3 (2 + 1), etc.

Example 4-6. Creating a Memory in JavaScript and importing it to the module

<!doctype html>
<html lang="en">
  <head>
    <meta charset="utf-8">
    <link rel="stylesheet" href="bootstrap.min.css">
    <title>Fibonacci</title>
    <script src="utils.js"></script>
  </head>
  <body>
    <div id="container"></div>

    <script>
      var memory = new WebAssembly.Memory({initial:10, maximum:100});

      var importObject = {
        js: { mem: memory }
      };

      fetchAndInstantiate('memory2.wasm', importObject).then(function(instance) {
	  var fibNum = 20;
	  instance.exports.fibonacci(fibNum);
	  var i32 = new Uint32Array(memory.buffer);

	  var container = document.getElementById('container');

	  for(var i = 0; i < fibNum; i++) {
	      container.innerText += `Fib[${i}]: ${i32[i]}\n`;
	  }
      });

    </script>
  </body>
</html>

The actual calculation is written in Wat and shown in Example 4-7, but before we get there, we see the creation of the Memory instance on the first line of the <script> element. We are using the JavaScript API, but the intent is the same as our use of the (memory) element in Example 4-3. We create an initial size of one page of memory and a maximum size of 10 pages. In this case, we will never need more than the one page, but you now see how to do it. The Memory instance is made available to the module via the importObject. As you will see momentarily, the function in the Wasm module will take a parameter indicating how many Fibonacci numbers to write into the Memory buffer. In this example, we will pass in a parameter of 20.

Once our module is instantiated, we call its exported fibonacci() function. We have access to the memory variable from above, so we can retrieve the underlying ArrayBuffer directly after the function invocation completes. Because Fibonacci numbers are integers, we wrap the buffer in a Uint32Array instance so we can iterate over the individual elements. As we retrieve the numbers, we do not have to worry about the fact that they are 4-byte integers. Upon reading each value, we extend the innerText of our container element with a string version of the number.

The calculation shown in Example 4-7 is going to be significantly more complicated than any Wat we have seen so far, but by approaching it in pieces you should be able to figure it out.

Example 4-7. Fibonacci calculations expressed in Wat

(module
  (memory (import "js" "mem") 1) 
  (func (export "fibonacci") (param $n i32) 
    (local $index i32) 
    (local $ptr i32) 

    (i32.store (i32.const 0) (i32.const 0)) 
    (i32.store (i32.const 4) (i32.const 1))

    (set_local $index (i32.const 2)) 
    (set_local $ptr (i32.const 8))

    (block $break 
      (loop $top 
        (br_if $break (i32.eq (get_local $n) (get_local $index))) 
        (i32.store 
          (get_local $ptr)
          (i32.add
            (i32.load (i32.sub (get_local $ptr) (i32.const 4)))
  	    (i32.load (i32.sub (get_local $ptr) (i32.const 8)))
	  )
        )
	(set_local $ptr (i32.add (get_local $ptr) (i32.const 4))) 
	(set_local $index (i32.add (get_local $index) (i32.const 1)))
	(br $top) 
      )
    )
  )
)

The Memory is imported from the host environment.

The fibonacci function is defined and exported.

$index is our number counter.

$ptr is our current position in the Memory instance.

The i32.store function writes a value to the specified location in the buffer.

The $index variable is advanced to 2 and the $ptr is set to 8.

We define a named block to return to in our loops.

We define a named loop in our block.

We break out of our loop when the $index variable equals the $n parameter.

We write the sum of the previous two elements to current location of $ptr.

We advance the $ptr variable by 4 and the $index variable by 1.

We break to the top of our loop.

Hopefully the numeric notes attached to Example 4-7 make sense, but given its complexity, it warrants a quick discussion. This is a stack-based virtual machine, so all of the instructions involve manipulating the top of the stack. In the first callout, we import the memory defined in the JavaScript. It represents the default allocation of one page, which should be enough for now. While this is a correct implementation, it is not an overly safe implementation. Bad inputs could mess up the flow, but we will be more concerned with that after we introduce higher-level language support where it is easier to handle those details.

The exported function is defined to take a parameter $n representing the number of Fibonacci numbers to calculate.⁷ We use two local variables defined at the third and fourth callouts. The first represents which number we are working on and defaults to 0. The second will act as a pointer in memory. It will serve as the index into the Memory buffer. Remember, i32 data values represent 4 bytes, so every advance of $index will involve advancing $ptr by 4. We do not have the benefit of TypedArrays on this side of the interaction, so we have to handle these details ourselves. Again, higher-level languages will shield us from many of these details.

By definition, the first two Fibonacci numbers are 0 and 1, so we write those into the buffer. i32.store writes an integer value to a location. It expects to find those values on the top of the stack, so the next two parts of the statement invoke the i32.const instruction, which pushes the specified values to the top of the stack. First, an offset of 0 indicates we want to write to the beginning of the buffer. The second one pushes the number 0 to the stack to indicate the value we want to write in position 0. The next line repeats the process for the next Fibonacci number. The i32 from the previous line takes up 4 bytes, so we write value 1 to position 4.

The next step is to iterate over the remaining numbers, which are each defined as the sum of the previous two. This is why we need to start the process with the two we just wrote. We advance our $index variable to 2, so we will need $n – 2 iterations of the loop. We have written two i32 integers, so we advance our $ptr to 8.

Wat references several WebAssembly instructions that you will be introduced to over the course of the book. Here you can see some of the looping constructs. We define a block at the seventh callout and give it a label of $break. The next step introduces a loop with an entry point called $top. The first instruction in the loop checks to see if $n and $index are equal, indicating we have handled all of our numbers. If so, it will break out of the loop. If not, it proceeds.

The i32.store instruction at the 10th callout writes to the $ptr location. The values of variables are pushed to the top of the stack with get_local. The value to write there is the addition of the values of the previous two numbers. i32.add expects to find its two addends at the top of the stack as well. So we load the integer location that is four less than $ptr. This represent $n – 1. We then load the integer stored at the location of $ptr minus 8, which represents $n – 2. i32.add pops these addends off the top of the stack and writes their sum back to the top. The stack now contains this value at the top and the location of the current $ptr value, which is what the i32.store is expecting.

The next step advances $ptr by four since we have now written another Fibonacci number to the buffer. We advance $n by one and then break to the top of the loop and repeat the process. Once we have written $n numbers to the buffer, the function returns. It does not need to return anything since the host environment has access to the Memory buffer and can read the results out directly with TypedArrays, as we saw earlier.

The result of loading our HTML into the browser and displaying the first 20 Fibonacci numbers is shown in Figure 4-3.

Figure 4-3. Reading the Fibonacci sequence from the Memory instance

This level of detail would be annoying to deal with regularly, but fortunately you will not have to. It is important to understand how things work at this level, though, and how we can emulate continguous blocks of linear memory for efficient processing.

Strings at Last!

One final discussion before we move on is about how we can finally add strings to our repertoire! There are many more tools coming in later chapters of the book to make things even easier, but we can take advantage of some conveniences in Wat to write strings into Memory buffer and read them out on the JavaScript side.

In Example 4-8, you can see a very simple module that exports a one-page Memory instance. It then uses a data instruction to write a sequence of bytes into a location in the module’s memory. It starts at location 0 and writes the bytes in the subsequent string. It is a convenience to not have to convert the multibyte strings into their component bytes, although you certainly can if you like. This string has a Japanese sentence and then its translation in English.⁸

Example 4-8. A simple use of strings in Wat

(module
 (memory (export "memory") 1)
 (data (i32.const 0x0) "私は横浜に住んでいました。I used to live in Yokohama.")
)

Once we compile the Wat to Wasm, we see that we have a new populated section in our module. You can see this with the wasm-objdump command:

brian@tweezer ~/g/w/s/ch04> wasm-objdump -x strings.wasm

strings.wasm:	file format wasm 0x1

Section Details:

Memory[1]:
 - memory[0] pages: initial=1
Export[1]:
 - memory[0] -> "memory"
Data[1]:
 - segment[0] memory=0 size=66 - init i32=0
  - 0000000: e7a7 81e3 81af e6a8 aae6 b59c e381 abe4  ................
  - 0000010: bd8f e382 93e3 81a7 e381 84e3 81be e381  ................
  - 0000020: 97e3 819f e380 8249 2075 7365 6420 746f  .......I used to
  - 0000030: 206c 6976 6520 696e 2059 6f6b 6f68 616d   live in Yokoham
  - 0000040: 612e

The Memory, Export, and Data sections are filled in with the details of our strings written to memory. The instance is initialized this way so when a host environment reads from the buffer, the strings will already be there.

In Example 4-9, you see that we have one <span> for our Japanese sentence and one for our English sentence. To extract the individual bytes, we can wrap a Uint8Array around the Memory instance buffer that we have imported from the module. Notice that we only wrap the first 39 bytes. These bytes are decoded to a UTF-8 string via a TextDecoder instance, and then we set the innerText of the <span> designated for the Japanese sentence. We then wrap a separate Uint8Array around the portion of the buffer starting at position 39 and including the subsequent 26 bytes.

Example 4-9. Reading strings from an imported Memory instance

<!doctype html>
<html lang="en">
  <head>
    <meta charset="utf-8">
    <link rel="stylesheet" href="bootstrap.min.css">
    <title>Reading Strings From Memory</title>
    <script src="utils.js"></script>
  </head>
  <body>
    <div>
      <div>Japanese: <span id="japanese"></span></div>
      <div>English: <span id="english"></span></div>
    </div>
    <script>
      fetchAndInstantiate('strings.wasm').then(function(instance) {
	  var mem = instance.exports.memory;

	  var bytes = new Uint8Array(mem.buffer, 0, 39);
          var string = new TextDecoder('utf8').decode(bytes);
          var japanese = document.getElementById('japanese');
	  japanese.innerText = string;

	  bytes = new Uint8Array(mem.buffer, 39, 26);
	  string = new TextDecoder('utf8').decode(bytes);
          var english = document.getElementById('english');
	  english.innerText = string;
      });

    </script>
  </body>
</html>

In Figure 4-4, we see the successful results of reading the bytes out of the buffer and rendering them as UTF-8 strings.

Figure 4-4. Reading strings from the Memory instance

As cool as these results are, how did we know how many bytes to wrap and in which location to look for the strings? A little detective work can help. A capital letter “I” is represented as 49 in hexadecimal. The output from wasm-objdump gives us the offset in the Data segment for each byte. We see the value 49 for the first time on the row that begins with 0000020:. The 49 represents the seventh byte over, so the second sentence begins at position 27, which is 2 × 16 + 7 in decimal, so, 39. The Japanese string represents the bytes between 0 and 39. The English string begins at position 39.

But, wait a minute! It turns out we miscounted on the English sentence and we were off by one. This seems like a troublesome and error-prone amount of effort to get strings out of a WebAssembly module. Even doing things the hard way at this low level can be handled better. We will write out the locations of the strings first so we do not have to figure it out on our own.

Look at Example 4-10 to see how we can be more sophisticated. We have two data segments now. The first writes the starting position and length of the first string followed by the same information for the second one. Because we are using the same buffer for the indices and the strings, we have to be careful about locations.

As our strings are not very long, we can use single bytes as offsets and lengths. This is probably not a good strategy in general, but it will show off some additional flexibility. So, we write out the value 4 and the value 27. This represents an offset of 4 bytes and a length of 39. The offset is 4 because we have these four numbers (as single bytes) at the beginning of the buffer and will need to skip over them to get to the strings. As you now know, 27 is hexadecimal for 39, the length of the Japanese string. The English sentence will begin at index 4 + 39 = 43, which is 2b in hexadecimal (2 × 16 + 11) and is 27 bytes long, which is 1b in hexadecimal (1 × 16 + 11).

The second data segment starts at position 0x4 because we need to skip over those offsets and lengths.

Example 4-10. A more sophisticated use of strings in Wat

(module
 (memory (export "memory") 1)
 (data (i32.const 0x0) "\04\27\2b\1b")
 (data (i32.const 0x4) "私は横浜に住んでいました。I used to live in Yokohama.")
)

In Example 4-11, we see the other side of reading the strings out. It is certainly more complicated now, but it is also less manual as the module tells us exactly where to look. Another option when using TypedArrays is a DataView, which allows you to pull arbitrary data types out of the Memory buffer. They do not need to be homogenous like the normal TypedArrays (e.g., Uint32Array).

Example 4-11. Reading our indexed strings from the Memory buffer

<!doctype html>
<html lang="en">
  <head>
    <meta charset="utf-8">
    <link rel="stylesheet" href="bootstrap.min.css">
    <title>Reading Strings From Memory</title>
    <script src="utils.js"></script>
  </head>
  <body>
    <div>
      <div>Japanese: <span id="japanese"></span></div>
      <div>English: <span id="english"></span></div>
    </div>
    <script>
      fetchAndInstantiate('strings2.wasm').then(function(instance) {
	  var mem = instance.exports.memory;

	  var dv = new DataView(mem.buffer);
	  var start = dv.getUint8(0);
	  var end = dv.getUint8(1);

	  var bytes = new Uint8Array(mem.buffer, start, end);
	  var string = new TextDecoder('utf8').decode(bytes);
          var japanese = document.getElementById('japanese');
	  japanese.innerText = string;

	  start = dv.getUint8(2);
	  end = dv.getUint8(3);

	  bytes = new Uint8Array(mem.buffer, start, end);
	  string = new TextDecoder('utf8').decode(bytes);
          var english = document.getElementById('english');
	  english.innerText = string;
      });

    </script>
  </body>
</html>

We therefore wrap the exported Memory buffer with a DataView instance and read in the first two bytes by calling the getUint8() function once at location 0 and once at location 1. These represent the location and offset in the buffer for the Japanese string. Other than no longer using hardcoded numbers, the rest of our previous code is the same. Next we read out the two bytes at location 2 and 3, representing the location and length of the English sentence. This too is converted to a UTF-8 string and updated correctly this time, as seen in Figure 4-5.

Figure 4-5. Reading indices and strings from the Memory instance

As a homework assignment, try creating an even more flexible approach that tells you how many strings there are to read and what their locations and lengths are. The JavaScript to read it in can be made into a loop, and the whole process should be more flexible.

There is more to know about Memory instances as you shall see later, but for now, we have covered enough of the basics of WebAssembly that trying to do anything more sophisticated by hand in Wat will be too painful. Thus, it is time to use a higher-level language like C!

¹ A register is an on-chip memory location that usually feeds an instruction what it needs to execute.

² My first computer, an Atari 800, started off with only 16 kilobytes of memory. It was a big to-do the day my dad came home with a 32-kilobyte expansion card!

³ Ryan Levick highlights this point in his discussion of Microsoft’s interest in Rust.

⁴ The NumPy library helps solve this by reimplementing homogenous storage in C arrays and having compiled forms of the mathematical functions to run on those structures.

⁵ This is a reference to Gulliver’s Travels by Jonathan Swift.

⁶ Nice try, but no, Bill Gates never said it!

⁷ As a thought exercise, what could $n potentially be set to before our i32 data type would overflow? How could you address that?

⁸ It’s true!