F# for Scientists

Chapter 4. NUMERICAL ANALYSIS

Computers can only perform finite computations. Consequently, computers only make use of finite precision representations of numbers. This has several important implications in the context of scientific computation.

This chapter provides an overview of the representations and properties of values of types int and float, used to represent members of the sets Z and R, respectively. Practical examples demonstrating the robust use of floating-point arithmetic are then given. Finally, some other forms of arithmetic are discussed.

NUMBER REPRESENTATION

In this section, we shall introduce the representation of integer and floating-point numbers before outlining some properties of these representations.

Machine-precision integers

Positive integers are represented by several, least-significant binary digits (bits). For example, the number 1 is represented by the bits ... 00001 and the number 11 is represented by the bits ... 01011. Negative integers are represented in twoscomplement format. For example, the number −1 is represented by the bits ... 11111 and the number −11 is represented by the bits ... 10101.

Figure 4.1. Values i of the type int, called machine-precision integers, are an exact representation of a consecutive subset of the set of integers i ϵ [I... u] C Z where I and u are given by min_int and max_int, respectively.

Consequently, the representation of ...