Introducing Geographic Information Systems with ArcGIS: A Workbook Approach to Learning GIS, 3rd Edition

Computers and Inexact Computation

The LENGTH, AREA, and PERIMETER field values in the attribute table were carried over from the coverages. They are, respectively, 100, 2000, and 240, exactly. The Shape_Length of each line, in Attributes of feat_num_lns, however, is calculated in the conversion. It displays as 99.999999. The Shape_Lengths and Shape_Areas in the polygon table are likewise slightly inaccurate. As previously mentioned, computers cannot be counted upon to give exact answers. The difficulty is that, while computers can usually do exact arithmetic with integers, they cannot be exact with floating-point numbers, which may have fractional parts.

These errors occur because humans do arithmetic with decimal (base 10), and computers do arithmetic with binary (base 2). For example, it is not possible to exactly represent the decimal number one-tenth (0.1) in binary. One-tenth in binary is represented (imprecisely) by the sum of some of the fractional powers of two: one-sixteenth, one-thirty-second, one-two-hundred-fifty-sixth, and so on. In binary one tenth looks something like 0.000110011... To be exact, the number would have to have an infinite number of bits.10 (Some decimal fractions can be represented exactly in binary. For example, decimal 0.5 is 0.1 in binary (that’s 2 to the minus one power); 0.25 is 0.01; 0.75 is 0.11; you see the pattern.)

The errors that can occur when using floating-point numbers with decimal parts are illustrated by the following steps.