Floating-point comparisons
The representation of floating-point numbers in computers is not exact. This leads to issues when comparing floating-point numbers. The assert_array_almost_equal_nulp() and assert_array_max_ulp() NumPy functions provide consistent floating-point comparisons. Unit of Least Precision (ULP) of floating-point numbers, according to the IEEE 754 specification, a half ULP precision is required for elementary arithmetic operations. You can compare this to a ruler. A metric system ruler usually has ticks for millimeters, but beyond that you can only estimate half millimeters.
Machine epsilon is the largest relative rounding error in floating-point arithmetic. Machine epsilon is equal to ULP relative to 1. The NumPy finfo() function ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access