Floating-Point

The book https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html is a must read about the subject.

Also, the site https://0.30000000000000004.com/ provides examples in several programming languages that help grasp the issue.

issues

floating-point issues

2**m * 2^(e)

m < 2**53 e < 2**1023

big numbers EAT small numbers

the precision of a double is a most 53 bits, meaning

return 1. + 2 ** 53 - 2**53

0.0

while

return 1. + 2 ** 52 - 2**52

1.0

import math
return math.log10(2**53)

15.954589770191003

Only 15 decimal digits of precision, so

return 1. + 10**16 - 10**16

0.0

while

return 1. + 10**15 - 10**15

1.0

error can accumulate

return 0.1 + 0.1 + 0.1, 0.3,

(0.30000000000000004, 0.3)

import math
def ratio(f):
    r = float.as_integer_ratio(f)
   return r[0], math.log2(r[1])
return ratio(0.1), ratio(0.2), ratio(0.3), ratio(0.1 + 0.1 + 0.1)

((3602879701896397, 55.0), (3602879701896397, 54.0), (5404319552844595, 54.0), (1351079888211149, 52.0))

Of course, 0.2 = 2 * 0.1, so the mantissa remains the same and the power of two that multiply it is simply decreased.

To do floating-point addition, one reduce the two numbers to the same exponent and then perform the integer addition on the mantissa, then normalize the power of two by dividing as much as possible by two.

So 0.1 + 0.2 means doing this

return 3602879701896397 * 2 + 3602879701896397, 55


10808639105689191	55

And then dividing by twos

return result[0] / 8, result[1] - 3

(1351079888211149.0, 52)

Which is what we found earlier, and is not the closest approximation of 0.3.

how far can we make the error big

People warn about how the error is an issue, but as far as I can tell, I could not find an error that lost more than 10 digits of precision. Can we actually loose more ?

See dent d’or de fontenelle

value = 0.
number = 10**8
for i in range(number):
    value += 0.1
return value, number / 10, abs((value - number / 10) / value)

(9999999.98112945, 10000000.0, 1.88705493120809e-09)

maximal mantissa per precision

floating-point maximal mantissa per precision

error simply put

floating-points number are not able to encode 0.1.

import decimal
decimal.Decimal(0.1)

0.1000000000000000055511151231257827021181583404541015625

Therefore, 0.1 added 10 times does not equal to 1.0

0.1 + 0.1 + 0.1 + 0.1 + 0.1 + 0.1 + 0.1 + 0.1 + 0.1 + 0.1 == 1.0

False

But, due to precision approximation, 0.9 and 0.1 equal to 1.0.

0.9 + 0.1 == 1.0

True

Konubinix' opinionated web of thoughts