Post Reply 
(42) Double precision summation and DOT
04-15-2020, 12:54 AM (This post was last modified: 04-15-2020 06:13 PM by Albert Chan.)
Post: #5
RE: (42) Double precision summation and DOT
For decimal version, we can get precision P with less code

3 1/X 3 × 1 −                                  // -10^(-P), or 0 if binary version (*)

this is longer, but will work for binary version too

3 1/X Enter Enter Enter 1 − + +        // -10^(-P), or (-2)^(-P) for binary version

Example, on python 2.6:

>>> x = 1/3.
>>> print (x-1+x+x).hex()
-0x1.0000000000000p-53

(*) with binary, x = 1/3 has error of ±1/3 ULP
2^P ULP = 1/2 → 3x = 1 ± 1/2^(P+1), which always rounds back to 1
Find all posts by this user
Quote this message in a reply
Post Reply 


Messages In This Thread
RE: (42) Double precision summation and DOT - Albert Chan - 04-15-2020 12:54 AM



User(s) browsing this thread: 1 Guest(s)