Accuracy and the power function
02-19-2014, 12:35 PM
Post: #10
 Dieter Senior Member Posts: 2,397 Joined: Dec 2013
RE: Accuracy and the power function
(02-19-2014 09:54 AM)Werner Wrote:  There are two things at work here.
First, the 15-digit routines indeed are not exact to the last digit. In your example, %%LN(6.28318530718) is off by one ULP.
Secondly, the 15-digit routines (%%*, %%+, %%-, %%/) do not round, they truncate.
Ah, thank you very much, that's exactly what I have been looking for. I tried to simulate the error on a 34s in 16-digit mode with a similar calculation. So the key here is the logarithm that's off by one ULP.

Truncation of the last (guard) digit(s) seems to be a common practice. IIRC the TI58/59 did so as well, and today you still can see truncation at work, for instance on the 35s. Try the tangent function with very small angles:
Code:
[MODE] DEG 0,00001   [tan]  →  1,74532920000 E-7 0,0000001 [tan]  →  1,74532000000 E-9 [MODE] RAD 0,0005  [tan]  →  5,00000041660 E-4 0,0001  [tan]  →  1,00000000330 E-4 0,00001 [tan]  →  1,00000000000 E-5
In degrees mode, arguments between 1E-3 and 1E-9 have approx. 11...6 valid digits, where the last is truncated or even 1 unit low.

Dieter
 « Next Oldest | Next Newest »

 Messages In This Thread Accuracy and the power function - Dieter - 02-16-2014, 02:43 PM RE: Accuracy and the power function - Joe Horn - 02-16-2014, 03:11 PM RE: Accuracy and the power function - Dieter - 02-16-2014, 05:45 PM RE: Accuracy and the power function - Joe Horn - 02-16-2014, 10:11 PM RE: Accuracy and the power function - Dieter - 02-18-2014, 08:28 PM RE: Accuracy and the power function - Joe Horn - 02-19-2014, 12:33 AM RE: Accuracy and the power function - Werner - 02-19-2014, 09:54 AM RE: Accuracy and the power function - walter b - 02-19-2014, 11:36 AM RE: Accuracy and the power function - Dieter - 02-19-2014 12:35 PM RE: Accuracy and the power function - Werner - 02-19-2014, 11:47 AM RE: Accuracy and the power function - Werner - 02-19-2014, 01:07 PM RE: Accuracy and the power function - Dieter - 02-20-2014, 05:17 PM RE: Accuracy and the power function - Werner - 02-21-2014, 07:11 AM RE: Accuracy and the power function - Dieter - 02-23-2014, 02:19 PM RE: Accuracy and the power function - Thomas Klemm - 02-23-2014, 08:09 PM RE: Accuracy and the power function - Werner - 02-23-2014, 09:06 PM

User(s) browsing this thread: 1 Guest(s)