HP 35s. FIX affects precision Message #1 Posted by Pablo P (Spain) on 22 Oct 2010, 4:03 p.m.
Hi all, by accident I have found that FIX mode affects the precision in some calculations.
FIX mode is used to control the number of decimal digits used in the display of numbers. Only to display them. But consider the next:
Insert this function in the equation list:
EXP(-SQ(U)/2)/SQRT(2*pi)
Set FIX to 3 and integrate the function between -5 and 2.88
You will get 0.997993696055
Set FIX to 5 and integrate the function between -5 and 2.88. You will get 0.99801133646
Similar results, but not identical.
Should this be added to the infamous bug list? I don`t know if I am missing something...
Edited: 22 Oct 2010, 4:36 p.m.
|