02-22-2021, 04:40 AM
Albert Chan,
Well, the fact that changing the Precision Control alters the results of the same operations is not a bug, but a well-known fact. As a matter of fact, is the basis of the example about the Forensics Algorithm:
Another example:
Antonio
Well, the fact that changing the Precision Control alters the results of the same operations is not a bug, but a well-known fact. As a matter of fact, is the basis of the example about the Forensics Algorithm:
RPN.exe users manual Wrote:This is an example of such an operation based on the Forensics Algorithm:. The exact result of this expression is 9, but the real results are 9.0338335037231445, 9.0000000001046097 and 8.9999999999999130 when the precision is set to 0, 2 and 3, respectively.Code:
RPN FPUPC:0 9 Deg Sin Cos Tan ATan ACos ASin FIX:16
Another example:
Code:
RPN: FPUPC:3 1e16 sci:16 out out:" = " eng:17 out out:" = " hex:3 out
1.0000000000000000E+016 = 10.0000000000000000E+0015 = 4034_8E1B_C9BF_03FF_FFF9
RPN: FPUPC:2 1e16 sci:16 out out:" = " eng:17 out out:" = " hex:3 out
1.0000000000000000E+016 = 99.9999999999999744E+0015 = 4034_8E1B_C9BF_03FF_FFF9
RPN: FPUPC:0 1e16 sci:16 out out:" = " eng:17 out out:" = " hex:3 out
9.9999999999999820E+015 = 9.22337203685477580E+0015 = 4034_8E1B_C9BF_03FF_B800
RPN: FPUPC:3 1e16 sci:16 out out:" = " eng:17 out out:" = " hex:3 out
9.9999916826296320E+015 = 9.22337203685477580E+0015 = 4034_8E1B_C200_0000_0000
Antonio