(12C Platinum) Internal Precision Test

01022019, 07:33 PM
Post: #8




RE: (12C Platinum) Internal Precision Test
(01012019 03:44 AM)Gamo Wrote: The Internal Precision as shown in brochure stated Let's look at this a bit more closely. The reason for the different number of iterations is not the 15digit vs. 13digit internal precision of both calculators. The program calculates with 12 and 10 digit precision, and this is what causes the difference. The e approximation changes as long as 1/n! stays ≥ 5E–12 or 5E–10, respectively. This happens up to n=14 and n=12, respectively. This is what causes the different number of iterations. The fact that you determined 14 and 11 (instead of 12) is cause by random roundoff errors. Dieter 

« Next Oldest  Next Newest »

User(s) browsing this thread: 1 Guest(s)