Free42 with IEEE 7542008 decimal floatingpoint  interested in a sneak preview?

03022014, 11:06 AM
Post: #28




RE: Free42 with IEEE 7542008 decimal floatingpoint  interested in a sneak preview?
The 34S comes up with ...502 following that sequence too so I kind of suspect that that the is the correctly rounded answer in 34 digit precision. Not that I trust the 34S's result in double precision mode all that much, but two separate implementations attempting to do the right thing coming up with the same answer is compelling to some degree.
I just checked with wolfram alpha and ACOS(0) ends ...751(44) so the correctly rounded result will be ...502. The joys of two rounding steps  Pauli 

« Next Oldest  Next Newest »

User(s) browsing this thread: 1 Guest(s)