HP 40gs, error in calculator?
11-04-2017, 01:41 AM (This post was last modified: 11-04-2017 01:46 AM by toml_12953.)
Post: #8
 toml_12953 Senior Member Posts: 1,141 Joined: Dec 2013
RE: HP 40gs, error in calculator?
(11-03-2017 09:37 PM)pier4r Wrote:  interesting the difference

sin(pi)
-1.158028306006248917735787454453501 E-34 WP-34S in "double mode"
-1.158028306006248941790250554076922 E-34 DM_42 with SHOW

I would have expected way more equal digits.

Maybe Thomas, Paul, Marcus, Walter, Dieter or Gerson (and I forget many) could explain why this difference.

It looks like one of them is using double precision mode to give 16-17 digits of precision. The DM-42 uses Intel's Decimal Floating-Point Math Library which conforms to the IEEE 754-2008 Decimal Floating-Point Arithmetic specification and uses quadruple-precision floating-point format (34.02 decimal digits.) What does the WP-34S use?

Tom L

French phrases you rarely hear: "trop de beurre" (too much butter)
 « Next Oldest | Next Newest »

 Messages In This Thread HP 40gs, error in calculator? - Skipjack - 11-01-2017, 07:31 PM RE: HP 40gs, error in calculator? - brickviking - 11-01-2017, 09:45 PM RE: HP 40gs, error in calculator? - Dieter - 11-01-2017, 09:46 PM RE: HP 40gs, error in calculator? - toml_12953 - 11-01-2017, 11:16 PM RE: HP 40gs, error in calculator? - Jake Schwartz - 11-03-2017, 05:40 PM RE: HP 40gs, error in calculator? - Massimo Gnerucci - 11-03-2017, 08:04 PM RE: HP 40gs, error in calculator? - Claudio L. - 11-04-2017, 05:02 AM RE: HP 40gs, error in calculator? - pier4r - 11-03-2017, 09:37 PM RE: HP 40gs, error in calculator? - toml_12953 - 11-04-2017 01:41 AM

User(s) browsing this thread: 1 Guest(s)