Post Reply 
NumWorks open source calculator
09-06-2017, 08:08 PM
Post: #42
RE: NumWorks open source calculator
(09-06-2017 05:59 PM)Chasfield Wrote:  The calculator forensics calculation returns 8.966758 (for 9 degrees) on this device and it seems to always display results to 7 significant figures. Calculation precision seems therefore to be on the low side.

Is it perhaps squandering higher levels of precision by carrying forward rounded up/down display values rather than underlying register values?

I tested the expression:

(355/113 - pi)/pi * 1e9

I got a unit 2 days ago!! I was able to update the firmware version this morning.

The new firmware version of 1.1.0 gave 84.91xxxx which is good. The previous firmware version of 1.0.3 gave a higher number (did not think to record it) which reflected poorer accuracy. SO it seems the new firmware version has enhanced accuracy.

Namir
Find all posts by this user
Quote this message in a reply
Post Reply 


Messages In This Thread
NumWorks open source calculator - BobVA - 08-28-2017, 10:33 PM
RE: NumWorks open source calculator - Dan - 08-29-2017, 02:37 AM
RE: NumWorks open source calculator - Namir - 09-06-2017 08:08 PM
NumWorks User manual!!!!!!!!!! - Namir - 09-06-2017, 08:10 PM
RE: NumWorks open source calculator - JDW - 07-07-2018, 10:30 PM
RE: NumWorks open source calculator - JDW - 07-08-2018, 12:55 PM
RE: NumWorks open source calculator - JDW - 07-09-2018, 12:03 AM
RE: NumWorks open source calculator - JDW - 07-10-2018, 12:04 AM
RE: NumWorks open source calculator - JDW - 07-15-2018, 01:11 PM
RE: NumWorks open source calculator - JDW - 07-12-2018, 12:03 AM
RE: NumWorks open source calculator - JDW - 07-14-2018, 01:41 PM



User(s) browsing this thread: 1 Guest(s)