Post Reply 
H->HMS conversion HP-15C vs. HP42S vs HP67
09-21-2018, 12:16 PM
Post: #66
RE: H->HMS conversion HP-15C vs. HP42S vs HP67
(09-21-2018 03:06 AM)Craig Bladow Wrote:  I ran into this issue when developing NQ41, which uses double precision binary math, and after looking into possible solutions decide not to address it and let it display 1.2960 as we all know what that means and the conversion back to decimal degrees can be done more accurately than if the rounding was used to obtain a display of 1.30.

An unintended consequence of deg_to_dms patch to fix 40-seconds bug, 60-seconds bug is shifted away.
Now, user need to input 16+ decimals to hit the display bug.

Example: angle = 2.05 degrees, X = 2.049999 ...

Pekis DEG->DMS(X) = 2.0259999 ... (Hit 60 seconds display bug)
My deg_to_dms(X) = 2.030000 ... (display bug shifted down, to 2.04999 99999 99999 degree)

So, the patch is Buy 1, Get 1 Free Smile
Find all posts by this user
Quote this message in a reply
Post Reply 


Messages In This Thread
RE: H->HMS conversion HP-15C vs. HP42S vs HP67 - Albert Chan - 09-21-2018 12:16 PM



User(s) browsing this thread: 1 Guest(s)