Post Reply 
Angle being re-writed on entering it
06-07-2016, 02:38 PM
Post: #12
RE: Angle being re-writed on entering it
It is normal that machines have rounding "problems" (actually these are not really problems...)

But then another question:
to avoid such "problems" when you work with real numbers you can set the number format to have only a certain number of digits displayed, I normally work with floating 3.
Now, also with floating 3, I obtain 1080°00'37.00001", with 5 digits after the digital point.

Why?

I know that 1080°00'37" has more then 4 digits, but I find the seconds value (which is the last) should be rounded at 4 (1+3) digits, so 1080°00'37.00 which would be represented as 1080°00'37" in floating mode, avoiding the rounding problem.
Find all posts by this user
Quote this message in a reply
Post Reply 


Messages In This Thread
RE: Angle being re-writed on entering it - retoa - 06-07-2016 02:38 PM



User(s) browsing this thread: 1 Guest(s)