HP Forums / HP Calculators (and very old HP Computers) / General Forum v / Programming the INT and FRAC functions

Post Reply 
Programming the INT and FRAC functions
01-05-2016, 12:42 PM
Post: #15
Pekis Offline
Member
 		Posts: 157
Joined: Aug 2014
RE: Programming the INT and FRAC functions
Hello,

Beware of the internal precision before applying INT ...

e.g. with a 12-digit mantissa and 10 digits displayed:

While the number 15.9999999999 (kept in internal memory) is displayed as 16, should the INT function have to display 15 (applying INT directly) or 16 (Rounding to 10 digits then applying INT) ?
Find all posts by this user
Quote this message in a reply
Post Reply 


Messages In This Thread
RE: Programming the INT and FRAC functions - Pekis - 01-05-2016 12:42 PM



User(s) browsing this thread: 2 Guest(s)

Contact Us | The Museum of HP Calculators | Return to Top | Return to Content | Lite (Archive) Mode | RSS Syndication