Post Reply 
Crash when Solve for Integral with 15C
09-02-2017, 01:40 PM
Post: #24
RE: Crash when Solve for Integral with 15C
(09-02-2017 05:53 AM)Dieter Wrote:  The result is about 6 E+91. FIX 2 means two digits after the decimal point, so that's actually 94 significant digits, in other words: it's (more than) full machine precision you're demanding. On the other hand SCI 4 sets only five significant digits which is returned much faster.

Thanks Dieter. All this makes sense to me, as is so often the case, after the explanation. Though I initially would have guessed that the 15C would treat FIX 2 as 2 significant places, even if it must shift to SCI mode, just as it does when the displayed number must be so shifted. Though subtle, this is a significant difference and highlights how the machine handles the numbers internally (vs. display) and so is an interesting exercise. I will read the referenced section in the 15C AF HB. Again! AFAIK, this is the best treatment of such topics in any HP manual; can you recommend others I may have missed?

--Bob Prosperi
Find all posts by this user
Quote this message in a reply
Post Reply 


Messages In This Thread
RE: Crash when Solve for Integral with 15C - rprosperi - 09-02-2017 01:40 PM



User(s) browsing this thread: 1 Guest(s)