Post Reply 
CAS and small numbers
04-22-2014, 07:28 PM (This post was last modified: 04-22-2014 07:29 PM by Han.)
Post: #3
RE: CAS and small numbers
When working in CAS, you are much better off using exact values. So instead of typing something like: 2.1E-10, I would recommend 21*10^(-11) as the value. Then you can convert to decimal form at the very end if needed. Most CAS will treat values having a fraction mark (in your case, a decimal point) as an implicit request to use approximate algorithms so that roundoff could possibly result in "useless" answers.

Graph 3D | QPI | SolveSys
Find all posts by this user
Quote this message in a reply
Post Reply 


Messages In This Thread
CAS and small numbers - Nigel (UK) - 04-22-2014, 06:49 PM
RE: CAS and small numbers - Tim Wessman - 04-22-2014, 07:21 PM
RE: CAS and small numbers - Nigel (UK) - 04-22-2014, 08:22 PM
RE: CAS and small numbers - Han - 04-22-2014, 08:53 PM
RE: CAS and small numbers - Nigel (UK) - 04-22-2014, 09:19 PM
RE: CAS and small numbers - Tugdual - 04-23-2014, 05:42 AM
RE: CAS and small numbers - Han - 04-22-2014 07:28 PM
RE: CAS and small numbers - parisse - 04-23-2014, 07:14 AM
RE: CAS and small numbers - Nigel (UK) - 04-23-2014, 06:35 PM
RE: CAS and small numbers - Tim Wessman - 04-23-2014, 07:12 PM
RE: CAS and small numbers - parisse - 04-23-2014, 07:05 PM
RE: CAS and small numbers - Nigel (UK) - 04-23-2014, 07:29 PM



User(s) browsing this thread: 1 Guest(s)