Base converter with fractions?

02222017, 06:03 PM
Post: #22




RE: Base converter with fractions?
(02212017 07:30 PM)Han Wrote: Be careful with using nonCAS variables for CAS results. The CAS can handle integers of order larger than \( 10^{15} \) whereas we lose significant digits using nonCAS variables. So for "small" cases, your modifications would work just fine. However, for "larger" numbers (where after conversion into base 10, we would exceed the internal limit on the size of an integer), you would likely get the wrong result due to the conversion of an exact value from the CAS side into an approximate value in the nonCAS side. You are completely right Han. I did some testing, but not enough. I examined the content of tmp(2) and found out that it contained fractions, like 31/32 instead of numeric values. Therefore I thought it did exact calculations, but you are right that this goes wrong for large integers, numerators or denominators. 

« Next Oldest  Next Newest »

User(s) browsing this thread: 1 Guest(s)