Post Reply 
Division in CAS
05-18-2022, 05:52 PM (This post was last modified: 05-18-2022 08:01 PM by matalog.)
Post: #1
Division in CAS
I was writing a long integer divider program, and was testing a value to see if it was working, and it was right, but it looks like the Prime was wrong, in CAS. I know this is very unlikely, and I hope someone will explain that it is down to the limited amount of information stored about the number, resulting in the wrong number being displayed after the place where E was.

I calculated 123456789123456789/127, in CAS mode. It gave the result 123456789123456789/127, then I pressed a b/c and it gave 9.72100701759E14 and I then pressed a b/c again, which resulted in the value that is not correct, which is 972100701759492.

I have just tried another 147258369147258369/127 and pressing a b/c twice again results in 1159514717694936, which is wrong by 11! The answer could have been as accurate as 1159514717694947.787401574803149606299212598425196850393700787401574803149606299​2125984251968503937007874015748031496.

Is there any way to get full length division results in CAS, just like the long results from multiplication? Or a way to get more accurate results from CAS?
Find all posts by this user
Quote this message in a reply
Post Reply 


Messages In This Thread
Division in CAS - matalog - 05-18-2022 05:52 PM
RE: Division in CAS - Nigel (UK) - 05-18-2022, 08:41 PM
RE: Division in CAS - Albert Chan - 05-18-2022, 09:37 PM



User(s) browsing this thread: 1 Guest(s)