Re: Possible bug in WP34s with big integers Message #3 Posted by Jake Schwartz on 22 June 2011, 4:08 p.m., in response to message #1 by Neil Hamilton (Ottawa)
Quote:
The answer WP34s gives is: 0x20ED7AD3E4850430.
....and leaving the overflow flag set. Since the MSB is the sign, the value would not be able to be represented in only 63 bits and if the MSB was able to be set, the value would erroneously go negative. If you run the program in the 34S and receive 20ED 7AD3 E485 0430 as a result, then get the leading hex digit to become an A by the following: 100000000000 ENTER 10000 * + 100000000000 ENTER 70000 * +, switching to decimal generates 6,850,684,407,670,045,648. Also, doing the problem in unsigned mode results in a decimal value of 11,596,059,666,039,505,968. The log(base 2) of this value is 63.33..., so since more than 63 bits would be needed to represent it, the signed 64bit modes could not hold it. I think the 34S is okay.
Jake
Jake
