34s/16C Compat. #6 Message #1 Posted by Jake Schwartz on 19 May 2011, 2:17 p.m.
Hi,
I was playing with some programs in the 16C manual and ran into something on the 34s which I thought should be shared. In unsigned, 64bit mode with base HEX, if I press 0 ENTER 1  it correctly yields "FFFF FFFF FFFF FFFF", and changing over to decimal base correctly shows 18,446,744,073,709,551,615. This value is greater than 1.8 * 10^19. Now I am aware that on the 34s, one cannot enter numeric values with more than 12 digits. I wanted to enter the decimal number 1*10^19, so in decimal base I pressed 1000000 ENTER * which correctly yielded 1,000,000,000,000 or 10^12. Doing another 1000000 * produced 1,000,000,000,000,000,000 (or 10^18) and converting to HEX correctly showed DE0 B6B3 A764 0000. Now, if I multiply by ten again by 10 * in base decimal, it suddenly shows 776,627,963,145,224,192 instead of the expected 10,000,000,000,000,000,000. Switching over to HEX at this point reveals AC7 2304 89E8 0000. For some reason, the machine will not allow values as large as 10^19 to be generated, despite the fact that a 64bit word should be able to hold numbers above 1.8 * 10^19.
Paul, any ideas?
Thanks,
Jake
