Post Reply 
help with exact command. [Solved]
04-25-2016, 08:33 PM (This post was last modified: 04-25-2016 08:56 PM by Spybot.)
Post: #3
RE: help with exact command.
(04-25-2016 08:13 PM)Didier Lachieze Wrote:  The issue is that 0.333 is quite different from 1/3, how can the Prime guess that the result you're expecting is the fraction 1/3 while the fraction 333/1000 is exactly equal to 0.333?
You need to provide at less 10 significant digits to get 1/3: exact(0.3333333333)=1/3

I can make 0.333 to become 1/3 by changing some HOMEsettings [number format:rounded & HDigits to 3]... it'll do the trick though, the thing is that... when I try to replicate those settings from within a program and attempt to do the same operation... it's not working as expected!

EXPORT z()
BEGIN
LOCAL M;
5▶HFormat;
3▶HDigits;
M:=0.333;
M:=exact(M);
PRINT(M);
END;

Spybot.
Find all posts by this user
Quote this message in a reply
Post Reply 


Messages In This Thread
help with exact command. [Solved] - Spybot - 04-25-2016, 07:54 PM
RE: help with exact command. - Spybot - 04-25-2016 08:33 PM
RE: help with exact command. - DrD - 04-25-2016, 09:27 PM
RE: help with exact command. - Spybot - 04-25-2016, 11:18 PM



User(s) browsing this thread: 1 Guest(s)