Post Reply 
newRPL: [UPDATED April 27-2017] Firmware for testing available for download
10-22-2015, 06:16 PM
Post: #77
RE: newRPL: [UPDATED Oct-19-2015] Firmware for testing available for download
But 527.67 x (5/9) does produce 293.15 ...
on a 12-digit machine:

527.67 x 0.555555555556
= 293.15000000023452

I see you multiply by 0.555555555555, that is not the correctly rounded 5/9
Alternatively you may multiply by 5 and then divide by 9 - only the latter will produce a rounding error.

Cheers, Werner

41CV†,42S,48GX,49G,DM42,DM41X,17BII,15CE,DM15L,12C,16CE
Find all posts by this user
Quote this message in a reply
Post Reply 


Messages In This Thread
RE: newRPL: [UPDATED Oct-19-2015] Firmware for testing available for download - Werner - 10-22-2015 06:16 PM



User(s) browsing this thread: 7 Guest(s)