Post Reply 
newRPL: [UPDATED April 27-2017] Firmware for testing available for download
03-29-2016, 05:20 PM (This post was last modified: 03-29-2016 07:31 PM by Visitor.)
Post: #246
RE: newRPL: [UPDATED Mar-09-16] Firmware for testing available for download
Hello Claudio,

in dectranscen.c many constants (e.g. decconst_PI() ) are given back with twice the precision as set
due to shift right of only 2
Code:

int nwords=253-((Context.precdigits+7)>>2);

is that intended?

currently a side effect will be that with the default settings (32 digits) if I enter pi0 i get back pi with almost 70 digits (instead of 32)

shouldn't the constant being rounded before pushing it to the display?

if I multiply this longer number by 1 it will be rounded to the system precision.

BTW I'm admiring your work and the time you are putting into this project. I just came recently across this project and am very excited of it.

regards

Martin
Find all posts by this user
Quote this message in a reply
Post Reply 


Messages In This Thread
RE: newRPL: [UPDATED Mar-09-16] Firmware for testing available for download - Visitor - 03-29-2016 05:20 PM



User(s) browsing this thread: 6 Guest(s)