@Thomas Klemm -> CORDIC Article
|
06-04-2014, 06:30 PM
(This post was last modified: 06-04-2014 06:41 PM by pito.)
Post: #26
|
|||
|
|||
RE: @Thomas Klemm -> CORDIC Article
WP34s calc and various other arbitrary precision decimal fp libs/wrappers are using the IBM's decNumber library (which sets the standards). There are some settings for the rounding inside (context something), I never messed with them in detail, though. Maybe the difference between the results is related to the proper rounding setting's "know-how" with the library..
mpdecimal for example: Quote:libmpdec - C/C++ library |
|||
« Next Oldest | Next Newest »
|
User(s) browsing this thread: 1 Guest(s)