Post Reply 
Summation based benchmark for calculators
12-21-2017, 12:40 PM (This post was last modified: 01-09-2018 10:22 PM by pier4r.)
Post: #1
Summation based benchmark for calculators
There are plenty of tests for calculators, some more organized, some less.

See http://www.hpmuseum.org/cgi-sys/cgiwrap/...i?read=700

or the ones collected in the wiki4hp. http://www.wiki4hp.com/doku.php?id=start&do=index (see the benchmark section).

Seeing the work of Eddie. http://www.hpmuseum.org/forum/thread-5047.html

I thought that when a benchmark includes a summation, like the savage benchmark a bit, but the savage benchmark uses the previous result to compute the next iteration. Such a benchmark could be generic enough for programmable and scientific calculators with summation abilities and could also include different instructions. While the 8 queens benchmark does not really scientific math functions.

I will try something when I have time.

Results in the post below.

note to myself: to collect info it helps to send PM . Thanks to all the contributors!

Wikis are great, Contribute :)
Find all posts by this user
Quote this message in a reply
12-22-2017, 09:41 PM (This post was last modified: 04-07-2018 10:39 AM by pier4r.)
Post: #2
RE: Summation based test for calculators
I did not forget this post (nor the topic about common calculators).

So as said the summation based test will be very similar to the savage benchmark and I wonder how many similar tests were developed on various forum or groups regarding calculators.

Anyway, inspired by the test done in the thread of Eddie linked in the 1st post, I decided to pick randomly some scientific functions to assemble a summation that may give an idea of the performances of a calculator for common math functions.

The idea is to have a loop working on an increment of the 'x' value that is somewhat independent from the step before (while the savage test uses the value picked from the step before) . Furthermore while the idea of using a function and its inverse is pretty neat (see savage benchmark), some calculators may be very carefully coded figuring this out and therefore simplifying the expression.

I am not that interested in the accuracy, as it is well analyzed in many other posts using other examples (especially by Dieter, that for me will always be the ULPs man).

Surely speed without accuracy is not that neat and so one could go on and build a metric that weights accuracy and speed, as done by a recent HHC (about egyptian fractions), anyway I will collect only timings.

Timings will be collected here, and then moved on the wiki4hp if there are enough of them.

People are encouraged to post timings and results.

So to recap the idea of this test is there because:
- with a summation with increasing x (not dependent on the previous computed value) also some advanced scientific calculators can be tested. See casio fx991EX.
- it is easier to type in and execute. It does not take much time (well unless one is forced to write a program to optimize some parts).
- it may be used with a metric that combines speed and accuracy (a problem would be to get the right accuracy for the problem).
- it avoids to use a function and its inverse to skip very careful optimizations done by the parser. (especially systems with CAS may use this)


Here is the assembled formula (I guess there can be an infinity of formulas that capture the idea I am going to expose).

\[ tan^{-1}(x) \]
So x can be incremented without problems. Picked among the trigonometric functions.

\[ sin \left (tan^{-1}(x) \right ) \]
To let the value increase towards 1, although, after a while, very slowly. Still trig.

\[ e^{ sin \left (tan^{-1}(x) \right ) } \]
to use either a common exponential or a logarithmic function without producing large numbers

\[ \sqrt[3]{e^{ sin \left (tan^{-1}(x) \right ) }} \]
To make the final number even smaller. Using powers.

Final formula to use (adapt the max 'x' value for speedy calculators. For example using 10k or 100k and so on)
\[ \sum_{x=1}^{1000} \sqrt[3]{e^{ sin \left (tan^{-1}(x) \right ) }} \]

As said, it is an arbitrary formula picking functions among some of the most common types of functions: trig, exp/ln, powers. Trying to keep the final numbers small. I may have picked some other functions easier to check (to compare the accuracy). This one is pretty nasty. Anyway as I said I am interested in timings assuming that the calculators are as precise as they could with the digits that they have.

I am really interested in what scientific calculators can do. I hope Eddie W. Shore sees this posts and contributes with a couple of models. Or jebem.

Mobile devices with calculator apps
max = 100000
~ 7.9s - Free42 iphone SE and 6s http://www.hpmuseum.org/forum/thread-975...l#pid94029

Results physical calculators

max = 100000
~ 17.7s - casio fx 9860GII power graphic 2 SHA4 overclock 236 +Ftune F5 preset C.Basic ver.1.73beta Sigma(Cbrte^sin tan^-1 X,X,1,100000) - 139560.976141052
~ 19.2s - HW-Prime , Home, Teval(), HP PPL. last os as 2017.12
~ 20.5s - casio fx cg50 SH4A 192MHz ptune C.BasicCG ver.0.45alpha Sigma(Cbrte^sin tan^-1 X,X,1,100000) - 139560.97614105
~ 23s - HP 50g HPGCC 2.0 - 75 Mhz - 139560.8013952589
~ 29.9s - casio fx cg20 SH4A 118MHz C.BasicCG ver.0.45alpha Sigma(Cbrte^sin tan^-1 X,X,1,100000) - 139560.976141052
~ 36s - casio fx cg50 SH4A 118MHz C.BasicCG ver.0.45alpha Sigma(Cbrte^sin tan^-1 X,X,1,100000) - 139560.976141052
~ 52s - casio fx 9860GII SH3 overclock 118MHz +Ftune F5 preset C.Basic ver.1.73beta Sigma(Cbrte^sin tan^-1 X,X,1,100000) - 139560.976141052
~ 84s - numworks 2017.12 python
~ 117s - casio fx 9860GII power graphic 2 SHA4 normal 29Mhz C.Basic ver.1.73beta Sigma(Cbrte^sin tan^-1 X,X,1,100000) - 139560.976141052
~ 147s - casio fx 9860GII SH3 normal 29 mhz C.Basic ver.1.73beta Sigma(Cbrte^sin tan^-1 X,X,1,100000) - 139560.976141052
~ 195s - Nspire CX sum function approx mode.
~ 222s - nspire CX CAS, sum function, approx mode.
~ 261s - DM42 on USB, RPN
~ 323s - 9750gII upgraded to 9860gII OS {sum:eqn,x,1, 100000} gives me 139560.9761 CPU frequency sitting at 235 MHz (F5 in FTune)
~ 653s - DM42 on batteries, RPN
~ 2009s - 9750gII upgraded to 9860gII OS {sum:eqn,x,1, 100000} gives me 139560.9761

max = 10000
~ 1.8s - casio fx 9860GII power graphic 2 SHA4 overclock 236 +Ftune F5 preset C.Basic ver.1.73beta Sigma(Cbrte^sin tan^-1 X,X,1,10000) - 13955.8579042916
~ 2.072s - HW-Prime , Home, Teval(), HP PPL. last os as 2017.12
~ 2s - HP 50g HPGCC 2.0 - 75 Mhz - 13955.85790429154
~ 2.1s - casio fx cg50 SH4A 192MHz ptune C.BasicCG ver.0.45alpha Sigma(Cbrte^sin tan^-1 X,X,1,10000) - 13955.8579042916
~ 3.1s - casio fx cg20 SH4A 118MHz C.BasicCG ver.0.45alpha Sigma(Cbrte^sin tan^-1 X,X,1,10000) - 13955.8579042916
~ 3.8s - casio fx cg50 SH4A 118MHz C.BasicCG ver.0.45alpha Sigma(Cbrte^sin tan^-1 X,X,1,10000) - 13955.8579042916
~ 5.3s - casio fx 9860GII SH3 overclock 118MHz +Ftune F5 preset C.Basic ver.1.73beta Sigma(Cbrte^sin tan^-1 X,X,1,10000) - 13955.8579042916
~ 8s - numworks 2017.12 python
~ 11.9s - casio fx 9860GII power graphic 2 SHA4 normal 29Mhz C.Basic ver.1.73beta Sigma(Cbrte^sin tan^-1 X,X,1,10000) - 13955.8579042916
~ 14.9s - casio fx 9860GII SH3 normal 29 mhz C.Basic ver.1.73beta Sigma(Cbrte^sin tan^-1 X,X,1,10000) - 13955.8579042916
~ 16s - numworks 2017.12 sum function
~ 19s - nspire CX, sum function approx mode
~ 22.5s - nspire CX CAS, sum function, approx mode.
~ 20 sec - ti nspire handheld (2006). sum function. Degrees, float 12, approx. 13955.8579044 . OS 3.9.0.463
~ 23.9s - HW-Prime , Home, sum function. last os as 2017.12
~ 26s - DM42 on USB, RPN
~ 32s - 9750gII upgraded to 9860gII OS {sum:eqn,x,1,10000} gives me 13955.8579 CPU frequency sitting at 235 MHz (F5 in FTune)
~ 56,51s - Hp 50g newRPL 2017.12 FOR loop., 12 digits precision
~ 65.73s - DM42 on batteries, RPN
~ 199s - 9750gII upgraded to 9860gII OS {sum:eqn,x,1,10000} gives me 13955.8579
~ 233s - casio fx 9860gII, sum function
~ 242s - hp 50g, sum function
~ 260s - hp 50g, sysRPL
~ 309s - hp 50g, userRPL
~ 402.91s - TI 84 Plus CE
~ 453s - Ti 92 plus, sum function
~ 465s - ti 89, approx mode, sum function.
~ 468s - 41CL, v5 board, TURBO 50. Using the Sigma+ function to accumulate intermediate values. 13955.84859
~ 472s - ti 89 titanium, 12 digits, approx mode, degrees, no pretty print. OS 3.10 (code sigma((e^(...))^(1/3), x, 1, 1000)) where 'sigma' is the greek letter . 13955.8579044
~ 541s - hp 48gx, userRPL
~ 554s - hp 48gx, sum function
~ 986s - Ti 92, sum function
~ 1105s - casio fx 9750g, sum function
~ 1184s - DM11L (48 mhz), RPN
~ 1185s - Ti 82, ti basic
~ 1246s - hp28s, userRPL
~ 1501s - casio fx880p, casio basic

max = 1000
~ 0.223s - HW-Prime , Home, Teval(), HP PPL. last os as 2017.12
~ 2s - HW-Prime , Home, sum function. last os as 2017.12
~ 2.5s - nspire CX CAS, sum function, approx mode.
~ 2.6s - DM42 on USB, RPN
~ 4s - casio fx-9750gII (upgraded with 9860gII OS 2.04) and overclocked to max speed with FTune (likely a ~ 5x overclock) sum function
~ 6.022s - Hp 50g newRPL 2017.12 FOR loop., 12 digits precision
~ 6.5s - Casio fx-CP400 (classpad 400) , 1395.346288 sum function
~ 6.64s - DM42 on batteries, RPN
~ 8.2s - Hp 50g newRPL 2017.12 FOR loop., 32 digits precision 1395.346287743423 (I assume the result returned truncated on the stack)
~ 12.582s - HW-Prime , CAS, approx mode. last os as 2017.12
~ 13.5s - Casio fx-CG50 (primz), 1395.346288 sum function
~ 18s - Casio ClassPad 330-A, 1395.346288 (ClassPad is labeled on the back side as CLASSPAD300PLS, the 330-A is written on the box) sum function
~ 20s - Casio fx-7400GII , 1395.346288 sum function
~ 22s - casio fx-9750gII (upgraded with 9860gII OS 2.04), 1395.346288 sum function
~ 23s - casio fx 9860gII, sum function
~ 23s - Casio fx-9860G Slim (hacked to 9860GII OS 2.04), 1395.346288 sum function
~ 24.5s - Hp 50g, 2.15, RPN mode, DEG, sum function . 1395.3462877 (approx mode)
~ 24.5 - CASIO Prizm fx-CG10, sum function
~ 25.5s - Hp 50g, 2.15, exact mode sum function
~ 26.5s - hp 50g, sysRPL
~ 29.4s - Hp 50g newRPL 2017.12 FOR loop., 128 digits precision
~ 29s - HP15C LE
~ 33.8s - Hp 50g, 2.15, RPN mode, DEG (quick userRPL FOR loop, using, instead of XROOT, the 1/3 power. Surprisingly slower than the summation) . 1395.3462877 (approx mode)
- 34s - HP-40gs sum function, 1395.3462877
~ 38s - sharp el-9950 , 1395.346288
~ 41.5s - TI 84 plus CE
~ 45s - Ti 92 plus, sum function
~ 46s - ti 89 titanium, 12 digits, approx mode, degrees, no pretty print. OS 3.10 (code sigma((e^(...))^(1/3), x, 1, 1000)) where 'sigma' is the greek letterl . 1395.34628774
~ 46s - ti 89, approx mode, sum function.
~ 48s - ti 89 titanium, 12 digits, approx mode, degrees, no pretty print. OS 3.10 (code sum(seq((e^(...))^(1/3), x, 1, 1000))) . 1395.34628774
~ 48s - 41CL, v5 board, TURBO 50. Using the Sigma+ function to accumulate intermediate values. 1395.346260
~ 54s - hp 48gx, userRPL
~ 55s - hp 48gx, sum function
~ 55s - 48G+ , sum function with speed ui
~ 62s - ti 89 OS 2.09. (exact mode plus likely with pretty print)
~ 64s - sharp el-w506x (abusing the integral function).
~ 76s - Casio fx-9700GE , 1395.34628774 sum function
~ 97s - sharp el-506x (abusing the integral function).
~ 99s - Ti 92, sum function
~ 99.5s - casio fx-9750g+ sum function
~ 102s - Sharp PC-G850VS Basic, 1395.346559
~ 103s - casio fx 9750g, sum function
~ 108s - casio fx991 version , 1395.346288 (the fast batch reported by some users) sum function
~ 121s - Ti 82, ti basic
~ 125s - DM11L (48 mhz), RPN
~ 130s - hp28s, userRPL
~ 131s - casio fx991 version , 1395.346288 (the slow batch reported by some users) sum function
~ 133s - 136s - DM15L , 48 mhz (interesting the difference with the 41L), 1395.346288
~ 148s - HP 75D http://www.hpmuseum.org/forum/thread-975...l#pid88453
~ 153s - casio fx880p, casio basic. Another run 158s (see post #76).
~ 166s - Casio fx-5800P, 1395.346288 sum function
~ 168s - WP 34S double off, fix 2, sum function
~ 173s - Epson Hx-20 . max 7 digits precision. 1395.36
~ 173s - Epson Hx-20 . max 16 digits precision. 1395.346369147301
~ 171-178s - HP 71B http://www.hpmuseum.org/forum/thread-975...l#pid88453 - http://www.hpmuseum.org/forum/thread-975...l#pid90635
~ 185s - WP 34S double on, fix 2, sum function
~ 200s - casio fx8500g, sum function
~ 206s - HP 32s RPN 1395.34628770 http://www.hpmuseum.org/forum/thread-975...l#pid89004
~ 209s - HP 33s RPN http://www.hpmuseum.org/forum/thread-982...l#pid87566
~ 218s - Ti 80, ti basic
~ 241s - HP 32sII RPN 1395.34628770
~ 245s - TI 36x Pro sum function
~ 253s - Hp 35s http://www.hpmuseum.org/forum/thread-982...l#pid87766
~ 256s - wp34s (DSE-based loop, DBLOFF), 1395.346287743423
~ 285s - wp34s (DSE-based loop, DBLON), 1395.346287743423256291575365067091
~ 287s - Hp35s, RPN program
~ 291s - HP 42s , RPN program
~ 298s - sharp el-506w (abusing the integral function. Not really returning the wanted result from the summation).
~ 303s - wp34s (S command, DBLOFF), 1395.346287743423
~ 332s - wp34s (S command, DBLON), 1395.346287743423256291575365067093
~ 404.93s - DM41L 48 mhz , RPN program
~ 502.2s - Sharp 516X sum function
~ 525s - Casio fx-115ES Plus: 1395.346288 sum function
~ 633s - Casio fx-3650pII , 1395.346288 for next
~ 633s - Casio fx-50F PLUS , 1395.346288 for next
~ 840-890s - Canon X Mark I Pro, 1395.346288, sum function

max = 100
~ 0.036s - HW-Prime , Home, Teval(), HP PPL. last os as 2017.12
~ 0.2s - HW-Prime , Home, sum function. last os as 2017.12
~ 0.28s - DM42 on USB, RPN
~ 0.6s - DM42 on batteries, RPN
~ 2.5s - casio fx 9860gII, sum function
~ 2.7s - hp 50g, sum function
~ 2.8s - hp 50g, sysRPL
~ 3.3s - hp 50g, userRPL
~ 4s - ti 89, approx mode, sum function.
~ 5s - Ti 92 plus, sum function
~ 5.13s - 41CL, v5 board, TURBO 50. Using the Sigma+ function to accumulate intermediate values. 139.297187
~ 5.5s - hp 48gx, userRPL
~ 5.9s - hp 48gx, sum function
~ 6s - 48G+ , sum function with speed ui
~ 10s - Ti 92, sum function
~ 11s - casio fx 9750g, sum function
~ 11s - Sharp PC-G850VS Basic, 139.2971873
~ 13s - Ti 82, ti basic
~ 14s - DM11L (48 mhz), RPN
~ 14s - dm15L 48 mhz, RPN, 139.2971874
~ 14s - casio fx991 version , 1395.346288 (the slow batch reported by other users) sum function
~ 14s - hp28s, userRPL
~ 15s - HP-41CL / x50 speed: 100 iterations (139.2926) RPN
~ 16s - casio fx880p, casio basic
~ 17s - Casio FX603P, casio basic
~ 18s - Epson Hx-20 . max 7 digits precision. 139.297
~ 18s - Epson Hx-20 . max 16 digits precision. 139.297193646431
~ 21s - casio fx8500g, sum function
~ 22s - sharp el-9300 http://www.hpmuseum.org/forum/thread-975...l#pid89345
~ 23s - HP 33s RPN http://www.hpmuseum.org/forum/thread-982...l#pid87566
~ 23s - HP 32s RPN 139.29718705 http://www.hpmuseum.org/forum/thread-975...l#pid89004
~ 23s - Ti 80, ti basic
~ 25s - TI 36x Pro
~ 26s - wp34s (DSE-based loop, DBLOFF), 139.2971870459241
~ 26s - HP 32sII RPN 139.29718705
~ 27s - wp34s (DSE-based loop, DBLON), 139.2971870459242385751150019615150
~ 27.5s - Hp 35s http://www.hpmuseum.org/forum/thread-982...l#pid87766
~ 30s - wp34s (S command, DBLOFF), 139.2971870459242
~ 31s - HP 42s , RPN program
~ 31s - Hp35s, RPN program
~ 33.8s - Sharp 516X
~ 34s - wp34s (S command, DBLON), 139.2971870459242385751150019615149
~ 36s - Sharp EL-5120 Solver 139.297187 http://www.hpmuseum.org/forum/thread-975...l#pid88303
~ 42.23s - DM41L 48 mhz , RPN program
~ 63s - dm15L 12mhz , RPN, 139.2971874
~ 65s - Casio fx-991ES PLUS sum function
~ 91s - Sharp PC-1401 Basic, 139.2971873
~ 91s - Olivetti M10, basic
~ 158s - hp 41CV (plain, fullnut, no turbo) , 139.2971874
~ 158s - Casio fx-2700P 139.29719 http://www.hpmuseum.org/forum/thread-975...l#pid91994
~ 313- 320s - hp15C , RPN , 139.2971874
~ 329s - hp-65 N=100, Result= 139.2971873
~ 340s - HP-67. RPN (139.2925695)
~ 434s - HP 55 , RPN program 139.2971873

max = 10
~ 1< seconds - Hp 50g, 2.15, RPN mode 13.7118350167 (approx mode)
~ 2s - Epson Hx-20 . max 7 digits precision. 13.7118
~ 2s - Epson Hx-20 . max 16 digits precision. 13.71183562278748
~ 3s - wp34s (DSE-based loop, DBLOFF), 13.71183501670439
~ 3s - wp34s (DSE-based loop, DBLON), 13.71183501670437880652763283584306
~ 3s - wp34s (S command, DBLOFF), 13.71183501670438
~ 3s - wp34s (S command, DBLON), 13.71183501670437880652763283584306
~ 3s - HP 32s RPN 13.71183502 http://www.hpmuseum.org/forum/thread-975...l#pid89004
~ 3s - HP 32sII RPN 13.71183502
~ 6s - dm15C 12mhz , RPN, 13.71183502
~ 9s - Sharp PC-1401 Basic, 13.71183501
~ 9.5s - Olivetti M10, basic
~ 16s - hp 41CV (plain, fullnut, no turbo), 13.71183502
~ 16s - Casio fx-2700P 13.711835 http://www.hpmuseum.org/forum/thread-975...l#pid91994
~ 29s - HP-25C (Woodstock): N=10, Result=13.71183501
~ 32s - hp15C , RPN, 13.71183502
~ 32s - hp-65, Time= 32 sec, Result= 13.71183501
~ 35s - HP-29C (Woodstock) N=10, Result=13.71183501
~ 43s - HP 55 , RPN program 13.71183501
~ 44s - HP-33C (Spice): N=10, Result=13.71183501 http://www.hpmuseum.org/forum/thread-975...l#pid88370
~ 46s - HP-34C (Spice): N=10, Result=13.71183501
~ 47s - sharp 506w (using X, formula memory F4 and M to sum) manually. 13.71183502 .
~ 56s - TI-53 "... at a rate of one iteration every 5.64 seconds. " http://www.hpmuseum.org/forum/thread-975...l#pid87729
~ 63.5s - TI-62, TI basic. 13.71183502
~ 99s - MK 54 , RPN program 13.711835
~ 100s - MK 61 , RPN program 13.711835
~ 110s - MK 56 , RPN program 13.711835

PS: I forgot how full of apps was the ti89 and how awful is the official command reference compared to the 50g AUR. But with the ti89 one finds everything online. Ex: https://education.ti.com/html/t3_free_co...sson2.html

edit1: some redditors helping - https://www.reddit.com/r/EngineeringStud...ur_trusty/

edit2: duplicated for versioning. http://www.wiki4hp.com/doku.php?id=bench...g_exp_root

Wikis are great, Contribute :)
Find all posts by this user
Quote this message in a reply
12-22-2017, 10:18 PM (This post was last modified: 12-22-2017 10:22 PM by Arno K.)
Post: #3
RE: Summation based test for calculators
I ususally am not that enthusiastic about these tests, but nevertheless I entered the sum on the HW-Prime, in CAS, time() said 12.582s, the result is, approx() used, 1395.34628774. Without approx it returns that sum of exact values. Then I switched to home, here the same commands needed 15.702s, which is some overhead produced by time, with TEVAL() only 0.223s were used to get the result you had.
Arno
Find all posts by this user
Quote this message in a reply
12-22-2017, 11:10 PM (This post was last modified: 12-22-2017 11:11 PM by pier4r.)
Post: #4
RE: Summation based test for calculators
(12-22-2017 10:18 PM)Arno K Wrote:  I ususally am not that enthusiastic about these tests

Yes they are nothing special, but they give an idea and moreover it is the first time I see the possibility to compare with scientific calculators (at least the last generation). Plus the formula is pretty easy to type. I did it on the ti89 and nspire that I almost never touch.

If I understood correctly, the 0.223s where obtained in the home using Teval(). Could you run with the same setup a test until 10k ?

I hope someone with a new dm42 does the test too. As well as some casio devices.

Wikis are great, Contribute :)
Find all posts by this user
Quote this message in a reply
12-23-2017, 12:57 AM
Post: #5
RE: Summation based test for calculators
(12-22-2017 09:41 PM)pier4r Wrote:  I did not forget this post (nor the topic about common calculators).

So as said the summation based test will be very similar to the savage benchmark and I wonder how many similar tests were developed on various forum or groups regarding calculators.

Anyway, inspired by the test done in the thread of Eddie linked in the 1st post, I decided to pick randomly some scientific functions to assemble a summation that may give an idea of the performances of a calculator for common math functions.

The idea is to have a loop working on an incrementing 'x' value that is somewhat independent from the step before (while the savage test uses the value picked from the step before) . Furthermore while the idea of using a function and its inverse is pretty neat (see savage benchmark), some calculators may be very carefully coded figuring this out and therefore simplifying the expression.

I am not that interested in the accuracy, as it is well analyzed in many other posts using other examples (especially by Dieter, that for me will always be the ULPs man).

Surely speed without accuracy is not that neat and so one could go on and build a metric that weights accuracy and speed, as done by a recent HHC (about egyptian fractions), anyway I will collect only timings.

Timings will be collected here, and then moved on the wiki4hp if there are enough of them.

People are encouraged to post timings and results.

So to recap the idea of this test is there because:
- with a summation with increasing x (not dependent on the previous computed value) also some advanced scientific calculators can be tested. See casio fx991EX.
- it is easier to type in and execute. It does not take much time (well unless one is forced to write a program to optimize some parts).
- it may be used with a metric that combines speed and accuracy (a problem would be to get the right accuracy for the problem).
- it avoids to use a function and its inverse to skip very careful optimizations done by the parser. (especially systems with CAS may use this)


Here is the assembled formula (I guess there can be an infinity of formulas that capture the idea I am going to expose).

\[ tan^{-1}(x) \]
So x can be incremented without problems. Picked among the trigonometric functions.

\[ sin \left (tan^{-1}(x) \right ) \]
To let the value increase towards 1, although, after a while, very slowly. Still trig.

\[ e^{ sin \left (tan^{-1}(x) \right ) } \]
to use either a common exponential or a logarithmic function without producing large numbers

\[ \sqrt[3]{e^{ sin \left (tan^{-1}(x) \right ) }} \]
To make the final number even smaller. Using powers.

Final formula to use (adapt the max 'x' value for speedy calculators. For example using 10k or 100k and so on)
\[ \sum_{x=1}^{1000} \sqrt[3]{e^{ sin \left (tan^{-1}(x) \right ) }} \]

As said, it is an arbitrary formula picking functions among some of the most common types of functions: trig, exp/ln, powers. Trying to keep the final numbers small. I may have picked some other functions easier to check (to compare the accuracy). This one is pretty nasty. Anyway as I said I am interested in timings assuming that the calculators are as precise as they could with the digits that they have.

I am really interested in what scientific calculators can do. I hope Eddie W. Shore sees this posts and contributes with a couple of models. Or jebem.

Results

max = 10000
1. ~ 20 sec - ti nspire handheld (2006). Degrees, float 12, approx. 13955.8579044 . OS 3.9.0.463

max = 1000
~ 0.223s - HW-Prime , Home, Teval().
~ 12.582s - HW-Prime , CAS, approx mode.
~ 24.5 seconds - Hp 50g, 2.15, RPN mode, DEG (done with the beautiful eq matrix and then EVAL. TEVAL did not help) . 1395.3462877 (approx mode)
~ 33.8 sec - Hp 50g, 2.15, RPN mode, DEG (quick userRPL FOR loop, surprisingly slower than the summation) . 1395.3462877 (approx mode)
~ 48 secs - ti 89 titanium, 12 digits, approx mode, degrees. OS 3.10 (code sum(seq((e^(...))^(1/3), x, 1, 1000))) . 1395.34628774

max = 10
1. 1< seconds - Hp 50g, 2.15, RPN mode 13.7118350167 (approx mode)
2. ~ 47 seconds - sharp 506w (using X, formula memory F4 and M to sum) manually. 13.71183502 . I did not yet found out what is the equivalent of the summation if I abuse the integration function. (that should use the trapezoidal rule)

PS: I forgot how full of apps was the ti89 and how awful is the official command reference compared to the 50g AUR.

Casio fx-115ES Plus: 1395.346288, time 8 minutes and 45 seconds
Visit this user's website Find all posts by this user
Quote this message in a reply
12-23-2017, 06:32 AM (This post was last modified: 02-03-2018 07:49 PM by brickviking.)
Post: #6
RE: Summation based test for calculators
On the HP-50G, I entered the formula into the Equation writer, popped it to the stack and hit EVAL. I get about 28.2" in exact mode and about 25.5" in Approx mode, giving a result of 1395.3462877 in both cases. I'm assuming that's the way I'm meant to do it? I also duplicated these results using ->NUM just to check, but got roughly the same measurements. EVAL in the equation editor also got a similar result to using EVAL on the stack entry.

It's interesting that my fx-9750gII (upgraded with 9860gII OS 2.04) beats out the 50G. My fx-9750gII gets about 22" for sum(1,1000) giving me 1395.346288 as a result. Then I tried it out on the fx-9750g+ (earlier generation) and got 1' 39.5". Just for laughs, I "souped up" the 9750gII with the FTune application (speeds up the CPU and bus clock) and got four seconds for 1,1000. Wow. I wonder what I'd get if I'd newRPL'd the 50G?

(Post 148)

BrickEdit: added gII to last instance of 9750, as it wasn't clear which machine I souped up. Could be a laugh trying to "soup up" the G+.

Regards, BrickViking
HP-50g |Casio fx-9750G+ |Casio fx-9750GII (SH4a)
Visit this user's website Find all posts by this user
Quote this message in a reply
12-23-2017, 07:56 AM
Post: #7
RE: Summation based test for calculators
You do realise that you are basically repeating the same calculation thousands of times?

The atan will quickly converge to pi/2 and so the summand to exp(1/3).
Find all posts by this user
Quote this message in a reply
12-23-2017, 09:05 AM
Post: #8
RE: Summation based test for calculators
(12-22-2017 11:10 PM)pier4r Wrote:  
(12-22-2017 10:18 PM)Arno K Wrote:  I ususally am not that enthusiastic about these tests

Yes they are nothing special, but they give an idea and moreover it is the first time I see the possibility to compare with scientific calculators (at least the last generation). Plus the formula is pretty easy to type. I did it on the ti89 and nspire that I almost never touch.

If I understood correctly, the 0.223s where obtained in the home using Teval(). Could you run with the same setup a test until 10k ?

I hope someone with a new dm42 does the test too. As well as some casio devices.

2.072s to get it to 10000.
Arno
Find all posts by this user
Quote this message in a reply
12-23-2017, 09:09 AM
Post: #9
RE: Summation based test for calculators
I've not tried this on the 34S and probably won't. The internal summation command will be slower than a short program that just adds the terms -- it goes to some extra effort to Kahan sum the terms in reverse order (which isn't ideal in this case).


Pauli
Find all posts by this user
Quote this message in a reply
12-23-2017, 09:59 AM (This post was last modified: 12-23-2017 10:28 AM by pier4r.)
Post: #10
RE: Summation based test for calculators
(12-23-2017 07:56 AM)AlexFekken Wrote:  You do realise that you are basically repeating the same calculation thousands of times?

The atan will quickly converge to pi/2 and so the summand to exp(1/3).

Yes, but the calculator should not know (although I am not sure what happens when x gets large and tan^-1 may report always the same value). Moreover as I wrote, the formula is pretty arbitrary and is one picked quickly that does not converge but also does not explode with larger values.

I was thinking about alternate series (using sin or cos) but at the end it would be similar. It is repeating thousands of times the same functions. At it is also what is wanted, so one gets an idea how those functions performs together, likely avoiding possible internal optimizations such as "a function and its inverse composed".

What is interesting is that with the 50g, in userRPL, I was not yet able to get a program faster than the summation. And the faster value is a summation, that is an algebraic object that is notoriously slower than pure userRPL!

Anyone with a 48 or other devices? Is it difficult to produce a RPN equivalent (for 41 and other systems) for the formula?

@Pauli: is it difficult to run an internal program on the 34s for this summation? If not, could you try?


edit: also newrpl should be interesting. Because the first second is done at 6 mhz and then the 50g switches to full 192 mhz according to what I read in the newRPL threads.

Wikis are great, Contribute :)
Find all posts by this user
Quote this message in a reply
12-23-2017, 11:38 AM (This post was last modified: 12-23-2017 11:43 AM by Gilles59.)
Post: #11
RE: Summation based test for calculators
NewRPL, precision 32 digits :

1000 Loop (FOR NEXT) : 8.2 sec
result : 1395.346287743423

12 digits = 6.022 sec
128 digits = 29.4 sec
Find all posts by this user
Quote this message in a reply
12-23-2017, 12:54 PM (This post was last modified: 12-23-2017 02:28 PM by pier4r.)
Post: #12
RE: Summation based test for calculators
Thanks gilles! Super neat.

Would be really nice to have the results (if possible with common entry mode and then in an optimized form, if it exists) for:
- a 41 version
- 41 CL
- dm41
- dm42
- hp 42s
- 35s
- a 48 version
- 12C (recent)
- 15C
- 50g with hpgcc (but this may be highly time consuming and not really usable in all day operations)
- 71B
- 67
- 34S
- Nspire CX
- A ti 84 (if possible recent)
- Casio primz
- Casio classpad (if possible 300 and then newer versions)
- casio 991ex
- Some sharp EL graphing or PC
- Also Nspire lua would be interesting (I can do it on the nspire handheld, although a CX should be better) but actually it would be very similar to a hpgcc version on the 50g, very unlikely to be used in all day operations since lua is clumsy to use on the nspire for actual math results.

Wikis are great, Contribute :)
Find all posts by this user
Quote this message in a reply
12-23-2017, 01:56 PM
Post: #13
RE: Summation based test for calculators
(12-23-2017 09:59 AM)pier4r Wrote:  I was thinking about alternate series (using sin or cos) but at the end it would be similar. It is repeating thousands of times the same functions.

I did not just mean "calling the same functions", but "calling the same functions with almost exactly the same (perhaps even the same) argument". I would say that any proper benchmark should call whatever functions it calls with a decent spread of the inputs. Using atan for (mainly) large values and then passing the almost constant result into sin (near an extremum!) pretty much garantees the exact opposite of that.

Just my 2c of course...
Find all posts by this user
Quote this message in a reply
12-23-2017, 02:27 PM (This post was last modified: 12-23-2017 03:05 PM by pier4r.)
Post: #14
RE: Summation based test for calculators
(12-23-2017 01:56 PM)AlexFekken Wrote:  I did not just mean "calling the same functions", but "calling the same functions with almost exactly the same (perhaps even the same) argument". I would say that any proper benchmark should call whatever functions it calls with a decent spread of the inputs. Using atan for (mainly) large values and then passing the almost constant result into sin (near an extremum!) pretty much garantees the exact opposite of that.

Just my 2c of course...

You also have a point. But for the objective of the test (not much accuracy, rather timing) I would suppose that whatever spread of the inputs would be more or less similar. If you have better compositions of functions (trig, exp/ln/, power, add/sub/mul/div ) let me know! It may be an interesting input to learn a thing or two.

At first I was exploring the hyperbolic functions (also because I never used them) but those may (a) be not so commonly implemented and (b) less often used.

Edit. Meanwhile I forced the sharp 506w to do a similar tasks like the summation using the integral calculation, although the result is off due to fixed coefficents.

The integral calculator in the sharp el-506w is done with the Simpson's rule:
[Image: 1VGLzjM.png]

This means that with a careful choice of the input parameters I can hope that the function is evaluated almost exactly in the 1000 points of the summation written above.
In particular:
a = 1
b = 1000
n = 500
N = 1000
h = 0.999
(if someone finds better parameters, please tell me!)

I do not know how the summation is done, but I assume that there are two internal registers for the sum of the "even" steps and the "odd" steps that then gets multiplied by 4 and 2. The computation is done in 4m 58s .

Wikis are great, Contribute :)
Find all posts by this user
Quote this message in a reply
12-23-2017, 05:35 PM
Post: #15
RE: Summation based test for calculators
Interesting report from reddit.

Quote:TI-89 (Original, not Titanium) OS 2.09
10000, ~62s
1000, ~62s
100, ~36s

TI-89 Titanium OS 3.10
10000, ~65s
1000, ~65s
100, ~37s

I repeated the 10000 and 1000 tests on both of the above calculators multiple times because it seems weird to me that both 1000 and 10000 would take the same amount of time, but the results were the same.

I will have to check with my ti 89

Wikis are great, Contribute :)
Find all posts by this user
Quote this message in a reply
12-23-2017, 07:21 PM
Post: #16
RE: Summation based test for calculators
summation test for n=1000
Casio fx-CG50 - approx 13.5s, 1395.346288
Casio fx-CP400 - approx. 6.5s, 1395.346288
Casio fx-991CE X - approx 1m 48s, 1395.346288
TI-Nspire CX CAS - after 47s resource exhaustion, cannot complete calculations,
Sharp EL-9950 - approx. 38s, 1395.346288
Find all posts by this user
Quote this message in a reply
12-23-2017, 08:48 PM
Post: #17
RE: Summation based test for calculators
Thanks for the answer!

Thanks to grsbanks that reported the first entry for the dm42. I asked him to check the results with the prime and 50g, because his devices are like 10x faster than the one reported by other users and by me.

The dm42 is pretty fast! I mean for a product done by a small company (compared to hp, ti, casio, sharp) plus one external indivisual (T.Okken) is quite amazing!

Wikis are great, Contribute :)
Find all posts by this user
Quote this message in a reply
12-23-2017, 08:49 PM
Post: #18
RE: Summation based test for calculators
(12-23-2017 07:21 PM)klesl Wrote:  TI-Nspire CX CAS - after 47s resource exhaustion, cannot complete calculations,

Thanks for the contributions. For the nspire be sure you are in approximate mode (and no pretty print when possible), otherwise the nspire tries to solve it symbolically. Like the ti89 as well.

Wikis are great, Contribute :)
Find all posts by this user
Quote this message in a reply
12-23-2017, 09:58 PM (This post was last modified: 12-23-2017 11:36 PM by Gilles59.)
Post: #19
RE: Summation based test for calculators
Casio FX603P
100 : 17 sec

HP15C LE
1000 : 29 sec

NewRPL HP50g hdw , 12 digits
10'000 : 56,51 sec

On the Prime
100'000 : 19,2 sec ( very fast ! )
Find all posts by this user
Quote this message in a reply
12-24-2017, 02:32 AM (This post was last modified: 12-24-2017 06:24 AM by AlexFekken.)
Post: #20
RE: Summation based test for calculators
(12-23-2017 02:27 PM)pier4r Wrote:  But for the objective of the test (not much accuracy, rather timing) I would suppose that whatever spread of the inputs would be more or less similar.
Incredible. We don't even seem to use the word "spread" with the same meaning... :-)

Wrt Simpsons rule, as someone has already pointed out in another thread: Simpson's rule is for student's only. Any serious, integration algorithm would be adaptive rather than using points with a fixed interval spacing. But perhaps it explains your obsession (in this thread) with summations.

My main personal suggestion for performance benchmarking, if you really, really feel that you have to create one, is that you do the exact opposite of what you are doing, i.e. the opposite of combining a bunch of ad hoc operations into a single benchmark and then hoping that your becnhmark becomes popular.

I would instead define and standardise a multitude of micro-benchmarks that test the simplest of operations in isolation (examples below). Then, on top of that, you can define your "Pier4r" aggregation benchmark(s), e.g. by taking some weighted average of the micro-benchmarks. That way others can define their own aggregated benchmarks, more suitable to their own use cases. Perhaps more importantly, it might help detect the relative strong and weak areas of each implementation (which can direct the implementers to the areas where they should look at improving things, or tap themselves on the back).

For example, to test the performance of loop summation I would simply write some code to calculate 1+1+1+1+1... etc, in a loop and another one to force non-integer arithmetic, e.g. sqrt(2)+sqrt(2)+sqrt(2)+sqrt(2)+ etc. I would experiment with, i.e. have different benchmarks for, getting the values out of variables versus using contstants as well. But I would certainly *avoid* calling transcendental functions (whose execution time would probably dominate the total execution time) and then suggest that I am testing the performance of a summation loop...

Wrt testing the performance of the transcendental functions, I would test them all (i.e. the ones you are interested in) in as much isolation as possible (e.g. separately time the overhead of the loops to set up the arguments; do *not* sum the values unless optimization forces you). And the basic trig functions already show that you may want multiple function-specific micro-benchmarks for every single function. For example, as some initial suggestions for the sine:

sin(x), x = -10..10 (0.001) in radians
sin(x), x = -1E6..1E6 (1) in radians
sin(x), x = -500..500 (sqrt(0.02)) in degress
sin(x), x = -1E8..1E8 (sqrt(20)) in degress
sin(x+iy), x = -10..10 (0.001), y = -10..10 (0.001) in radians
etc

EDIT: it might be a good idea to adjust the numbers above so as to have the same number of function evaluations per micro-benchmark. And to keep that number the same for the other functions.

Some of the thoughts (and correct or incorrect assumptions about implementations) behind the reason for having these different micro-benchmarks:
- Test a decent spread of unrelated (from the function's perspective) inputs
- Calculations for small and large arguments may perform very differently and not everyone may care about (very) large arguments (in this case) for their personalised aggregated benchmark
- Calculations in radians and degress may perform very differently (especially if the implementation has been careful to deal with rounding appropriately in each mode) and, again, people may only care about one of these modes
- Distinguish between real and complex calculations because the performance may be very different (perhaps predictably so for simple trig and exponential functions, but probably not for others, e.g. Gamma) and, again, not everyone will care about complex calculations.

Lots of words, encouraging you to do lots of work, by someone who isn't going to do that work. But well-intended nevertheless....
Find all posts by this user
Quote this message in a reply
Post Reply 




User(s) browsing this thread: 1 Guest(s)