Re: Good (simple?) calculation for benchmarking?
Message #8 Posted by Mike Morrow on 26 Feb 2013, 3:05 p.m.,
in response to message #1 by Jedidiah Smith
An iterative calculation of some sort is the only usable type of benchmark, preferably one that uses the results of the previous iteration in the next to produce a final result that is useful to gauge the propagation of truncation and accuracy errors. For scientific applications that likely make up a large portion of the applications for our HPs, it is logical to include various standard mathematical functions in the loop, in preference to some other benchmarks that spend most of the execution time moving data to and fro, and operating on them with basic arithmetic. However, that type of benchmark would provide better results for applications that use those capabilities, such as matrix manipulation intensive applications.
For several decades one of the benchmarks often utilized has been
the Savage Benchmark. It was useful in the early personal computing
era when desktops were much slower than many handheld devices are
today.
One common version of this benchmark in RPL follows that is based
on the exact series of operations that were easily programmed using
the typical syntax of BASIC on early personal computers for the
statement 10 A=TAN(ATN(EXP(LOG(SQR(A*A)))))+1 .
Note: "sqrt" below is the square root symbol.
<<
RAD
1.
1. 2499. START
SQ sqrt LN EXP ATAN TAN 1. +
NEXT
>>
The HP 50G (version 2.15) completes the above in about 65 seconds
with this result: 2499.99948647 .
I prefer a slight modification that produces more meaningful results
that takes advantage of the versatility of RPL syntax by:
(1) Arranging for exactly 2500 iterations (first version does 2499),
(2) Taking the square root, followed by the square (first version
reverses this order and thus operates only on integer values),
(3) Add steps to automatically indicate the execution time in
seconds, after iterations are complete,
(4) Add step to return calculator to DEG mode, and
(5) Add 800 Hz two-second beep at end.
<<
RAD TICKS
0.
1. 2500. START
1. + sqrt SQ LN EXP ATAN TAN
NEXT
SWAP TICKS SWAP - B->R 8192. /
DEG
800. 2. BEEP
>>
The HP 50G (version 2.15) completes the above in about 65 seconds
with this result: 2499.99946106 (and 64.8814697266 seconds for
this run).
Note: For the 49G+ and 50G, one will not want the calculations done
using the binary integer types that exist on those machines. That's
the reason for the decimal point appearing with each value in the
programs above. Remove those decimal points, and execution times
increase very significantly.
Very simple benchmarks, such as a loop that just increments a value by 1 each iteration for a specified interval, produce very inaccurate estimates of the practical or typical speeds of two machines relative to each other.
For example, Free42 completes the Savage Benchmark in about 1.5 seconds on my Android smartphone, 400 times faster than a real HP 42S. But just counting the iterations of a X=X+1 type of loop for 60 seconds on these two machines indicates that Free42 on my Android is a whopping 7800 times faster than a real HP 42S. That's a factor of almost 20 in the comparative results caused simply by the choice of benchmark method!
There is no "universal" benchmark that is meaningful for every application.
Edited: 26 Feb 2013, 4:40 p.m.
| 
Go back to the main exhibit hall