WP-34S N-Queens Benchmark Based on Patrice Torchet's BITMAP Solution

[Claudio L.]

(10-18-2014 01:36 AM)xerxes Wrote:  ... I also see a disadvantage for some RPN versions,
because there is only one register for indirect addressing unlike the WP-34S, that makes the inner loop more complicated
than neccessary. So a RPN friendly translation of the code would be more fair.

And don't forget that the userRPL version for the HP28/48/49/50 is using lists, which is known to be very slow. It could very well leave all 8 values on the stack and use PICK/UNPICK instead of GET/PUT, taking advantage of the unlimited stack that is the most distinctive feature of this calculator series.
But people trying to solve a problem would probably use lists, so it is representative of the normal use. For the same reason people wouldn't normally use a bitmap solution, or be concerned with the number of indirect addressing done in a loop (making the BASIC version and its port for all targets also representative of normal use).
On the other hand, people using C would instantly get the C/regvars optimization without knowing, so it is representative as well.
I think your article is the most serious attempt ever made to have a benchmark for calculators, and wouldn't call it flawed in any way. For a cross-language, cross-platform benchmark with so many different hardware targets, it's nearly impossible to track every optimization on every target.
The fact that in 2014 people are still talking about it means overall it's a great benchmark.
I used it myself to compare newRPL speed to traditional RPL, so it's actually a development tool too.