Can HP Prime be faster? - firmware performance comparison

[komame]

A few years ago, while writing various programs and experimenting with the HP Prime, I noticed that after one of the firmware updates, something that used to work smoothly suddenly started lagging.
I know that the HP Prime can execute programs at different speeds, even when running the same program multiple times. However, this has already been discussed on this forum, and that's not what I want to talk about now. I observed that different firmware releases perform certain operations at different speeds.

Several years have passed, and I finally found the time to dedicate to this issue, and believe me, it took me many hours.

All the actions described below were performed on a real HP Prime G1 rev. A.

Firstly, I wrote a program in HPPPL and prepared an equation - a sum to be calculated:

Code:

EXPORT PERF_TEST()

BEGIN
  local s_tck,e_tck,x,y,c,lst,st;
  PRINT();

// ===> test 1 <===
// list manipulation
  PRINT("=== TEST 1: list manipulation ===");

  TICKS▶s_tck;
  FOR c FROM 1 TO 10 DO
    lst:=MAKELIST(X,X,1,10000);
    FOR x FROM 1 TO 10000 DO
      lst(x):=10000-x;
    END;
  END;
  e_tck:=TICKS;
  PRINT("TIME: " + (e_tck-s_tck) + " ms");

// ===> test 2 <===
// string manipulation
  PRINT("=== TEST 2: string manipulation ===");
  TICKS▶s_tck;
  st:="";
  FOR x FROM 1 TO 800 DO
    st:=st+"ABCDEFGHIJKLMNOPQRSTUVWXYZ";
  END;
  REPEAT
    st:=RIGHT(st,DIM(st)-1);
  //PRINT(st);
  UNTIL DIM(st)<2;
  e_tck:=TICKS;
  PRINT("TIME: " + (e_tck-s_tck) + " ms");

// ===> test 3 <===
// fill screen
  PRINT("=== TEST 3: fill screen ===");
  RECT_P(0); //clear screen
  TICKS▶s_tck;
  FOR y FROM 1 TO 239 DO
    FOR x FROM 1 TO 319 DO
      PIXON_P(x,y,255+2*x+y);
    END;
  END;
  e_tck:=TICKS;
  PRINT("TIME: " + (e_tck-s_tck) + " ms");

  PRINT("=> done");  
END;

   

This program performs three tests:
operations on lists
operations on text strings
filling the screen with pixels in shades of blue color (pixel by pixel)

Time measurements are calculated as the difference between invoking the TICKS command.
For summation, the TEVAL command was used.

To ensure the most reliable results, it was necessary to eliminate any additional factors that could affect the performance of the HP Prime. We know that having more programs in the program catalog can impact performance, so before each test run on a specific firmware (also after each flashing), I always restored the factory settings (F+C+O reset), clearing everything from memory.
Then, I connected the HP Prime to the HP Connectivity Kit momentarily and uploaded only the above PERF_TEST program, making it the only program in the catalog. After that, I disconnected the HP Prime from USB, as the connection negatively affects performance due to continuous communication with the PC.
With a clean stack, I ran this program three times directly from the program catalog for each firmware. After completing the first run, I pressed ON key and immediately started the second run, then the third run in the same way - without leaving the program catalog, while simultaneously recording the results in a spreadsheet on my PC. Next, I went to the HOME screen and ran the summation according to the above example - also three times, but this time there were three consecutive stack entries:
   

This concludes the test for a single firmware.

I noticed that even if I restore the factory settings twice (F+C+O) and load this program into memory after each restoration, the execution time might differ minimally. However, these differences were rather at the level of 1-2% (which sometimes still resulted in 150ms, but nevertheless, these differences fell within the tolerance range for each firmware). If someone were to attempt to replicate my test, it's possible that instead of 8700 ms, they could get results like 8580 ms or 8790 ms. However, running the test multiple times in a row always yielded similar results.

This program and the summation calculation were run on many firmwares. I had to reflash my G1 over 20 times in total. Sometimes, the results were so surprising that I went back to previously tested versions to run the test again and make sure the initial measurement wasn't done incorrectly.

I performed some tests on beta versions as well. If anyone considers these cases unreliable, they can simply ignore them.


RESULTS.
Here are the performance differences of the firmware versions presented as percentages relative to the current version 14730:
   
From left to right, we can see the current firmware 14730, and the further to the right, the older releases. As you can see, only in a few cases were certain operations slower than the current firmware, the vast majority are faster.

In an extreme case, filling the screen with pixels (yellow series) in firmware 7820 is almost 48% faster than in the current version 14730. This is a colossal difference! It's visible to the naked eye.

During that time (2015), I was developing a game for the HP Prime, and between firmware versions 8151 and 10637 there was a sudden drop in graphics performance. I mentioned this on the forum then, but I didn't have enough time to prove it with these tests. The first reason was that back then, the performance significantly dropped during drawing (as seen for firmware 10637). The second reason was that when pressing a key, the performance could further drop by as much as 30-35% while holding down the key. Coupled with graphic operation issues, this performance drop became even more pronounced. However, Tim explained to me that there are areas within the firmware that HP programmers don't have control over. If that's indeed the case, there might not be much that can be done regarding this specific keyboard handling issue, but there could still be potential to address other instances where performance used to be better, and it's unclear why it's not anymore

Of course, my tests don't cover everything, and they could be more detailed, including operations on matrices, trigonometric operations, or more advanced graphics operations. Maybe then there would be more conclusions to discuss. What I did was a very time-consuming process for me, so I limited it to these four factors.

I'd like to draw attention to how much these results fluctuate over time. This could suggest that there's still a lot that can be done with the performance of the HP Prime. The summation calculation (green series) even had an increasing trend, only experiencing a minor dip in the latest versions. I'm not referring to the sharp drop in 14588, as that's clearly a bug; something was off with the calculations in that firmware. Even calling the summation calculation twice on the stack caused a crash, and in this case, measurements were done after a soft-reset for each run, as there was no other way (I think this isolated result can be ignored). Irrespective of everything, this part of the test exhibited the most unstable results. It's likely that these areas undergo the most changes.

During the tests, I also noticed a change in pixel filling between firmware versions 12066 and 13333:
   
You should know that this is not visible in the emulator; the effect can only be observed on a real HP Prime. I had to take photos of the screens to show you this comparison.

The difference in rendering color shade thresholds for individual values is clearly visible. In version 13333, they are very smooth. If this had an impact on performance, I think it would be beneficial if the programmer had the option to choose the rendering mode. However, looking at the graph (yellow series), we can see that version 13333 experiences a significant drop in the screen-filling test, yet version 13865 shows a substantial increase despite using this smoothing technique. This serves as further evidence that there is still much that can be done in this area, as the current firmware performs this nearly 14% slower.

Notice that this doesn't just apply to running an HPPPL program; since the equation also significantly changes its execution time, I suspect that tasks like plotting graphs or symbolic calculations (CAS) might also vary in execution time depending on the firmware version.

WHAT ABOUT THE G2?
I don't own an HP Prime G2, so I don't know how it works "under the hood" or how different from G1 the core system is. However, looking at firmware version 13865, which probably was the first for G2 and as we can see, one of the most efficient, especially from a G1 perspective (considering all measured factors), it's quite possible that the G2 could work faster too. Maybe someone could conduct a similar test for the G2.
I would also appreciate tests on other G1 models; a comparison between the current firmware and 7820 or 8151 would suffice.

I hope these pieces of information will be useful to someone and will positively impact the ongoing development of HP Prime calculators.

I'm also attaching a spreadsheet with raw data for individual tests (results in milliseconds).

That's all from my side.