[Help]Hp 50g : working with larger-than-memory datasets

[pier4r]

I apologize if this problem was already discussed and solved, in that case a link to the discussion would be appreciated since the only one that I found with a brief search is this.

The use case:
- Imagine to have a list of numbers (or rows containing more than one value) that cannot be saved in the memory (~230kb on the hp 50g). Or it can be saved in the memory but it would leave few KB available for the computations.

The problem:
- At least on the hp50g, a list like this would not be a big problem to store. One has the internal flash available (Port2 on my 50g has still 600kb+ of memory free) or the sd card (mine has still 2gb +). The problem is to use this list for computations, since it cannot fit the ram.

A possible solution:
- I remember I have read something about "packing" data in blocks to work with smaller blocks every time that would fit the ram leaving enough space for computations. This was a technique used for games on older game consoles like PS1, gameboy, nes, that nevertheless are computing systems (also quite powerful).

Now I do not know if a similar technique was developed for the hp 50g (or calculators using useRPL/sysRPL ) to split data in smaller blocks before feeding it to a program "on the fly". More in detail, I do not know if I save, say, a list on the FLASH I can access only part of it, without splitting it before the computation in smaller lists.

So what I'm searching for is, in the best case, something like this

Code:


- Given list X stored on Port2 or Port3 on the 50g
- read only elements from N to N+K of list X, 
  without loading the entire X in ram, 
  but only the read elements and the list structure.

In the second best case I'm searching for something like

Code:


- Given list X stored on Port2 or Port3 on the 50g
- split the list X in smaller list X1,X2,....,XN of size max S, 
  then feeding one after another in the main program in a "transparent way". 
  (Like it is a unique list)

(I do deem this unlikely to be possible, without modifying the main program)

If those methods/programs do not exists, of course it will be a pleasure for me to try to make little programs implementing them.

One could say: why don't you use a computer that has more capabilities?

That's true, but due to the math library already present (and partially known by me) on the 50g, implementing a solution is way faster than searching for proper math library for computer programming languages (dunno SciPy or equivalents for Java, C#, PHP, Lua and what not), learning to use them, debug the implementation, etc. While the 50g is way more handy for this with built in functions and well documented additional libraries.
This also because the computation time is not a big deal, at least not yet.

Then there is a second reason: I don't see why the 50g cannot sweat making use of the Port2/3 on larger-than-memory datasets. Sure, with the emulation layer the 50g works a lot already for whatever action, but with the newRPL coming (great project!) it will be less work. Anyway even with the current emulation layer, it can work more and I don't feel that it lays around doing a little, but I feel that the 50g earns his period of little usage after working for long time.