Calculators that allow direct operations on data stored on persistent storage?

[DavidM]

There's a case to be made for simply breaking down your data into smaller "chunks". Those chunks could be based on some logical grouping, such as months or weeks. I'm not entirely sure what your data represents, but it appears that it might be something along the lines of 19 data points for each designated day. If that's the case, 31 day's worth of data would consume about 5K in uncompressed format.

While not a space saver, you might also consider using arrays to store the data chunks instead of lists. Some of the built-in commands for arrays might be very useful for you. Converting the data from your existing format wouldn't be too difficult using AXL along with some of the ListExt commands.

Just to give you an idea of one possible way to do that, I copied your lists onto the stack of an emulated 50g and then performed the following steps:

Code:

20 →LIST
LCLLT
20 LSDIV
AXL

The result is a 257-row by 20-column array with the dates in the first column and each subsequent column containing the apparent data values for those dates. While there's no space savings in doing this, it does make it relatively easy to extract the data of a specific day if you know its offset in the array:

Code:

13 ROW- NIP

...yields the following in stack level 1:

Code:

[ 20.062017 5. 4. 0. 2. 4. 3. 3. 4. 2. 3. 0. 4. 4. 3. 4. 2. 0. 0. 360. ]

Extracting columns for date ranges and storing as ΣDAT would allow you to use some of the statistical functions that Carsen was mentioning. If you prefer the data in list format, AXL will convert the given row (or column) into a list. It would then be a simple matter, for example, to compute the average of column 20 over a given range of rows.

Finally, I know you're not interested in trying to use an alternate format for the data right now, but if you should ever reconsider that, here's a couple of thoughts:

1) It looks as though the first list may be a series of dates. Furthermore, it appears that those dates may actually be in sequence. You may not even need to notate the dates at all, and simply rely on the relative positions of the data elements to imply the date instead. For example, the first data element in each list may be thought of as being associated with the "base date" of 2017-06-08, the second being associated with 2017-06-09, etc. Determining the date for a given column is then a simple computation of "8.062017 (row-1) DATE+".

If all dates aren't actually in the list (I didn't inspect each one for gaps), you could still save some space by saving the offset from your base instead of the actual date itself. In other words, 0 instead of 8.062017, 1 instead of 9.062017, ... 256 instead of 19.022018, etc.

2) It looks like all but the first and last lists contain only single digit numbers. Couldn't those be combined easily into groups of (say) 10 at a time? Using the above row as an example, the first entry would be "5402433423". That would cut down considerably on the storage requirements. And since I know you have the ListExt library, it would be simple to extract the individual values from the combined number.

Just providing more things to think about!