Post Reply 
newRPL - build 1255 released! [updated to 1261]
06-23-2019, 11:57 AM
Post: #521
RE: newRPL - build 1255 released! [updated to 1261]
(06-23-2019 02:38 AM)Claudio L. Wrote:  The file looks OK. Did you use newRPL Desktop to flash it or SD card? Looks like your calc jumped out of place. Try flashing again, and put new batteries just in case. When flashing with newRPL desktop it doesn't check the battery voltage. Perhaps it wasn't sufficient to write to flash successfully.
Also, try the download again, it's not the first time a download gets corrupted.

I don't think it's a batteries issue because I've been able to reflash it back to build 1255 successfully with the same batteries. Anyway I tried with a fresh pack and got the same result. All my attempts were made using SD card.

I never attempted before using the PC version but now decided a try: using build 1255 the progress bar advanced to 20-25% then it became all black, returned white with an animation and the calc reset to a blank screen.

I panicked a bit because afterwards the calc seemed unresponsive to reset and I feared having bricked it. However I've been able to resurrect it and reflashed it to 1255.

To ensure that there was no problem with my cable I used the PC simulator to flash build 1255 on itself: this time the progress bar grew up to 50% (I assume now it's calibrated to 2MB RAM) and everything went OK - I'd exclude a cable problem.

I've redownloaded the file and checked it against the copy I've used. The files are identical and their checksum is:

newrplfw.bin (application/octet-stream) - 1045356 bytes
SHA-256 3c112f108f5e512d0f93237762777c01a41918e4d013a0e2984ee28136e272f6
Find all posts by this user
Quote this message in a reply
06-23-2019, 05:51 PM
Post: #522
RE: newRPL - build 1255 released! [updated to 1261]
(06-23-2019 11:57 AM)JoJo1973 Wrote:  
(06-23-2019 02:38 AM)Claudio L. Wrote:  The file looks OK. Did you use newRPL Desktop to flash it or SD card? Looks like your calc jumped out of place. Try flashing again, and put new batteries just in case. When flashing with newRPL desktop it doesn't check the battery voltage. Perhaps it wasn't sufficient to write to flash successfully.
Also, try the download again, it's not the first time a download gets corrupted.

I don't think it's a batteries issue because I've been able to reflash it back to build 1255 successfully with the same batteries. Anyway I tried with a fresh pack and got the same result. All my attempts were made using SD card.

I never attempted before using the PC version but now decided a try: using build 1255 the progress bar advanced to 20-25% then it became all black, returned white with an animation and the calc reset to a blank screen.

I panicked a bit because afterwards the calc seemed unresponsive to reset and I feared having bricked it. However I've been able to resurrect it and reflashed it to 1255.

To ensure that there was no problem with my cable I used the PC simulator to flash build 1255 on itself: this time the progress bar grew up to 50% (I assume now it's calibrated to 2MB RAM) and everything went OK - I'd exclude a cable problem.

I've redownloaded the file and checked it against the copy I've used. The files are identical and their checksum is:

newrplfw.bin (application/octet-stream) - 1045356 bytes
SHA-256 3c112f108f5e512d0f93237762777c01a41918e4d013a0e2984ee28136e272f6

I downloaded the file on a different PC from the same link in this forum, flashed using SD card and it works just fine here. Try clearing your RAM completely (On-A-F and all 3 shifts+clear).
Find all posts by this user
Quote this message in a reply
06-23-2019, 05:57 PM
Post: #523
RE: newRPL - build 1255 released! [updated to 1261]
(06-23-2019 11:57 AM)JoJo1973 Wrote:  I panicked a bit because afterwards the calc seemed unresponsive to reset and I feared having bricked it. However I've been able to resurrect it and reflashed it to 1255.

No need to panic, I have safeguards in place that if the program requests to erase or write to the bootloader it will reset the calc immediately. My calculator gave its life to make me realize the safeguards were needed. Now is next to impossible to brick a calc during firmware upgrade.
The blank screen is normal since you don't have a firmware, just press + and - during reset to bring up the menu.
Find all posts by this user
Quote this message in a reply
06-23-2019, 07:53 PM
Post: #524
RE: newRPL - build 1255 released! [updated to 1261]
(06-23-2019 05:57 PM)Claudio L. Wrote:  
(06-23-2019 11:57 AM)JoJo1973 Wrote:  I panicked a bit because afterwards the calc seemed unresponsive to reset and I feared having bricked it. However I've been able to resurrect it and reflashed it to 1255.

My calculator gave its life to make me realize the safeguards were needed. Now is next to impossible to brick a calc during firmware upgrade.

Fallen in the line of duty :,(

The memory clean did the trick! Build 1261 installed successfully!
Find all posts by this user
Quote this message in a reply
06-24-2019, 01:34 PM
Post: #525
RE: newRPL - build 1255 released! [updated to 1261]
(06-23-2019 07:53 PM)JoJo1973 Wrote:  The memory clean did the trick! Build 1261 installed successfully!

Great to hear! Some memory corruption must've happened while you were bug-hunting and crashing the calc on purpose to reproduce them. However, your service is crucial to the project so please don't stop doing it :-), just backup often to different files to make sure you don't lose anything.
Find all posts by this user
Quote this message in a reply
06-24-2019, 10:51 PM
Post: #526
RE: newRPL - build 1255 released! [updated to 1261]
(06-24-2019 01:34 PM)Claudio L. Wrote:  Great to hear! Some memory corruption must've happened while you were bug-hunting and crashing the calc on purpose to reproduce them. However, your service is crucial to the project so please don't stop doing it :-), just backup often to different files to make sure you don't lose anything.

Oh, I've found some interesting ways to crash the calc Wink)) , in fact there's one I want to verify with the new build... I'll keep you informed! Wink

BTW I probably found the culprit of the memory corruption: a formerly good dirobject that now hangs the calc when I enter it; curiously I can retrieve its contents if I use a program to enter it and RCL a variable in it.

I tried to PGDIR it, but the calc errors with an Out of Memory error. It's not a problem since I can wipe out the memory and have plenty of backups: one of them will be surely ok.

My question is if is it possible in the future to add a command like UserRPL PINIT that returns the memory to a sane state so that a corrupted object can be safely purged (recovery of the corrupted object wouldn't be important)?
Find all posts by this user
Quote this message in a reply
06-25-2019, 03:06 AM
Post: #527
RE: newRPL - build 1255 released! [updated to 1261]
(06-24-2019 10:51 PM)JoJo1973 Wrote:  
(06-24-2019 01:34 PM)Claudio L. Wrote:  Great to hear! Some memory corruption must've happened while you were bug-hunting and crashing the calc on purpose to reproduce them. However, your service is crucial to the project so please don't stop doing it :-), just backup often to different files to make sure you don't lose anything.

Oh, I've found some interesting ways to crash the calc Wink)) , in fact there's one I want to verify with the new build... I'll keep you informed! Wink

BTW I probably found the culprit of the memory corruption: a formerly good dirobject that now hangs the calc when I enter it; curiously I can retrieve its contents if I use a program to enter it and RCL a variable in it.

I tried to PGDIR it, but the calc errors with an Out of Memory error. It's not a problem since I can wipe out the memory and have plenty of backups: one of them will be surely ok.

My question is if is it possible in the future to add a command like UserRPL PINIT that returns the memory to a sane state so that a corrupted object can be safely purged (recovery of the corrupted object wouldn't be important)?

There's no need for a specific command. Every time you turn the calculator on, or restore a backup, the entire memory is scanned and every single object is checked for validity. Of course, validity depends on the object type: a real number for example, as long as it has consistent headers is deemed valid, if some digits got corrupted it can't be detected (I'd have to waste a lot of memory in storing a CRC for every object). Any invalid object is purged (but this is extremely rare, hardly ever happened). Directory entries are scanned and checked for validity names as well. Invalid names with valid objects are given an 8-letter name.
As I find more creative ways to corruption memory, I also find ways to improve the sanity checks.
Now I don't want this to sound like newRPL is unstable, memory corruption is extremely rare, and mostly due to testing unfinished, buggy code that 90% of the time never gets to the final user. In your case, you are trying to crash it on purpose to help the project, and your success means you might corrupt your memory every now and then.
Because backups do sanity checks, most of the time recovering from backup fixes any issues, even if the backup was done after the corruption happened.

I'll see if I can harden detection of malformed directory structures during the sanity checks, can you describe in more detail which things crash your bad directory and which ones work OK? This way I can at least try to guess what could be wrong and add a sanity check for it.
For example, you mentioned entering the directory crashes. I need to know exactly what happens in all of the following cases:
Entering from the menu?
Typing the name then doing EVAL?
Executing the unquoted name within a program?
Going into the bad directory then doing UPDIR?
In the parent directory, type the name quoted, then do RCL, do we get a DirObject? If so, go to HOME, type another name and do STO. This does a deep copy of the entire subtree, does it crash? If not, is the copy having the same issues as the original?
Going into the directory then doing PATH, does it crash?

To avoid annoying our readers, perhaps you should create the issue in the bug tracker and add all this info.
Find all posts by this user
Quote this message in a reply
06-25-2019, 11:21 PM
Post: #528
RE: newRPL - build 1255 released! [updated to 1261]
(06-25-2019 03:06 AM)Claudio L. Wrote:  
(06-24-2019 10:51 PM)JoJo1973 Wrote:  Oh, I've found some interesting ways to crash the calc Wink)) , in fact there's one I want to verify with the new build... I'll keep you informed! Wink

BTW I probably found the culprit of the memory corruption: a formerly good dirobject that now hangs the calc when I enter it; curiously I can retrieve its contents if I use a program to enter it and RCL a variable in it.

I tried to PGDIR it, but the calc errors with an Out of Memory error. It's not a problem since I can wipe out the memory and have plenty of backups: one of them will be surely ok.

My question is if is it possible in the future to add a command like UserRPL PINIT that returns the memory to a sane state so that a corrupted object can be safely purged (recovery of the corrupted object wouldn't be important)?

There's no need for a specific command. Every time you turn the calculator on, or restore a backup, the entire memory is scanned and every single object is checked for validity. Of course, validity depends on the object type: a real number for example, as long as it has consistent headers is deemed valid, if some digits got corrupted it can't be detected (I'd have to waste a lot of memory in storing a CRC for every object). Any invalid object is purged (but this is extremely rare, hardly ever happened). Directory entries are scanned and checked for validity names as well. Invalid names with valid objects are given an 8-letter name.
As I find more creative ways to corruption memory, I also find ways to improve the sanity checks.
Now I don't want this to sound like newRPL is unstable, memory corruption is extremely rare, and mostly due to testing unfinished, buggy code that 90% of the time never gets to the final user. In your case, you are trying to crash it on purpose to help the project, and your success means you might corrupt your memory every now and then.
Because backups do sanity checks, most of the time recovering from backup fixes any issues, even if the backup was done after the corruption happened.

I'll see if I can harden detection of malformed directory structures during the sanity checks, can you describe in more detail which things crash your bad directory and which ones work OK? This way I can at least try to guess what could be wrong and add a sanity check for it.
For example, you mentioned entering the directory crashes. I need to know exactly what happens in all of the following cases:
Entering from the menu?
Typing the name then doing EVAL?
Executing the unquoted name within a program?
Going into the bad directory then doing UPDIR?
In the parent directory, type the name quoted, then do RCL, do we get a DirObject? If so, go to HOME, type another name and do STO. This does a deep copy of the entire subtree, does it crash? If not, is the copy having the same issues as the original?
Going into the directory then doing PATH, does it crash?

To avoid annoying our readers, perhaps you should create the issue in the bug tracker and add all this info.

Ok, I'm preparing a bug report answering all the question provided. I can confirm that newRPL is pretty robust: it's likely that the object stayed corrupted for a while, and overall the system was unaffected and worked very reliably.

Moreover, as you remember I failed to flash build 1261 but succeeded to reflash build 1255 after each failed attempt: every time I recovered programs and settings without any loss (I can safely assume that it was the pre-existence of the corrupted dir the cause of the failures) - quite an accomplishment!
Find all posts by this user
Quote this message in a reply
06-26-2019, 01:58 PM
Post: #529
RE: newRPL - build 1255 released! [updated to 1261]
(06-25-2019 11:21 PM)JoJo1973 Wrote:  Moreover, as you remember I failed to flash build 1261 but succeeded to reflash build 1255 after each failed attempt: every time I recovered programs and settings without any loss (I can safely assume that it was the pre-existence of the corrupted dir the cause of the failures) - quite an accomplishment!

Thanks!, I put a lot of effort to make sure TTRM messages are a thing of the past, there's no flashy graphics advertising that effort, no section on the wiki, it's completely transparent to the final user with no delay or bragging messages (like "we just scanned your computer and found no security threats!", which I find most annoying). There's just a sad file called 'sanity.c' buried in the middle of the source code. I'm glad somebody noticed that after a bad crash, it used to be normal to lose it all, and now is not.
Find all posts by this user
Quote this message in a reply
Post Reply 




User(s) browsing this thread: 1 Guest(s)