50g: copy all stored objects to SD card individually
10-17-2016, 08:32 AM (This post was last modified: 10-17-2016 10:55 AM by matthiaspaul.)
Post: #21
 matthiaspaul Senior Member Posts: 385 Joined: Jan 2015
RE: 50g: copy all stored objects to SD card individually
(10-16-2016 07:51 PM)DavidM Wrote:  ...and yes, copying the text and pasting into a code block creates some confusion due to mixing code pages again.
Thanks a lot! Unfortunately, I have difficulties explaining some of the differences between these two views - some seem to make sense in the first, some in the second view.

Are you familiar with using a disk editor? I guess we could more reliably see what's actually going on under the hood by bypassing Windows and having a look at a hex dump of the on-disk structures. If you'd want to give it a try, I can recommend HxD, a small free disk editor for Windows: https://www.mh-nexus.de/en/hxd/ . In "Extras" you'd have to open a view of the FAT partition (drive I: in your case) as a logical volume, and then use "Search" to find f.e. the directory entry holding the string "MIXCASE". A hex dump of perhaps about hundred bytes before and five-hundred bytes after that location would be helpful.

Greetings,

Matthias

--
"Programs are poems for computers."
10-17-2016, 04:42 PM
Post: #22
 Claudio L. Senior Member Posts: 1,505 Joined: Dec 2013
RE: 50g: copy all stored objects to SD card individually
(10-16-2016 05:30 PM)DavidM Wrote:
(10-16-2016 05:24 PM)DavidM Wrote:  Just tried this, ...

One more thing I just noticed when looking at DIR/X results:
Code:
10/16/2016  01:14 PM                14 LCASE        lcase 10/16/2016  01:14 PM                14 lcase_~1     lcase;;;;;;;

Interesting that the SFN is showing as lower case here, but the file was still accessible and I had no trouble deleting it with Win7. So perhaps the problem is actually still there and Win7 was handling it anyway.

So it seems it still creates some lowercase SFN, unless the SFN is stored as uppercase but displayed lowercase because the lowercase flag is set?
What would directly accessing the short name do?
DIR lcase_~1
DEL lcase_~1
10-17-2016, 04:47 PM
Post: #23
 Claudio L. Senior Member Posts: 1,505 Joined: Dec 2013
RE: 50g: copy all stored objects to SD card individually
(10-16-2016 07:26 PM)matthiaspaul Wrote:  :3:"SPECIÄL" STO no reason to create a LFN entry but some system may do

There's one reason: The character >127. Creating an LFN is the only way to store it as Unicode. I modified newRPL to always create LFN's in cases like this, just to eliminate the ambiguity of using CP translation. Technically not needed if you are willing to put up with old CP mess, though.
10-17-2016, 05:04 PM
Post: #24
 DavidM Senior Member Posts: 725 Joined: Dec 2013
RE: 50g: copy all stored objects to SD card individually
(10-17-2016 08:32 AM)matthiaspaul Wrote:  Are you familiar with using a disk editor? I guess we could more reliably see what's actually going on under the hood by bypassing Windows and having a look at a hex dump of the on-disk structures.

Deep back in my history I created a disk imaging app for a large well-known computer company in the late '80s. So, yes, I've got some familiarity with disk editors (though not FAT-based, and it's been a while).

I'll check out the app you mentioned and start digging around with it on the SD cards. I was already thinking of doing that anyway, as I've been trying to remember the sequence of things I did prior to the corruption of the SD card file system. I'd love to be able to recreate that so as to know what to avoid in the future.
10-17-2016, 05:21 PM
Post: #25
 DavidM Senior Member Posts: 725 Joined: Dec 2013
RE: 50g: copy all stored objects to SD card individually
(10-17-2016 04:42 PM)Claudio L. Wrote:  So it seems it still creates some lowercase SFN, unless the SFN is stored as uppercase but displayed lowercase because the lowercase flag is set?
What would directly accessing the short name do?
DIR lcase_~1
DEL lcase_~1

Neither "DIR lcase_~1" nor "DIR LCASE_~1" succeeded, both failing with "File Not Found" (same with DEL, msg: "Could Not Find I:\<fn>").
Both "DIR lcase" and "DIR LCASE" succeeded for the other file.

I'm starting to wonder if DIR /X is populating the LFN and SFN fields with presumptions in cases where the actual data is questionable. All the more reason to use the disk editor to look at the actual stored strings.

10-17-2016, 06:15 PM (This post was last modified: 10-19-2016 07:10 PM by matthiaspaul.)
Post: #26
 matthiaspaul Senior Member Posts: 385 Joined: Jan 2015
RE: 50g: copy all stored objects to SD card individually
(10-17-2016 04:47 PM)Claudio L. Wrote:
(10-16-2016 07:26 PM)matthiaspaul Wrote:  :3:"SPECIÄL" STO no reason to create a LFN entry but some system may do
There's one reason: The character >127. Creating an LFN is the only way to store it as Unicode. I modified newRPL to always create LFN's in cases like this, just to eliminate the ambiguity of using CP translation. Technically not needed if you are willing to put up with old CP mess, though.
I agree, at least in this context. (I personally try to avoid *VFAT* LFNs whenever possible, for I consider the way they were retrofitted into the FAT filesystem a design "nightmare". There would have been more robust methods to integrate long file names into FAT while still maintaining full backward compatibility - and without speed or size penalties. But keeping FAT volumes free of VFAT LFN entries only makes sense for as long as the whole medium can be kept free of them realistically (for example on local partitions using partition IDs not recognized by systems using VFAT LFNs). In the case of an exchange medium of a calculator using a custom character set it would be unrealistic to maintain this goal. Actually, given that the RPL character set is based on ISO/IEC 8859-1 and thus many codepoints are identical to Unicode codepoints, being able to store characters in Unicode is highly desirable - assuming a bug-free implementation... ;-)

So, my comment wasn't meant to imply that this would be a bad idea, just that it is technically not required from the viewpoint of the filesystem. And in fact, as we can see from the DIR /X output above, the calculator does not create a LFN entry in this case.

Greetings,

Matthias

--
"Programs are poems for computers."
10-17-2016, 06:26 PM
Post: #27
 matthiaspaul Senior Member Posts: 385 Joined: Jan 2015
RE: 50g: copy all stored objects to SD card individually
(10-17-2016 04:42 PM)Claudio L. Wrote:  So it seems it still creates some lowercase SFN, unless the SFN is stored as uppercase but displayed lowercase because the lowercase flag is set?
I found another description matching our observation in a bug-report by James M. Prange for firmware 2.09:

http://bugs.hpcalc.org/show_bug.cgi?id=235
Quote:Invalid "short filename" aliases including lower-case letters are sometimes used with "long filename" directory entries on the MMC/SD card. This doesn't seem to cause any problem on the calculator itself (the filer displays the alias in all-capitals), and MS Windows Explorer correctly displays the long filename, but
disk checking utilities such as MS ScanDisk or Norton Disk Doctor detect the errors (and optionally correct them).

When a filename doesn't meet the no-lower-case "8.3" short filename requiremnents, the system generates a valid short filename alias for the directory entry and writes the original long filename (in unicode) in "slots" above the short filename entry. For example, with a long filename of "LongFileName", a unique short filename alias of (assuming that it doesn't already exist) "LONGFI~1" should be generated, but the calculator fails to uppercase the letters, using the invalid alias "LongFi~1" instead.

But I notice that when the length of the name and extension meet the requirements and the only problems are lower-case letters, (such as in "LongName.Txt"), then a valid short filename alias with all letters upper-cased (such as LONGNAME.TXT) is generated.

Regards,
James
Greetings,

Matthias

--
"Programs are poems for computers."
10-17-2016, 07:29 PM
Post: #28
 DavidM Senior Member Posts: 725 Joined: Dec 2013
RE: 50g: copy all stored objects to SD card individually
(10-17-2016 08:32 AM)matthiaspaul Wrote:  A hex dump of perhaps about hundred bytes before and five-hundred bytes after that location would be helpful.

It appeared that all of the directory entries existed in a single sector for this particular scenario. Using a width of 32 bytes seemed to nicely align with entries, so I used that for the following output:
Code:
Offset(h) 00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F 10 11 12 13 14 15 16 17 18 19 1A 1B 1C 1D 1E 1F 000F5000  41 6C 00 63 00 61 00 73 00 65 00 0F 00 E6 00 00 FF FF FF FF FF FF FF FF FF FF 00 00 FF FF FF FF  Al.c.a.s.e...æ..ÿÿÿÿÿÿÿÿÿÿ..ÿÿÿÿ 000F5020  4C 43 41 53 45 20 20 20 20 20 20 20 00 00 02 69 51 49 51 49 00 00 02 69 51 49 03 00 13 00 00 00  LCASE       ...iQIQI...iQI...... 000F5040  41 6C 00 63 00 61 00 73 00 65 00 0F 00 D0 3B 00 3B 00 3B 00 3B 00 3B 00 3B 00 00 00 3B 00 00 00  Al.c.a.s.e...Ð;.;.;.;.;.;...;... 000F5060  6C 63 61 73 65 5F 7E 31 20 20 20 20 00 00 03 69 51 49 51 49 00 00 03 69 51 49 04 00 13 00 00 00  lcase_~1    ...iQIQI...iQI...... 000F5080  41 4D 00 69 00 78 00 63 00 61 00 0F 00 05 73 00 65 00 00 00 FF FF FF FF FF FF 00 00 FF FF FF FF  AM.i.x.c.a....s.e...ÿÿÿÿÿÿ..ÿÿÿÿ 000F50A0  4D 49 58 43 41 53 45 20 20 20 20 20 00 00 05 69 51 49 51 49 00 00 05 69 51 49 05 00 13 00 00 00  MIXCASE     ...iQIQI...iQI...... 000F50C0  41 43 00 61 00 6D 00 65 00 6C 00 0F 00 D2 43 00 00 00 FF FF FF FF FF FF FF FF 00 00 FF FF FF FF  AC.a.m.e.l...ÒC...ÿÿÿÿÿÿÿÿ..ÿÿÿÿ 000F50E0  43 41 4D 45 4C 43 20 20 20 20 20 20 00 00 06 69 51 49 51 49 00 00 06 69 51 49 06 00 13 00 00 00  CAMELC      ...iQIQI...iQI...... 000F5100  55 50 43 41 53 45 20 20 20 20 20 20 00 00 06 69 51 49 51 49 00 00 06 69 51 49 07 00 13 00 00 00  UPCASE      ...iQIQI...iQI...... 000F5120  53 50 45 43 49 C4 4C 20 20 20 20 20 00 00 06 69 51 49 51 49 00 00 06 69 51 49 08 00 13 00 00 00  SPECIÄL     ...iQIQI...iQI...... 000F5140  41 73 00 70 00 65 00 7A 00 69 00 0F 00 1A E4 00 6C 00 00 00 FF FF FF FF FF FF 00 00 FF FF FF FF  As.p.e.z.i....ä.l...ÿÿÿÿÿÿ..ÿÿÿÿ 000F5160  53 50 45 5A 49 E4 4C 20 20 20 20 20 00 00 06 69 51 49 51 49 00 00 06 69 51 49 09 00 13 00 00 00  SPEZIäL     ...iQIQI...iQI...... 000F5180  41 73 00 70 00 65 00 63 00 FF 00 0F 00 BE 61 00 6C 00 00 00 FF FF FF FF FF FF 00 00 FF FF FF FF  As.p.e.c.ÿ...¾a.l...ÿÿÿÿÿÿ..ÿÿÿÿ 000F51A0  53 50 45 43 FF 41 4C 20 20 20 20 20 00 00 06 69 51 49 51 49 00 00 06 69 51 49 0A 00 13 00 00 00  SPECÿAL     ...iQIQI...iQI...... 000F51C0  41 73 00 70 00 65 00 63 00 69 00 0F 00 CE 9E 00 6C 00 00 00 FF FF FF FF FF FF 00 00 FF FF FF FF  As.p.e.c.i...Îž.l...ÿÿÿÿÿÿ..ÿÿÿÿ 000F51E0  53 50 45 43 49 9E 4C 20 20 20 20 20 00 00 06 69 51 49 51 49 00 00 07 69 51 49 0B 00 13 00 00 00  SPECIžL     ...iQIQI...iQI......
10-20-2016, 03:23 AM (This post was last modified: 10-20-2016 04:46 AM by DavidM.)
Post: #29
 DavidM Senior Member Posts: 725 Joined: Dec 2013
RE: 50g: copy all stored objects to SD card individually
The same test case, using Claudio's SDSto routine from SDLIB to store the files:
Code:
Offset(h) 00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F 10 11 12 13 14 15 16 17 18 19 1A 1B 1C 1D 1E 1F 000F5000  4C 43 41 53 45 20 20 20 20 20 20 00 10 64 CC B6 53 49 53 49 00 00 CC B6 53 49 03 00 13 00 00 00  LCASE      ..dÌ¶SISI..Ì¶SI...... 000F5020  41 6C 00 63 00 61 00 73 00 65 00 0F 00 E8 3B 00 00 00 FF FF FF FF FF FF FF FF 00 00 FF FF FF FF  Al.c.a.s.e...è;...ÿÿÿÿÿÿÿÿ..ÿÿÿÿ 000F5040  4C 43 41 53 45 5F 7E 31 20 20 20 00 00 00 CF B6 53 49 53 49 00 00 CF B6 53 49 04 00 13 00 00 00  LCASE_~1   ...Ï¶SISI..Ï¶SI...... 000F5060  41 4D 00 69 00 78 00 63 00 61 00 0F 00 05 73 00 65 00 00 00 FF FF FF FF FF FF 00 00 FF FF FF FF  AM.i.x.c.a....s.e...ÿÿÿÿÿÿ..ÿÿÿÿ 000F5080  4D 49 58 43 41 53 45 20 20 20 20 00 00 00 D0 B6 53 49 53 49 00 00 D0 B6 53 49 05 00 13 00 00 00  MIXCASE    ...Ð¶SISI..Ð¶SI...... 000F50A0  41 43 00 61 00 6D 00 65 00 6C 00 0F 00 D2 43 00 00 00 FF FF FF FF FF FF FF FF 00 00 FF FF FF FF  AC.a.m.e.l...ÒC...ÿÿÿÿÿÿÿÿ..ÿÿÿÿ 000F50C0  43 41 4D 45 4C 43 20 20 20 20 20 00 00 64 D0 B6 53 49 53 49 00 00 D0 B6 53 49 06 00 13 00 00 00  CAMELC     ..dÐ¶SISI..Ð¶SI...... 000F50E0  55 50 43 41 53 45 20 20 20 20 20 00 00 00 D1 B6 53 49 53 49 00 00 D1 B6 53 49 07 00 13 00 00 00  UPCASE     ...Ñ¶SISI..Ñ¶SI...... 000F5100  53 50 45 43 49 C4 4C 20 20 20 20 00 00 64 D1 B6 53 49 53 49 00 00 D1 B6 53 49 08 00 13 00 00 00  SPECIÄL    ..dÑ¶SISI..Ñ¶SI...... 000F5120  53 50 45 5A 49 E4 4C 20 20 20 20 00 10 64 D1 B6 53 49 53 49 00 00 D1 B6 53 49 09 00 13 00 00 00  SPEZIäL    ..dÑ¶SISI..Ñ¶SI...... 000F5140  53 50 45 43 FF 41 4C 20 20 20 20 00 10 00 D2 B6 53 49 53 49 00 00 D2 B6 53 49 0A 00 13 00 00 00  SPECÿAL    ...Ò¶SISI..Ò¶SI...... 000F5160  53 50 45 43 49 9E 4C 20 20 20 20 00 10 64 D2 B6 53 49 53 49 00 00 D2 B6 53 49 0B 00 13 00 00 00  SPECIžL    ..dÒ¶SISI..Ò¶SI...... 000F5180  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................................ 000F51A0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................................ 000F51C0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................................ 000F51E0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................................

So it appears that all-lower-case-or-high-ascii-names-that-fit-within-8.3 object names get UPCASEd by the version of SDLIB included with SDFiler, and with no LFN entry created. Mixed case gets the LFNs. I'm assuming that all longer-than-8.3 names end up with the LFNs as well.

All of which leaves my head spinning with regards to a reasonable path forward for the treatment of object names for this app, which was on shaky ground from the start. Given that the number of people in the entire world that could make use of such an app is probably countable on one hand (possibly one finger), this seems like more effort than it's worth at this point.

The amount of manual manipulation of files, object names, and cross-checking is not much different than simply copying the files with existing tools. The calculator doesn't track modification dates for individual objects, so relying on the SD card file system dates isn't particularly meaningful (those dates only tell you when the file was written, not when the actual object was updated). Encapsulating the real names as a string within a list object is a concept which could be useful, but IMHO doesn't add enough value to this app's concept to pursue.

But if nothing else, perhaps the discussion has had some value for NewRPL's implementation.

Thanks to Matthias, Claudio and 3298 for their participation!
10-20-2016, 07:11 PM
Post: #30
 Claudio L. Senior Member Posts: 1,505 Joined: Dec 2013
RE: 50g: copy all stored objects to SD card individually
(10-20-2016 03:23 AM)DavidM Wrote:  The same test case, using Claudio's SDSto routine from SDLIB to store the files:

So it appears that all-lower-case-or-high-ascii-names-that-fit-within-8.3 object names get UPCASEd by the version of SDLIB included with SDFiler, and with no LFN entry created. Mixed case gets the LFNs. I'm assuming that all longer-than-8.3 names end up with the LFNs as well.

Don't forget that the name is stored as uppercase (as it should be) but there's a flag telling the OS if the actual name is uppercase, lowercase, or first-letter-uppercase-only.
10-20-2016, 08:56 PM
Post: #31
 DavidM Senior Member Posts: 725 Joined: Dec 2013
RE: 50g: copy all stored objects to SD card individually
(10-20-2016 07:11 PM)Claudio L. Wrote:  Don't forget that the name is stored as uppercase (as it should be) but there's a flag telling the OS if the actual name is uppercase, lowercase, or first-letter-uppercase-only.

I haven't, but it appears that condition doesn't apply when a high-ascii character is encountered in the object name. Either that, or Win7 Explorer and dir/x aren't respecting it for the directory entries at the bottom of the list. For what it's worth, the following SysRPL code generated those directory entries:

Code:
    BINT0 BINT1 %0 "lcase" BINT1 SDLIB DROP          BINT0 BINT1 %0 "lcase;;;;;;;" BINT1 SDLIB DROP          BINT0 BINT1 %0 "Mixcase" BINT1 SDLIB DROP     BINT0 BINT1 %0 "CamelC" BINT1 SDLIB DROP     BINT0 BINT1 %0 "UPCASE" BINT1 SDLIB DROP     BINT0 BINT1 %0 "SPECI\C4L" BINT1 SDLIB DROP     BINT0 BINT1 %0 "spezi\E4l" BINT1 SDLIB DROP     BINT0 BINT1 %0 "spec\FFal" BINT1 SDLIB DROP     BINT0 BINT1 %0 "speci\9El" BINT1 SDLIB DROP
("SDLIB" is essentially a call to XLIB 3A7 15, but done without using a library).

I'm not being critical of that design choice -- there aren't any "easy" answers for the best approach here. But it seems clear that there is a good likelihood that object names will get mangled enough to require extra (manual) effort on the user's part to make sure they organize their files properly, which negates some of the marginal value an app like this would provide.

Perhaps if I thought that I would be likely to use it, I'd see more value in it. It turns out that I almost never create programs on the calculator itself, so it's actually pretty rare for me to have a need to extract objects from the calculator... I almost always go in the opposite direction (computer -> calculator). When doing so, individual objects are usually sent via wire. Wholesale restore operations (main and port memory) are handled via custom apps loaded from the SD card. This makes it fairly easy for me to replicate the exact same configurations for my real calcs and the Emu48 counterparts.
10-20-2016, 10:19 PM
Post: #32
 matthiaspaul Senior Member Posts: 385 Joined: Jan 2015
RE: 50g: copy all stored objects to SD card individually
I have reformatted the SDLIB hex dump somewhat:
Quote:"lcase":
+000: 4C 43 41 53-45 20 20 20-20 20 20 00-10 64 CC B6 LCASE
+010: 53 49 53 49-00 00 CC B6-53 49 03 00-13 00 00 00
All-lowercase name fitting into 8.3 scheme, so the case info is stored in the NT case bits. IIRC, bit 4 was meant for the extension, not the name, so shouldn't this read 08h instead of 10h?
Quote:"lcase;;;;;;;":
+020: 41 6C 00 63-00 61 00 73-00 65 00 0F-00 E8 3B 00
+030: 00 00 FF FF-FF FF FF FF-FF FF 00 00-FF FF FF FF
Al.c.a.s.e...è;...ÿÿÿÿÿÿÿÿ..ÿÿÿÿ

+040: 4C 43 41 53-45 5F 7E 31-20 20 20 00-00 00 CF B6 LCASE_~1
+050: 53 49 53 49-00 00 CF B6-53 49 04 00-13 00 00 00
All but one semicolon stripped off by library code as they are not needed here. LFN entry created because ; is not a valid filename character in an SFN.

Claudio, would the remaining ; have been stripped off as well, if the volume would not already hold another file named "lcase"?
Quote:"Mixcase":
+060: 41 4D 00 69-00 78 00 63-00 61 00 0F-00 05 73 00
+070: 65 00 00 00-FF FF FF FF-FF FF 00 00-FF FF FF FF
AM.i.x.c.a....s.e...ÿÿÿÿÿÿ..ÿÿÿÿ

+080: 4D 49 58 43-41 53 45 20-20 20 20 00-00 00 D0 B6 MIXCASE
+090: 53 49 53 49-00 00 D0 B6-53 49 05 00-13 00 00 00
Quote:"CamelC":
+0A0: 41 43 00 61-00 6D 00 65-00 6C 00 0F-00 D2 43 00
+0B0: 00 00 FF FF-FF FF FF FF-FF FF 00 00-FF FF FF FF
AC.a.m.e.l...ÒC...ÿÿÿÿÿÿÿÿ..ÿÿÿÿ

+0C0: 43 41 4D 45-4C 43 20 20-20 20 20 00-00 64 D0 B6 CAMELC
+0D0: 53 49 53 49-00 00 D0 B6-53 49 06 00-13 00 00 00
Quote:"UPCASE":
+0E0: 55 50 43 41-53 45 20 20-20 20 20 00-00 00 D1 B6 UPCASE
+0F0: 53 49 53 49-00 00 D1 B6-53 49 07 00-13 00 00 00
No LFN created as all-uppercase and still fitting into 8.3 scheme.
Quote:"SPECIÄL":
+100: 53 50 45 43-49 C4 4C 20-20 20 20 00-00 64 D1 B6 SPECIÄL
+110: 53 49 53 49-00 00 D1 B6-53 49 08 00-13 00 00 00
No LFN created as detected as all-uppercase and still fitting into 8.3 scheme. In general, if codepoint C4h will be considered as an uppercase letter depends on codepage and country settings, so other operating systems may treat it as "special character" and create a LFN entry instead.
Quote:"speziäl":
+120: 53 50 45 5A-49 E4 4C 20-20 20 20 00-10 64 D1 B6 SPEZIäL
+130: 53 49 53 49-00 00 D1 B6-53 49 09 00-13 00 00 00
No LFN created as detected as all-lowercase and still fitting into 8.3 scheme. In general, if codepoint E4h will be considered as lowercase letter depends on codepage and country settings, so other operating systems may treat it as "special character" and create a LFN entry instead. This is a bit risky, because if E4h will be treated as lowercase letter on some system, it may not be able to access the file (as no lowercase letters are allowed in SFNs).
Since this file has only a name and no extension, shouldn't the bit flag read 08h rather than 10h?
Quote:"specÿal":
+140: 53 50 45 43-FF 41 4C 20-20 20 20 00-10 00 D2 B6 SPECÿAL
+150: 53 49 53 49-00 00 D2 B6-53 49 0A 00-13 00 00 00
Same here.
Quote:"specil":
+160: 53 50 45 43-49 9E 4C 20-20 20 20 00-10 64 D2 B6 SPECIžL
+170: 53 49 53 49-00 00 D2 B6-53 49 0B 00-13 00 00 00
Greetings,

Matthias

--
"Programs are poems for computers."
10-21-2016, 02:44 AM
Post: #33
 Claudio L. Senior Member Posts: 1,505 Joined: Dec 2013
RE: 50g: copy all stored objects to SD card individually
(10-20-2016 10:19 PM)matthiaspaul Wrote:  I have reformatted the SDLIB hex dump somewhat:
Quote:"lcase":
+000: 4C 43 41 53-45 20 20 20-20 20 20 00-10 64 CC B6 LCASE
+010: 53 49 53 49-00 00 CC B6-53 49 03 00-13 00 00 00
All-lowercase name fitting into 8.3 scheme, so the case info is stored in the NT case bits. IIRC, bit 4 was meant for the extension, not the name, so shouldn't this read 08h instead of 10h?

Yes, don't forget this is a very old binary, there were multiple bugs fixed after SDFiler was released. It's almost pointless to analyze this now.

(10-20-2016 10:19 PM)matthiaspaul Wrote:  Claudio, would the remaining ; have been stripped off as well, if the volume would not already hold another file named "lcase"?
Yes, trailing semicolons are treated as a system feature. The system will remove them and/or ignore them on comparisons, and add them as needed on creation.
The user can't create or see a file with a trailing ;
It is possible for the user to create with another OS 2 files with identical name except for a trailing semicolon. hpgcc/newRPL will open the first one that matches in the current directory, both will match in searches so you'll see both in the directory, but as identical files.
10-22-2016, 06:53 PM
Post: #34
 matthiaspaul Senior Member Posts: 385 Joined: Jan 2015
RE: 50g: copy all stored objects to SD card individually
(10-21-2016 02:44 AM)Claudio L. Wrote:
(10-20-2016 10:19 PM)matthiaspaul Wrote:  IIRC, bit 4 was meant for the extension, not the name, so shouldn't this read 08h instead of 10h?
Yes, don't forget this is a very old binary, there were multiple bugs fixed after SDFiler was released. It's almost pointless to analyze this now.
Yes, but unfortunately this means that this version of SDLIB does not preserve the case properly for some types of filenames fitting into the 8.3 scheme, and so it won't be of much help for David...

Is there a newer version of SDLIB publically available somewhere? If so, for the benefit of the users the SDFiler package should be updated with the latest version of SDLIB, I think. (And once the SD/FS related stuff in newRPL is fully debugged, it might even make sense to backport this to SDLIB - not now, but perhaps in a year or so.)

Regarding David's problem, both the original HP firmware and SDLIB seem to provide usable results for filenames not fitting into the 8.3 scheme. In the case of the original firmware we already know that it may still write buggy (lowercase) SFNs in this case, and that characters 80h..A0h are not translated into Unicode properly, but at least this happens in a predictable and therefore correctable way.
So, enforcing the creation of LFNs by appending some long enough dummy string to shorter (or all) file names still appears to be a basis for a possible workaround, although not a particularly nice looking one...

Greetings,

Matthias

--
"Programs are poems for computers."
10-22-2016, 07:30 PM
Post: #35
 DavidM Senior Member Posts: 725 Joined: Dec 2013
RE: 50g: copy all stored objects to SD card individually
(10-22-2016 06:53 PM)matthiaspaul Wrote:  Yes, but unfortunately this means that this version of SDLIB does not preserve the case properly for some types of filenames fitting into the 8.3 scheme, and so it won't be of much help for David...

...but that's OK. I was already a bit dubious about putting any significant effort into this project, as with further investigation it seemed more like "a nail in search of a hammer" than a useful tool. Regardless of the reasons and tools used, it became obvious that copying individual files to the SD card was going to result in name conversions that would make the user have to spend extra time dealing with them. To me, that negated the benefit of the tool in the first place.

(10-22-2016 06:53 PM)matthiaspaul Wrote:  Is there a newer version of SDLIB publically available somewhere? If so, for the benefit of the users the SDFiler package should be updated with the latest version of SDLIB, I think. (And once the SD/FS related stuff in newRPL is fully debugged, it might even make sense to backport this to SDLIB - not now, but perhaps in a year or so.)

I agree that it would be good to replace the version of SDLIB included with SDFiler once the charset conversions and file system handling are finalized. The modularity of the existing SDFiler package should make this fairly straightforward, provided the new SDLIB contains a similar interface. I have my doubts that Tim is very focused on such old products nowadays, but perhaps he wouldn't mind if someone else updated the library for him (he did provide the Debug4x source, after all).
10-22-2016, 07:57 PM
Post: #36
 matthiaspaul Senior Member Posts: 385 Joined: Jan 2015
RE: 50g: copy all stored objects to SD card individually
(10-20-2016 03:23 AM)DavidM Wrote:  Encapsulating the real names as a string within a list object is a concept which could be useful, but IMHO doesn't add enough value to this app's concept to pursue.
Another idea along that same path, but different anyway: ;-) Are you familiar with "DESCRIPT.ION" files? These are (normally hidden) 8-bit text files holding one line of information for each file in the current directory - typically the information is just a descriptive string, but it could also be any other info like extended attributes or configuration data. These files are created and maintained alongside file operations by JPSoft's advanced command line processors (4DOS, 4OS2, 4NT, TC). Descriptions are created using the DESCRIBE command (or a normal text editor), and are shown similar to long filenames by DIR. Various third-party file managers and applications support them as well.

Compared with storing the long filenames inside the files, storing the long names in DESCRIPT.ION files would have the advantage that the original files can remain unchanged and that they are easier retrievable.

http://jpsoft.com/ascii/descfile.txt (basic format description)

Greetings,

Matthias

--
"Programs are poems for computers."
 « Next Oldest | Next Newest »

User(s) browsing this thread: 2 Guest(s)