Post Reply 
Unexpected result from STRING command
05-07-2023, 06:27 PM
Post: #2
RE: Unexpected result from STRING command
For reference, I compiled results with different HFormat settings. I just left HDigits at 3 for all these tests.

I set variable num:=0.123456789, and then used STRING(num,mode) to get these results. Bolded lines look incorrect to me.

The first result with HFormat set to 0, and then specifying fixed mode (2) adds an extra two zeroes at the end, certianly looks like a bug to me.

I would expect 0.123456789 for all strings where mode was specified as standard (1), but htis only happens when HFormat is 0. Looks like a bug to me.

HFormat = 0 (Standard) HDigits = 3
0.123456789 - no argument
0.123456789 - Mode = 0 (current)
0.123456789 - Mode = 1 (standard)
0.12345678900 - Mode = 2 (fixed)
1.23456789000ᴇ−1 - Mode = 3 (scientific)
123.456789000ᴇ−3 - Mode = 4 (engineering)
0.123456789 - Mode = 5 (floating)
0.123456789 - Mode = 6 (rounded)

HFormat = 1 (Fixed) HDigits = 3
0.123 - no argument
0.123 - Mode = 0 (current)
0.1235 - Mode = 1 (standard)
0.123 - Mode = 2 (fixed)
1.235ᴇ−1 - Mode = 3 (scientific)
123.5ᴇ−3 - Mode = 4 (engineering)
0.1235 - Mode = 5 (floating)
0.123 - Mode = 6 (rounded)

HFormat = 2 (Scientific) HDigits = 3
1.235ᴇ−1 - no argument
1.235ᴇ−1 - Mode = 0 (current)
0.1235 - Mode = 1 (standard)
0.123 - Mode = 2 (fixed)
1.235ᴇ−1 - Mode = 3 (scientific)
123.5ᴇ−3 - Mode = 4 (engineering)
0.1235 - Mode = 5 (floating)
0.123 - Mode = 6 (rounded)

HFormat = 3 (Engineering) HDigits = 3
123.5ᴇ−3 - no argument
123.5ᴇ−3 - Mode = 0 (current)
0.1235 - Mode = 1 (standard)
0.123 - Mode = 2 (fixed)
1.235ᴇ−1 - Mode = 3 (scientific)
123.5ᴇ−3 - Mode = 4 (engineering)
0.1235 - Mode = 5 (floating)
0.123 - Mode = 6 (rounded)

HFormat = 4 (Floating) HDigits = 3
0.1235 - no argument
0.1235 - Mode = 0 (current)
0.1235 - Mode = 1 (standard)
0.123 - Mode = 2 (fixed)
1.235ᴇ−1 - Mode = 3 (scientific)
123.5ᴇ−3 - Mode = 4 (engineering)
0.1235 - Mode = 5 (floating)
0.123 - Mode = 6 (rounded)

HFormat = 5 (Rounded) HDigits = 3
0.123 - no argument
0.123 - Mode = 0 (current)
0.1235 - Mode = 1 (standard)
0.123 - Mode = 2 (fixed)
1.235ᴇ−1 - Mode = 3 (scientific)
123.5ᴇ−3 - Mode = 4 (engineering)
0.1235 - Mode = 5 (floating)
0.123 - Mode = 6 (rounded)
Visit this user's website Find all posts by this user
Quote this message in a reply
Post Reply 


Messages In This Thread
RE: Unexpected result from STRING command - Jacob Wall - 05-07-2023 06:27 PM



User(s) browsing this thread: 1 Guest(s)