Recent

Author Topic: DataProblems Maybe  (Read 4198 times)

JLWest

  • Hero Member
  • *****
  • Posts: 542
DataProblems Maybe
« on: April 23, 2019, 07:35:43 am »
Code: Pascal  [Select]
  1. procedure TForm1.Edit3AndRefDataSet(aIdx : Integer);
  2.   Var i : Integer = -1;
  3.    LB3Rcd  : String = '';
  4.    Bit1    : String = '';
  5.   begin
  6.    i := aIdx;
  7.    ListBox3.ItemIndex := i;
  8.    Ed3String  =  Listbox3.Items[i];
  9.    LB3Rcd := Listbox3.Items[i];
  10.  
  11.   Bit1  := ExtractWord(6,LB3Rcd,['|']);
  12.   RefData.Lat := StrToFloat(Bit1);      <-- Run Time error 'K2' not a valid float
  13.  
  14.   Bit1 := ExtractWord(7,Ed3String,['|']);  <-- Run Time error 'K2' not a valid float
  15.   RefData.Lat := StrToFloat(Bit1);
  16.  
  17.    Ed3String  := '|00CA||7826300||Barstow||United States||K2||35.349333||-116.893333|'; <-- Actual Data
  18.    Bit1  := ExtractWord(6,Ed3String,['|']);
  19.    RefData.Lat := StrToFloat(Bit1);
  20. {  RefData.Lat is defined as a Double:
  21.    RefData.Lat was = 35.349333          <-- Correct value}
  22.  
  23.   end;    

 I copied the data as Raw Data under the debugger and set this test up.
I don't understand wha'ts going on.

Is there a function to convert text to raw data? I have been trying to write one but cant get it to work right.
JLWEST
Lazuras ver 2.0.2 
 FPC 3.0.4, Lazarus IDE v1.8.2 Windows 10 Pro
 Intel i7 770K CPU 4.2GHz 32702MB Ram
GeForce GTX 1080 Graphics - 8 Gig
3952 GB (1.5 SSD)

six1

  • Jr. Member
  • **
  • Posts: 73
Re: DataProblems Maybe
« Reply #1 on: April 23, 2019, 07:43:23 am »
'|00CA||7826300||Barstow||United States||K2||35.349333||-116.893333|'
  1       2   3        4  5        6   7                8 9  10    11       12   13


maybe it isn't the 6th and 7th more 11th and 13s word?

Also check result!
The N-th word, or empty if N is out of range.
https://www.freepascal.org/docs-html-3.0.0/rtl/strutils/extractword.html

JLWest

  • Hero Member
  • *****
  • Posts: 542
Re: DataProblems Maybe
« Reply #2 on: April 23, 2019, 07:52:53 am »
I ran a test program and

 S := ExtractWord(1,Data,['|']); is using the delimiter as right and left as a pair So:

     1           2            3                 4            5         6               7

'|00CA||7826300||Barstow||United States||K2||35.349333||-116.893333|

It's giving me the error saying 'K2' is not a valid Float.

So my thinking is the data coming out of the Listbox is different that the Raw Data I copied from the debugger and it makes a difference of one field.

I think I need to take the data out of the listbox. Somehow convert it to Raw Data and then extract what I need.

Maybe as a RawByteString but I don't know how to write the function.
JLWEST
Lazuras ver 2.0.2 
 FPC 3.0.4, Lazarus IDE v1.8.2 Windows 10 Pro
 Intel i7 770K CPU 4.2GHz 32702MB Ram
GeForce GTX 1080 Graphics - 8 Gig
3952 GB (1.5 SSD)

bytebites

  • Full Member
  • ***
  • Posts: 181
Re: DataProblems Maybe
« Reply #3 on: April 23, 2019, 08:25:53 am »
'|00CA||7826300||Barstow||United States||K2||35.349333||-116.893333|'
  1       2   3        4  5        6   7                8 9  10    11       12   13


maybe it isn't the 6th and 7th more 11th and 13s word?
...
This is how ExtractDelimited works.
https://www.freepascal.org/docs-html-3.0.0/rtl/strutils/extractdelimited.html

sstvmaster

  • Full Member
  • ***
  • Posts: 100
Re: DataProblems Maybe
« Reply #4 on: April 23, 2019, 08:28:56 am »
@JLWEST

this could help (in FormCreate?):
Code: Pascal  [Select]
  1. DefaultFormatSettings.DecimalSeparator := '.';

If this works, your decimal separator is "," Comma not point.

See attachment, this works.
Lazarus 2.0.2 x32
Lazarus 2.1.0 Trunk x32
OS Win 7 32bit

JLWest

  • Hero Member
  • *****
  • Posts: 542
Re: DataProblems Maybe
« Reply #5 on: April 23, 2019, 08:40:54 am »
@bytebites Maybe your right about the field counts?

I have been running this way for some time. I got the following function going and passing the RCDString thru it.

Limited testing but it seems to work.

'|00CA||7826300||Barstow||United States||K2||35.349333||-116.893333|' is my created data format and I would rather have it look like this:

'[00CA|[7826300][Barstow][United States][K2][35.349333][-116.893333]'

But then how do you write ExtractWord?

S := ExtractWord(6,RCDString,['['..']'); or mybe

S := ExtractWord(6,RCDString, ['[']  , [']']  );

Code: Pascal  [Select]
  1. function TForm1.ToAlphanumericString(const AString: String): String;
  2.     var
  3.       Character: Char;
  4.     begin
  5.       Result := '';
  6.  
  7.       for Character in AString do
  8.         if Character in ['0'..'9', 'A'..'Z', '|'] then
  9.           Result += Character;
  10.     end;        

JLWEST
Lazuras ver 2.0.2 
 FPC 3.0.4, Lazarus IDE v1.8.2 Windows 10 Pro
 Intel i7 770K CPU 4.2GHz 32702MB Ram
GeForce GTX 1080 Graphics - 8 Gig
3952 GB (1.5 SSD)

six1

  • Jr. Member
  • **
  • Posts: 73
Re: DataProblems Maybe
« Reply #6 on: April 23, 2019, 08:50:34 am »
'|00CA||7826300||Barstow||United States||K2||35.349333||-116.893333|'
  1       2   3        4  5        6   7                8 9  10    11       12   13


maybe it isn't the 6th and 7th more 11th and 13s word?
...
This is how ExtractDelimited works.
https://www.freepascal.org/docs-html-3.0.0/rtl/strutils/extractdelimited.html

Help says:
This means that if 2 delimiter characters appear next to each other, there is an empty part between it.

in my interpretation above countwise is right!
2 delimiter characters appear next to each other counts!
or am i totaly wrong?

i'm using TStringlist with delimiter in such cases and this "||" will give two parts...

JLWest

  • Hero Member
  • *****
  • Posts: 542
Re: DataProblems Maybe
« Reply #7 on: April 23, 2019, 08:52:57 am »

Thoid Works: And it returns the 6 th field  39.297611111


   s := '[KMCI][5502297][Kansas City][United States][K3|[39.297611111][-94.713888889]';

  A := ExtractWord(6,s,['['..']']);

Just have to change an a lot of code to but maybe it will be better. 
JLWEST
Lazuras ver 2.0.2 
 FPC 3.0.4, Lazarus IDE v1.8.2 Windows 10 Pro
 Intel i7 770K CPU 4.2GHz 32702MB Ram
GeForce GTX 1080 Graphics - 8 Gig
3952 GB (1.5 SSD)

JLWest

  • Hero Member
  • *****
  • Posts: 542
Re: DataProblems Maybe
« Reply #8 on: April 23, 2019, 08:54:32 am »
'|00CA||7826300||Barstow||United States||K2||35.349333||-116.893333|'
  1       2   3        4  5        6   7                8 9  10    11       12   13


maybe it isn't the 6th and 7th more 11th and 13s word?
...
This is how ExtractDelimited works.
https://www.freepascal.org/docs-html-3.0.0/rtl/strutils/extractdelimited.html

Help says:
This means that if 2 delimiter characters appear next to each other, there is an empty part between it.

in my interpretation above countwise is right!
2 delimiter characters appear next to each other counts!
or am i totaly wrong?

i'm using TStringlist with delimiter in such cases and this "||" will give two parts...

No I think maybe your right.

JLWEST
Lazuras ver 2.0.2 
 FPC 3.0.4, Lazarus IDE v1.8.2 Windows 10 Pro
 Intel i7 770K CPU 4.2GHz 32702MB Ram
GeForce GTX 1080 Graphics - 8 Gig
3952 GB (1.5 SSD)

six1

  • Jr. Member
  • **
  • Posts: 73
Re: DataProblems Maybe
« Reply #9 on: April 23, 2019, 09:11:10 am »
'|00CA||7826300||Barstow||United States||K2||35.349333||-116.893333|'
  1       2   3        4  5        6   7                8 9  10    11       12   13


maybe it isn't the 6th and 7th more 11th and 13s word?
...
This is how ExtractDelimited works.
https://www.freepascal.org/docs-html-3.0.0/rtl/strutils/extractdelimited.html

in addition:
'|00CA||7826300||Barstow||United States||K2||35.349333||-116.893333|'
0  1       2   3        4  5        6   7                8 9  10    11       12   13         14

this is how it will count by using TStringlist delimit

lucamar

  • Hero Member
  • *****
  • Posts: 1538
Re: DataProblems Maybe
« Reply #10 on: April 23, 2019, 03:24:00 pm »
Since you're using ExtracWord() and not ExtractDelimited(), all that talk about empty returns is futile. As the documentation says:
Quote from: Ref. for 'strutils' (#rtl) - ExtractWords
Unlike ExtractDelimited, an empty string is not a valid return value, i.e. is not a word. If an empty string is returned, the index N was out of range.

For ExtractWords(N, AString, ['|']) the count should be:
Code: [Select]
|00CA||7826300||Barstow||United States||K2||35.349333||-116.893333|'
 1     2        3        4              5   6          7
as demonstrated by the fact that it works when you test directly with that string.

Rather add something to show you what is getting read from the ListBox. If it's like the string you're expecting then something is wrong with ExtractWord().

And to use '[' and ']' as delimiters it should suffice with:
Code: Pascal  [Select]
  1.  A := ExtractWord(6,s,['[',']']);
« Last Edit: April 23, 2019, 03:25:55 pm by lucamar »
Turbo Pascal 3 CP/M - Amstrad PCW 8256 (512 KB !!!) :P
Lazarus 1.8.4 & 2.0.2 w/FPC 3.0.4 on:
(K|L)Ubuntu 12..16, Windows XP SP3, various DOSes.

Thausand

  • Full Member
  • ***
  • Posts: 227
Re: DataProblems Maybe
« Reply #11 on: April 23, 2019, 04:52:20 pm »
Is correct user lucamar. I think user six1 not understand maybe ? user bytebites good explain...

error JLWest problem StrToFloat... he write so is.

User sstvmaster write good test. Many country have other float separate. Is two "." and "," (but can make any use freepascal if want). Some time data storage float not same separate have you system and make error when use StrToFloat.

add:

I make program test and see better make float string. Can make more difficult if want. Is make freepascal and can test data if file data.LoadFromFile('Name_File');

Code: Pascal  [Select]
  1. program floatfloat;
  2.  
  3. {$MODE OBJFPC}{$H+}
  4.  
  5. uses
  6.   Classes, SysUtils, StrUtils;
  7.  
  8. procedure Info(S: String; const F: array of const);
  9. begin
  10.   WriteLn('Info: ', Format(S, F));
  11. end;
  12.  
  13. procedure Error(S: String; const F: array of const);
  14. begin
  15.   WriteLn('Error: ', Format(S, F));
  16. end;
  17.  
  18. procedure Test(Lines: TStrings);
  19. const
  20.   worddelim             = ['|'];
  21.   radixdelim            = ['.', ','];
  22. var
  23.   line_no               : integer = 0;
  24.   line                  : string;
  25.   nr_words_in_line      : integer;
  26.   float_part_count      : integer;
  27.   float_integer_part    : string;
  28.   float_fractional_part : string;
  29.   bit1, newbit1         : string;
  30.   bit2, newbit2         : string;
  31.   lat1, lat2            : double;
  32. begin
  33.   for line in Lines do
  34.   begin
  35.     inc(line_no);
  36.     nr_words_in_line := WordCount(line, worddelim);
  37.  
  38.     if (nr_words_in_line = 7) then
  39.     begin
  40.  
  41.       // have float bit1
  42.  
  43.       bit1  := ExtractWord(6, line, worddelim);
  44.       float_part_count := WordCount(bit1, radixdelim);
  45.       if (float_part_count = 2) then
  46.       begin
  47.         float_integer_part    := ExtractWord(1, bit1, radixdelim);
  48.         float_fractional_part := ExtractWord(2, bit1, radixdelim);
  49.         newbit1               := float_integer_part + DefaultFormatSettings.DecimalSeparator + float_fractional_part;
  50.         if TryStrToFloat(newbit1, lat1)
  51.         then Info('line %d have %s valid float value = %f', [line_no, 'bit1', lat1])
  52.         else Error('have error data %s. Not have validate float "%s" line = %d', ['bit1', bit1, line_no]);
  53.       end
  54.       else Error('have error data %s. Not have validate float "%s" line = %d', ['bit1', bit1, line_no]);
  55.  
  56.       // have float bit2
  57.  
  58.       bit2  := ExtractWord(7, line, worddelim);
  59.       float_part_count := WordCount(bit2, radixdelim);
  60.       if (float_part_count = 2) then
  61.       begin
  62.         float_integer_part    := ExtractWord(1, bit2, radixdelim);
  63.         float_fractional_part := ExtractWord(2, bit2, radixdelim);
  64.         newbit2               := float_integer_part + DefaultFormatSettings.DecimalSeparator + float_fractional_part;
  65.         if TryStrToFloat(newbit2, lat2)
  66.         then Info('line %d have %s valid float value = %f', [line_no, 'bit2', lat2])
  67.         else Error('have error data %s. Not have validate float "%s" line = %d', ['bit2', bit2, line_no]);
  68.       end
  69.       else Error('have error data %s. Not have validate float "%s" line = %d', ['bit2', bit2, line_no]);
  70.     end
  71.     else Error('not know line %d and have %d word', [line_no, nr_words_in_line]);
  72.   end;
  73. end;
  74.  
  75. var
  76.   data: TStringList;
  77. begin
  78.   data := TStringList.Create;
  79.   data.append('|00CA||7826300||Barstow||United States||K2||35,349333||-116,893333|');
  80.   data.append('|7826300||Barstow||United States||K2||35,349333||-116,893333|');
  81.   data.append('|00CA||7826300||Barstow||United States||K2||35.349333||-116.893333|');
  82.   data.append('|1||hello||12345||67,78||1111K2||fun||calculate|');
  83.   data.append('|00CA||7826300||Barstow||United States||K2||35,349333||-116.893333|');
  84.   data.append('|00CA||7826300||Barstow||United States||K2||35.349333||-116,893333|');
  85.  
  86.   // make many append data for want test
  87.  
  88.   Test(data);
  89.   data.free;
  90. end.
  91.  

that make write

Code: [Select]
Info: line 1 have bit1 valid float value = 35.35
Info: line 1 have bit2 valid float value = -116.89
Error: not know line 2 and have 6 word
Info: line 3 have bit1 valid float value = 35.35
Info: line 3 have bit2 valid float value = -116.89
Error: have error data bit1. Not have validate float "fun" line = 4
Error: have error data bit2. Not have validate float "calculate" line = 4
Info: line 5 have bit1 valid float value = 35.35
Info: line 5 have bit2 valid float value = -116.89
Info: line 6 have bit1 valid float value = 35.35
Info: line 6 have bit2 valid float value = -116.89

And have not error runtime  :)
« Last Edit: April 23, 2019, 07:19:44 pm by Thausand »

JLWest

  • Hero Member
  • *****
  • Posts: 542
Re: DataProblems Maybe
« Reply #12 on: April 23, 2019, 09:47:55 pm »
@lucamar

Changed all my data to '['..']'.
I used: A := ExtractWord('NIL', s, ['['..']']);
Compared to A := ExtractWord(6,s,['[',']']); and it seems to work.

Still testing.
JLWEST
Lazuras ver 2.0.2 
 FPC 3.0.4, Lazarus IDE v1.8.2 Windows 10 Pro
 Intel i7 770K CPU 4.2GHz 32702MB Ram
GeForce GTX 1080 Graphics - 8 Gig
3952 GB (1.5 SSD)

lucamar

  • Hero Member
  • *****
  • Posts: 1538
Re: DataProblems Maybe
« Reply #13 on: April 23, 2019, 10:37:20 pm »
I had a little after-supper free-time and nothing better to do so I made a small test/demo. Find it attached.

The "meat" of it is the method Decompose(Index: Integer) which does, more or less, the same as your code in the first post. Rather heavily commented, but WTH! :)
Code: Pascal  [Select]
  1. procedure TMainForm.Decompose(Index: Integer);
  2. { Decompose "Index"th string from SrcList and add the words to DestList }
  3. var
  4.   Count: Integer;
  5.   AWord: String;
  6.   Delims: TSysCharSet;
  7.   AFloat: Extended;
  8. begin
  9.   if Index in [0..SrcList.Count-1] then begin { Sanity check }
  10.     { Select the proper delimiters }
  11.     if SrcList.Items[Index].StartsWith('|') then
  12.       Delims := ['|']     { String format: "|something||other||...| }
  13.     else if SrcList.Items[Index].StartsWith('[') then
  14.       Delims := ['[',']'] { String format: "[something][other][...] }
  15.     else                { Free form string? Use standard delimiters }
  16.       Delims := StdWordDelims;
  17.     { Extract all words }
  18.     DestList.Clear;
  19.     Count := 0;
  20.     repeat
  21.       Inc(Count);
  22.       AWord := ExtractWord(Count, SrcList.Items[Index], Delims);
  23.       if not AWord.IsEmpty then
  24.         if TryStrToFloat(AWord, AFloat) then
  25.           { If a float, show also the StrToFloat conversion}
  26.           DestList.Items.Add(sfWithFloat,[Count, AWord, AFloat])
  27.         else
  28.           DestList.Items.Add(sfNoFloat,[Count, AWord]);
  29.     until AWord.IsEmpty;
  30.     ShowMessageFmt('Found %d words', [Count-1])
  31.   end;
  32. end;
« Last Edit: April 23, 2019, 10:40:07 pm by lucamar »
Turbo Pascal 3 CP/M - Amstrad PCW 8256 (512 KB !!!) :P
Lazarus 1.8.4 & 2.0.2 w/FPC 3.0.4 on:
(K|L)Ubuntu 12..16, Windows XP SP3, various DOSes.

JLWest

  • Hero Member
  • *****
  • Posts: 542
Re: DataProblems Maybe
« Reply #14 on: April 23, 2019, 11:53:20 pm »
I thought it was working but that's not the case.
 
ED2String = '[00C][28944][Nil][Nil][K2][37.203177778][-107.869194444]'

RCD.ICAO := ExtractWord(3,ED2String ,['['..']']); <-- Different than your's

RCD.ICAO = ''

Changed to:

RCD.ICAO := ExtractWord(1,ED2String ,['[',']']); <-- Changed to your's

RCD.ICAO = ''

Still not working.

Will take a look at your code example.
 

JLWEST
Lazuras ver 2.0.2 
 FPC 3.0.4, Lazarus IDE v1.8.2 Windows 10 Pro
 Intel i7 770K CPU 4.2GHz 32702MB Ram
GeForce GTX 1080 Graphics - 8 Gig
3952 GB (1.5 SSD)