### Bookstore

 Computer Math and Games in Pascal (preview) Lazarus Handbook

### Author Topic: Sorting and Counting  (Read 14660 times)

#### julkas

• Hero Member
• Posts: 628
• KISS principle / Lazarus 2.0.6 / FPC 3.0.4
##### Re: Sorting and Counting
« Reply #135 on: November 10, 2019, 07:19:42 pm »
We can't compare Python (interpreted Lang) with Pascal.
Yes, we can since Python relies so heavily on library code compiled in other languages. Python is just glue. Much more so than pure scripting.
(You can also use FPC to write and add Python libraries)
And indeed:
I use Python and I like Python...
PYTHON rocks.
PYTHON is great.
PYTHON forever.
PYTHON νῦν και ἀεὶ.
« Last Edit: November 10, 2019, 07:38:38 pm by julkas »
procedure mulu64(a, b: QWORD; out clo, chi: QWORD); assembler;
asm
mov rax, a
mov rdx, b
mul rdx
mov [clo], rax
mov [chi], rdx
end;

• Hero Member
• Posts: 10290
##### Re: Sorting and Counting
« Reply #136 on: November 10, 2019, 07:29:25 pm »
Well, as it seems it needs some FPC compiled sort libraries,,,,,
I am more like donkey than shrek

#### mpknap

• Full Member
• Posts: 135
##### Re: Sorting and Counting
« Reply #137 on: November 10, 2019, 08:11:23 pm »
Ultimately, I use this algorithm, it is clear to me and because I have a problem installing the LG package ....
30MB file counted and sorted is 6 minutes;)

I think TStringList is eminently suitable for this task.
Here's an alternative solution, which may use less resources.
Code: Pascal  [Select][+][-]
1. unit mainSortCount;
2.
3. {\$mode objfpc}{\$H+}
4.
5. interface
6.
7. uses
8.    Classes, SysUtils, Forms, StdCtrls;
9.
10. type
11.    TForm1 = class(TForm)
12.      Memo1: TMemo;
13.      procedure FormCreate(Sender: TObject);
14.    end;
15.
16. var
17.    Form1: TForm1;
18.
19.   procedure SortCount(const anInFile: String; out aList: TStringList);
20.
21.   procedure ShowListInMemo(constref aList: TStringList; aMemo: TMemo);
22.
23. implementation
24.
25. {\$R *.lfm}
26.
27. { TForm1 }
28.
29. procedure SortCount(const anInFile: String; out aList: TStringList);
30. const
31.   one = PtrUInt(1);
32. var
33.   textf: TextFile;
34.   s: String;
35.   idx: Integer;
36.
37.   function GetSuccObj(anIntObj: TObject): TObject;
38.   var
39.     i: PtrUInt absolute anIntObj;
40.   begin
41.     Inc(i);
42.     Exit(anIntObj);
43.   end;
44.
45. begin
46.   Assert(FileExists(anInFile), 'cannot find file "'+anInFile+'"');
47.   aList := TStringList.Create;
48.   aList.Duplicates := dupError;
49.   aList.Sorted := True;
50.   AssignFile(textf, anInFile);
51.   try
52.     Reset(textf);
53.     while not EOF(textf) do
54.       begin
56.         s := Trim(s);
57.         idx := aList.IndexOf(s);
58.         case idx of
60.           else
61.             aList.Objects[idx] := GetSuccObj(aList.Objects[idx]);
62.         end;
63.       end;
64.   finally
65.     CloseFile(textf);
66.   end;
67. end;
68.
69. procedure ShowListInMemo(constref aList: TStringList; aMemo: TMemo);
70. var
71.   i: Integer;
72. begin
73.   if Assigned(aList) and Assigned(aMemo) then
74.     begin
75.       aMemo.Clear;
76.       for i := 0 to aList.Count-1 do
77.         aMemo.Lines.Add('%s - %d', [aList[i], PtrUInt(aList.Objects[i])]);
78.     end;
79. end;
80.
81. procedure TForm1.FormCreate(Sender: TObject);
82. var
83.   sl: TStringList;
84. begin
85.   SortCount('infile.txt', sl);
86.   try
87.     ShowListInMemo(sl, Memo1);
88.     Memo1.Lines.SaveToFile('outfile.txt');
89.   finally
90.     sl.Free;
91.   end;
92. end;
93.
94. end.

As for Python, that was just curiosity, but thank you.

In general, the problem is a little different. Unixtime is one variable of a certain record. It's about displaying records and their numbers with the same Unixtime. After sorting, I don't know how to refer to the other variables in the record, but I don't want to ask for more

Well, unless it's interesting for you;)

#### 440bx

• Hero Member
• Posts: 1929
##### Re: Sorting and Counting
« Reply #138 on: November 10, 2019, 08:20:42 pm »
After sorting, I don't know how to refer to the other variables in the record, but I don't want to ask for more
Question for you, do you want your program to run on multiple platforms (e.g, Linux, Windows, other) or is Windows only acceptable ?

ETA: if Windows-only is acceptable then, another question: are the records in the file fixed length or variable length ?
« Last Edit: November 10, 2019, 08:24:41 pm by 440bx »
FPC v3.0.4 and Lazarus 1.8.2 on Windows 7 64bit.

#### avk

• Sr. Member
• Posts: 276
##### Re: Sorting and Counting
« Reply #139 on: November 11, 2019, 06:13:33 am »
... I have a problem installing the LG package ....

What kind of problem?

• Hero Member
• Posts: 10290
##### Re: Sorting and Counting
« Reply #140 on: November 11, 2019, 08:54:20 am »
What kind of install? Just make sure it is in your path.
I am more like donkey than shrek

#### mpknap

• Full Member
• Posts: 135
##### Re: Sorting and Counting
« Reply #141 on: November 11, 2019, 09:28:59 am »
After sorting, I don't know how to refer to the other variables in the record, but I don't want to ask for more
Question for you, do you want your program to run on multiple platforms (e.g, Linux, Windows, other) or is Windows only acceptable ?

ETA: if Windows-only is acceptable then, another question: are the records in the file fixed length or variable length ?

I work in Windows 10.
At the beginning I asked about sorting Unix times alone, because I thought I could do it myself. Therefore, as an example I gave a different TXT file format.
The base is originally TXT in the format:

1570485826087           - UnixTime
0,0                                -lat
0,0                                -lon
Patrycja Maca┼éa         -user Name
16584                           -User Number
ekonomik1                    -Team Name
1581                             -Team Number
- Free line as separator
1570485826087
0,0
0,0
Gargastw├│r
17943
kosmos
1243

1570485909840
41,6548226860975
12,523491502443182
Emanuele Maria Latorre
4125
Divulgazione Libera
829

1570485929612
50,63142735
19,63182841
Przemek
137
Przemek
148

1570485941031
42,30451192
-71,22277853
Asaf
15190
Israel
2610

Sample file in the attachment, the smallest due to forum restrictions.

These are the times of detection received by users' smartphones in the CREDOSCIENCE project.

I am interested in obtaining such information :
- how many detections are in the same second / minute / hour. (I already got it thanks to your help)
- If there are several in the same second / minute / hour, display which users and their other data as number, coordinates, Team name etc ..

As I wrote, this is not my job, I'm not a programmer, it's fun for me in my free time. Thank you for your interest

What kind of install? Just make sure it is in your path.
I will try as you write. I installed the Lgenerics.LPK file, there was an error and I gave up.

#### 440bx

• Hero Member
• Posts: 1929
##### Re: Sorting and Counting
« Reply #142 on: November 11, 2019, 03:10:10 pm »

1570485826087           - UnixTime
0,0                     -lat
0,0                     -lon
Patrycja Maca┼éa        -user Name
16584                   -User Number
ekonomik1               -Team Name
1581                    -Team Number
- Free line as separator
1570485826087
0,0
0,0
Gargastw├│r
17943
kosmos
1243

<many more>

1567365497795
0,0
0,0
J.C.K.
6037

1

1567367488139
0,0
0,0
J.C.K.
6037

1

Just to make sure I understand the structure of your file and its records.  It looks like every record in the file consists of seven (7) fields (one per line) and that, sometimes, a field may be blank.  I want to confirm that every record is always 7 fields (though one or more, except the unixtime, may be blank), is this correct ?

I am interested in obtaining such information :
- how many detections are in the same second / minute / hour. (I already got it thanks to your help)
Mostly with the help of the other contributors in this thread since my implementation doesn't even use the appropriate collating sequence.

- If there are several in the same second / minute / hour, display which users and their other data as number, coordinates, Team name etc ..
Piece of pie.  I got several things to do today so I won't commit to a solid timeframe but, I'll give you something soon.

FPC v3.0.4 and Lazarus 1.8.2 on Windows 7 64bit.

#### mpknap

• Full Member
• Posts: 135
##### Re: Sorting and Counting
« Reply #143 on: November 11, 2019, 03:42:51 pm »

[/font][/size]
Just to make sure I understand the structure of your file and its records.  It looks like every record in the file consists of seven (7) fields (one per line) and that, sometimes, a field may be blank.  I want to confirm that every record is always 7 fields (though one or more, except the unixtime, may be blank), is this correct ?

Yes. One record is 7 lines. The eighth line is the separator between records.

Piece of pie.  I got several things to do today so I won't commit to a solid timeframe but, I'll give you something soon.
Thanks .

#### mpknap

• Full Member
• Posts: 135
##### Re: Sorting and Counting
« Reply #144 on: November 11, 2019, 08:32:44 pm »
I will tell you what it is needed for.
Detections in smartphones are the impact of cosmic ray particles on the phone's camera (photons, electrons, muons). They are mainly single, not regular impacts on the camera matrix.
Scientists, creators of the CREDO project, are looking for the so-called showers, i.e. rain, hitting many phones at one time, or a short time interval.

Scientists are also looking for links between detection frequencies and other phenomena, with an earthquake, with medicine, and solar wind. I also want to put these data on top of each other. maybe someday I will compare with the results of scientists, maybe I will be the first, maybe not, but I will definitely not be bored with such tasks
I "play" such searches on my own.

Anyone can join the project. Data are widely available. They mainly operate on Python.

Currently, the biggest problem is the elimination of false detections. They are mainly committed by new users or those who want to be better in the ranking. For this you need specialists in image analysis.

I don't want you to think that I'm using you, what I do is on my own, for my own curiosity.

#### 440bx

• Hero Member
• Posts: 1929
##### Re: Sorting and Counting
« Reply #145 on: November 12, 2019, 02:23:06 am »
I don't want you to think that I'm using you, what I do is on my own, for my own curiosity.

Detections in smartphones are the impact of cosmic ray particles on the phone's camera (photons, electrons, muons).
<snip>
Scientists are also looking for links between detection frequencies and other phenomena,
Experiments like that have the potential to yield surprising results.  Many scientific discoveries resulted from unexpected side effects.  Some that come to mind are microwave ovens, the scientists were constantly having headaches, that's how they figured out that the microwaves were frying their brains (not kidding... though, saying "frying" is a bit of an exaggeration.)

One my favorite unexpected side effects is related to the smartphones you mention.  You may have noticed that cell phones/smart phones no longer have an antenna that has to be pulled out of the phone in order to get reception (old cell phones used to have an antenna.) The reason there _seems_ to be no antenna goes back all the way to Georg Cantor, a mathematician who studied infinities, his work influenced Benoit Mandelbrot, who defined and studied the behavior of fractals, subsequently it was found, apparently mostly by accident, that fractal antennas had a significantly greater wave reception spectrum (due, in part, to their being partially dimensional.)

Without fractals antennas, smartphones would need about half a dozen different antennas protruding out of them in order to capture the spectrum of waves they need to implement all their nifty functions.

I didn't have any time to work of the program today but... I haven't forgotten.
FPC v3.0.4 and Lazarus 1.8.2 on Windows 7 64bit.

#### avk

• Sr. Member
• Posts: 276
##### Re: Sorting and Counting
« Reply #146 on: November 12, 2019, 05:56:56 am »
@mpknap, did you mean something like that?

#### mpknap

• Full Member
• Posts: 135
##### Re: Sorting and Counting
« Reply #147 on: November 12, 2019, 07:09:00 am »
@mpknap, did you mean something like that?
that's exactly how I would see it. I have to test. I want to transfer this data to Google map.

#### avk

• Sr. Member
• Posts: 276
##### Re: Sorting and Counting
« Reply #148 on: November 12, 2019, 07:27:12 am »
What is essential:
Your test file contained a BOM, I just deleted it manually. Of course, you can load the data file in another way.
The project uses TVirtualStringTree, you must make sure that you have the virtualtreeview package installed.
And of course, the project uses LGenerics.
« Last Edit: November 12, 2019, 07:29:28 am by avk »

#### 440bx

• Hero Member
• Posts: 1929
##### Re: Sorting and Counting
« Reply #149 on: November 12, 2019, 04:45:14 pm »
@mpknap

Avk's program is about as good as it gets.  I guess you're all set.

@avk

Nice!  thank you.
FPC v3.0.4 and Lazarus 1.8.2 on Windows 7 64bit.