Recent

Author Topic: Sorting and Counting  (Read 7933 times)

avk

  • Full Member
  • ***
  • Posts: 154
    • my self-education project
Re: Sorting and Counting
« Reply #120 on: August 01, 2019, 06:09:12 pm »
@BrunoK, do you want the new version to be added to the benchmark and runned?

mpknap

  • Jr. Member
  • **
  • Posts: 92
Re: Sorting and Counting
« Reply #121 on: November 09, 2019, 08:10:04 am »
Gentlemen, does anyone know how to write it in Python?

bytebites

  • Full Member
  • ***
  • Posts: 213
Re: Sorting and Counting
« Reply #122 on: November 09, 2019, 09:46:44 am »
Without sorting
Code: Python  [Select]
  1. from collections import defaultdict
  2. d=defaultdict(int)
  3. with open("infile") as f:
  4.   for s in f:
  5.     k=s.strip(chr(10))
  6.     if k:
  7.       d[k]+=1
  8. with open("outfile","w") as o:
  9.   for (k,v) in d.items():
  10.     print(f'{k} - {v}',file=o)    

jamie

  • Hero Member
  • *****
  • Posts: 2083
Re: Sorting and Counting
« Reply #123 on: November 09, 2019, 03:36:14 pm »
Ugg , looks too much like BASIC.....
 >:(
Number 1 at blue screen app creations!

julkas

  • Sr. Member
  • ****
  • Posts: 412
  • KISS principle / Lazarus 2.0.0 / FPC 3.0.4
Re: Sorting and Counting
« Reply #124 on: November 09, 2019, 03:46:39 pm »
Ugg , looks too much like BASIC.....
 >:(
BASIC rocks.
BASIC is great.
BASIC forever.
« Last Edit: November 09, 2019, 03:55:14 pm by julkas »
procedure mulu64(a, b: QWORD; out clo, chi: QWORD); assembler;
asm
  mov rax, a
  mov rdx, b
  mul rdx
  mov [clo], rax
  mov [chi], rdx
end;

mpknap

  • Jr. Member
  • **
  • Posts: 92
Re: Sorting and Counting
« Reply #125 on: November 09, 2019, 09:05:41 pm »
Ugg , looks too much like BASIC.....
 >:(

but it works :)
only sorting necessary for me ....

440bx

  • Hero Member
  • *****
  • Posts: 1199
Re: Sorting and Counting
« Reply #126 on: November 09, 2019, 10:02:56 pm »
but it works :)
Just curiosity, it would be interesting to see how the performance of that Python implementation compares with the various Pascal implementations.
using FPC v3.0.4 and Lazarus 1.8.2 on Windows 7 64bit.

avk

  • Full Member
  • ***
  • Posts: 154
    • my self-education project
Re: Sorting and Counting
« Reply #127 on: November 10, 2019, 10:18:55 am »
You only have to wish, sir. :)
Python code:
Code: Python  [Select]
  1. from collections import defaultdict
  2. import timeit
  3. total=0
  4. d=defaultdict(int)
  5. stime=timeit.default_timer()
  6. with open("infile") as f:
  7.   for s in f:
  8.     total +=1
  9.     k=s.strip(chr(10))
  10.     if k:
  11.       d[k]+=1
  12. with open("outfile","w") as o:
  13.   for k in sorted(d.keys()):
  14.     print(f'{k} - {d[k]}',file=o)
  15. stime=timeit.default_timer()-stime
  16. print('Time elapsed: ', stime, ', #unique: ', len(d), ', #total: ', total)
  17.  
Output:
Code: Text  [Select]
  1. Time elapsed:  12.047126958000263 , #unique:  999955 , #total:  10000000
  2.  

Pascal code:
Code: Pascal  [Select]
  1. program sort_count;
  2. {$mode objfpc}
  3. {$MODESWITCH NESTEDPROCVARS}
  4. uses
  5.   SysUtils, DateUtils,
  6.   LGUtils, LGHashMultiSet, LGHelpers;
  7. procedure SortCount;
  8. type
  9.   TCounter  = specialize TGHashMultiSetLP<Integer>;
  10.   TCountRef = specialize TGAutoRef<TCounter>;
  11.   TEntry    = TCounter.TEntry;
  12.   function EntryCmp(constref L, R: TEntry): SizeInt;
  13.   begin Result := Integer.Compare(L.Key, R.Key); end;
  14. var
  15.   CountRef: TCountRef;
  16.   InOut: Text;
  17.   Counter: TCounter;
  18.   e: TEntry;
  19.   I: Integer;
  20.   stime: TTime;
  21. begin
  22.   Counter := CountRef;
  23.   Assign(InOut, 'infile');
  24.   Reset(InOut);
  25.   stime := Time;
  26.   while not EOF(InOut) do
  27.     begin
  28.       ReadLn(InOut, I);
  29.       Counter.Add(I);
  30.     end;
  31.   Close(InOut);
  32.   if Counter.NonEmpty then
  33.     begin
  34.       Assign(InOut, 'outfile');
  35.       Rewrite(InOut);
  36.       for e in Counter.Entries.Sorted(@EntryCmp) do
  37.         WriteLn(InOut, e.Key, ' - ', e.Count);
  38.       Close(InOut);
  39.     end;
  40.   WriteLn('Time elapsed: ', MillisecondsBetween(Time, stime)/1000:0:4,
  41.           ', #unique: ', Counter.EntryCount, ', #total: ', Counter.Count);  
  42. end;
  43. begin
  44.   SortCount;
  45. end.
  46.  
Output:
Code: Text  [Select]
  1. Time elapsed: 3.2300, #unique: 999955, #total: 10000000
  2.  

julkas

  • Sr. Member
  • ****
  • Posts: 412
  • KISS principle / Lazarus 2.0.0 / FPC 3.0.4
Re: Sorting and Counting
« Reply #128 on: November 10, 2019, 10:38:42 am »
but it works :)
Just curiosity, it would be interesting to see how the performance of that Python implementation compares with the various Pascal implementations.
Python sort implementation is exelent. It's based on Team sort algo.
https://stackoverflow.com/questions/1517347/about-pythons-built-in-sort-method
@avk Can you compare only sorting phase time?. (Python default I/O is slow).
Try with PyPy compiler also.
 
« Last Edit: November 10, 2019, 10:49:05 am by julkas »
procedure mulu64(a, b: QWORD; out clo, chi: QWORD); assembler;
asm
  mov rax, a
  mov rdx, b
  mul rdx
  mov [clo], rax
  mov [chi], rdx
end;

440bx

  • Hero Member
  • *****
  • Posts: 1199
Re: Sorting and Counting
« Reply #129 on: November 10, 2019, 10:57:36 am »
You only have to wish, sir. :)
Thank you very much Avk. :)
using FPC v3.0.4 and Lazarus 1.8.2 on Windows 7 64bit.

avk

  • Full Member
  • ***
  • Posts: 154
    • my self-education project
Re: Sorting and Counting
« Reply #130 on: November 10, 2019, 11:12:46 am »
Can you compare only sorting phase time?. (Python default I/O is slow).
Sort phase only:
Python
Code: Text  [Select]
  1. Time elapsed:  1.1443881560007867 , #unique:  999955 , #total:  10000000
  2.  
Pascal
Code: Text  [Select]
  1. Time elapsed: 0.1750, #unique: 999955, #total: 10000000
  2.  

julkas

  • Sr. Member
  • ****
  • Posts: 412
  • KISS principle / Lazarus 2.0.0 / FPC 3.0.4
Re: Sorting and Counting
« Reply #131 on: November 10, 2019, 11:34:04 am »
Can you compare only sorting phase time?. (Python default I/O is slow).
Sort phase only:
Python
Code: Text  [Select]
  1. Time elapsed:  1.1443881560007867 , #unique:  999955 , #total:  10000000
  2.  
Pascal
Code: Text  [Select]
  1. Time elapsed: 0.1750, #unique: 999955, #total: 10000000
  2.  
@avk Thanks.
https://ideone.com/9n168K
procedure mulu64(a, b: QWORD; out clo, chi: QWORD); assembler;
asm
  mov rax, a
  mov rdx, b
  mul rdx
  mov [clo], rax
  mov [chi], rdx
end;

Thaddy

  • Hero Member
  • *****
  • Posts: 9183
Re: Sorting and Counting
« Reply #132 on: November 10, 2019, 12:20:34 pm »
@avk Thanks.
https://ideone.com/9n168K
Julkas, ideone lags behind in compiler version for FPC and that can make a big difference. You should run the complete test suite on ideone to get any meaningful comparison.
Better run 3.2.0  or test everything with 3.0.4.
« Last Edit: November 10, 2019, 12:25:37 pm by Thaddy »
also related to equus asinus.

julkas

  • Sr. Member
  • ****
  • Posts: 412
  • KISS principle / Lazarus 2.0.0 / FPC 3.0.4
Re: Sorting and Counting
« Reply #133 on: November 10, 2019, 06:11:24 pm »
We can't compare Python (interpreted Lang) with Pascal.
Anyway, I use Python, I like Python.
procedure mulu64(a, b: QWORD; out clo, chi: QWORD); assembler;
asm
  mov rax, a
  mov rdx, b
  mul rdx
  mov [clo], rax
  mov [chi], rdx
end;

Thaddy

  • Hero Member
  • *****
  • Posts: 9183
Re: Sorting and Counting
« Reply #134 on: November 10, 2019, 06:17:20 pm »
We can't compare Python (interpreted Lang) with Pascal.
Yes, we can since Python relies so heavily on library code compiled in other languages. Python is just glue. Much more so than pure scripting.
(You can also use FPC to write and add Python libraries)
And indeed:
I use Python and I like Python... :P :o
« Last Edit: November 10, 2019, 06:19:53 pm by Thaddy »
also related to equus asinus.