Recent

Author Topic: Sorting and Counting  (Read 36422 times)

avk

  • Hero Member
  • *****
  • Posts: 752
Re: Sorting and Counting
« Reply #120 on: August 01, 2019, 06:09:12 pm »
@BrunoK, do you want the new version to be added to the benchmark and runned?

mpknap

  • Full Member
  • ***
  • Posts: 155
Re: Sorting and Counting
« Reply #121 on: November 09, 2019, 08:10:04 am »
Gentlemen, does anyone know how to write it in Python?

bytebites

  • Hero Member
  • *****
  • Posts: 633
Re: Sorting and Counting
« Reply #122 on: November 09, 2019, 09:46:44 am »
Without sorting
Code: Python  [Select][+][-]
  1. from collections import defaultdict
  2. d=defaultdict(int)
  3. with open("infile") as f:
  4.   for s in f:
  5.     k=s.strip(chr(10))
  6.     if k:
  7.       d[k]+=1
  8. with open("outfile","w") as o:
  9.   for (k,v) in d.items():
  10.     print(f'{k} - {v}',file=o)    

jamie

  • Hero Member
  • *****
  • Posts: 6090
Re: Sorting and Counting
« Reply #123 on: November 09, 2019, 03:36:14 pm »
Ugg , looks too much like BASIC.....
 >:(
The only true wisdom is knowing you know nothing

julkas

  • Guest
Re: Sorting and Counting
« Reply #124 on: November 09, 2019, 03:46:39 pm »
Ugg , looks too much like BASIC.....
 >:(
BASIC rocks.
BASIC is great.
BASIC forever.
« Last Edit: November 09, 2019, 03:55:14 pm by julkas »

mpknap

  • Full Member
  • ***
  • Posts: 155
Re: Sorting and Counting
« Reply #125 on: November 09, 2019, 09:05:41 pm »
Ugg , looks too much like BASIC.....
 >:(

but it works :)
only sorting necessary for me ....

440bx

  • Hero Member
  • *****
  • Posts: 3945
Re: Sorting and Counting
« Reply #126 on: November 09, 2019, 10:02:56 pm »
but it works :)
Just curiosity, it would be interesting to see how the performance of that Python implementation compares with the various Pascal implementations.
(FPC v3.0.4 and Lazarus 1.8.2) or (FPC v3.2.2 and Lazarus v3.2) on Windows 7 SP1 64bit.

avk

  • Hero Member
  • *****
  • Posts: 752
Re: Sorting and Counting
« Reply #127 on: November 10, 2019, 10:18:55 am »
You only have to wish, sir. :)
Python code:
Code: Python  [Select][+][-]
  1. from collections import defaultdict
  2. import timeit
  3. total=0
  4. d=defaultdict(int)
  5. stime=timeit.default_timer()
  6. with open("infile") as f:
  7.   for s in f:
  8.     total +=1
  9.     k=s.strip(chr(10))
  10.     if k:
  11.       d[k]+=1
  12. with open("outfile","w") as o:
  13.   for k in sorted(d.keys()):
  14.     print(f'{k} - {d[k]}',file=o)
  15. stime=timeit.default_timer()-stime
  16. print('Time elapsed: ', stime, ', #unique: ', len(d), ', #total: ', total)
  17.  
Output:
Code: Text  [Select][+][-]
  1. Time elapsed:  12.047126958000263 , #unique:  999955 , #total:  10000000
  2.  

Pascal code:
Code: Pascal  [Select][+][-]
  1. program sort_count;
  2. {$mode objfpc}
  3. {$MODESWITCH NESTEDPROCVARS}
  4. uses
  5.   SysUtils, DateUtils,
  6.   LGUtils, LGHashMultiSet, LGHelpers;
  7. procedure SortCount;
  8. type
  9.   TCounter  = specialize TGHashMultiSetLP<Integer>;
  10.   TCountRef = specialize TGAutoRef<TCounter>;
  11.   TEntry    = TCounter.TEntry;
  12.   function EntryCmp(constref L, R: TEntry): SizeInt;
  13.   begin Result := Integer.Compare(L.Key, R.Key); end;
  14. var
  15.   CountRef: TCountRef;
  16.   InOut: Text;
  17.   Counter: TCounter;
  18.   e: TEntry;
  19.   I: Integer;
  20.   stime: TTime;
  21. begin
  22.   Counter := CountRef;
  23.   Assign(InOut, 'infile');
  24.   Reset(InOut);
  25.   stime := Time;
  26.   while not EOF(InOut) do
  27.     begin
  28.       ReadLn(InOut, I);
  29.       Counter.Add(I);
  30.     end;
  31.   Close(InOut);
  32.   if Counter.NonEmpty then
  33.     begin
  34.       Assign(InOut, 'outfile');
  35.       Rewrite(InOut);
  36.       for e in Counter.Entries.Sorted(@EntryCmp) do
  37.         WriteLn(InOut, e.Key, ' - ', e.Count);
  38.       Close(InOut);
  39.     end;
  40.   WriteLn('Time elapsed: ', MillisecondsBetween(Time, stime)/1000:0:4,
  41.           ', #unique: ', Counter.EntryCount, ', #total: ', Counter.Count);  
  42. end;
  43. begin
  44.   SortCount;
  45. end.
  46.  
Output:
Code: Text  [Select][+][-]
  1. Time elapsed: 3.2300, #unique: 999955, #total: 10000000
  2.  

julkas

  • Guest
Re: Sorting and Counting
« Reply #128 on: November 10, 2019, 10:38:42 am »
but it works :)
Just curiosity, it would be interesting to see how the performance of that Python implementation compares with the various Pascal implementations.
Python sort implementation is exelent. It's based on Team sort algo.
https://stackoverflow.com/questions/1517347/about-pythons-built-in-sort-method
@avk Can you compare only sorting phase time?. (Python default I/O is slow).
Try with PyPy compiler also.
 
« Last Edit: November 10, 2019, 10:49:05 am by julkas »

440bx

  • Hero Member
  • *****
  • Posts: 3945
Re: Sorting and Counting
« Reply #129 on: November 10, 2019, 10:57:36 am »
You only have to wish, sir. :)
Thank you very much Avk. :)
(FPC v3.0.4 and Lazarus 1.8.2) or (FPC v3.2.2 and Lazarus v3.2) on Windows 7 SP1 64bit.

avk

  • Hero Member
  • *****
  • Posts: 752
Re: Sorting and Counting
« Reply #130 on: November 10, 2019, 11:12:46 am »
Can you compare only sorting phase time?. (Python default I/O is slow).
Sort phase only:
Python
Code: Text  [Select][+][-]
  1. Time elapsed:  1.1443881560007867 , #unique:  999955 , #total:  10000000
  2.  
Pascal
Code: Text  [Select][+][-]
  1. Time elapsed: 0.1750, #unique: 999955, #total: 10000000
  2.  

julkas

  • Guest
Re: Sorting and Counting
« Reply #131 on: November 10, 2019, 11:34:04 am »
Can you compare only sorting phase time?. (Python default I/O is slow).
Sort phase only:
Python
Code: Text  [Select][+][-]
  1. Time elapsed:  1.1443881560007867 , #unique:  999955 , #total:  10000000
  2.  
Pascal
Code: Text  [Select][+][-]
  1. Time elapsed: 0.1750, #unique: 999955, #total: 10000000
  2.  
@avk Thanks.
https://ideone.com/9n168K

Thaddy

  • Hero Member
  • *****
  • Posts: 14205
  • Probably until I exterminate Putin.
Re: Sorting and Counting
« Reply #132 on: November 10, 2019, 12:20:34 pm »
@avk Thanks.
https://ideone.com/9n168K
Julkas, ideone lags behind in compiler version for FPC and that can make a big difference. You should run the complete test suite on ideone to get any meaningful comparison.
Better run 3.2.0  or test everything with 3.0.4.
« Last Edit: November 10, 2019, 12:25:37 pm by Thaddy »
Specialize a type, not a var.

julkas

  • Guest
Re: Sorting and Counting
« Reply #133 on: November 10, 2019, 06:11:24 pm »
We can't compare Python (interpreted Lang) with Pascal.
Anyway, I use Python, I like Python.

Thaddy

  • Hero Member
  • *****
  • Posts: 14205
  • Probably until I exterminate Putin.
Re: Sorting and Counting
« Reply #134 on: November 10, 2019, 06:17:20 pm »
We can't compare Python (interpreted Lang) with Pascal.
Yes, we can since Python relies so heavily on library code compiled in other languages. Python is just glue. Much more so than pure scripting.
(You can also use FPC to write and add Python libraries)
And indeed:
I use Python and I like Python... :P :o
« Last Edit: November 10, 2019, 06:19:53 pm by Thaddy »
Specialize a type, not a var.

 

TinyPortal © 2005-2018