Recent

Author Topic: Class works much slower in main project than in test project  (Read 3630 times)

kr1d

  • New Member
  • *
  • Posts: 11
Class works much slower in main project than in test project
« on: March 07, 2019, 05:45:23 am »
Testing using same input data.
By counting time difference I found out that "for" loop inside main method of my class works slower and slower from iteration to iteration. Again: this class with same data works fine in test project and can be run multiple times.
Heaptrc found nothing.

Any idea where to look at?

Handoko

  • Hero Member
  • *****
  • Posts: 5131
  • My goal: build my own game engine using Lazarus
Re: Class works much slower in main project than in test project
« Reply #1 on: March 07, 2019, 06:19:32 am »
When asking question related with your source code problem, you need to show us the code. Without source code to test, not much we can say.

Show us the code.

kr1d

  • New Member
  • *
  • Posts: 11
Re: Class works much slower in main project than in test project
« Reply #2 on: March 07, 2019, 06:35:33 am »
Cant show the code, but lets think in this way:
Myclassunit.pas used in two projects.
In both projects execution of main method in this class is looks like this:
MyClassVar.ParseData('data.txt');
File data.txt is same file used in both projects has same content.
Inside ParseData I have few stringlists and one big "for" loop.
To run ParseData in both projects I execute that method in tform.OnClick.
So its identical run in both project.

Whats different is that main project is much larger in its size. More functions and stuff like that.
This class is runned inside second or third form, not that mainform.

Bram71

  • New Member
  • *
  • Posts: 26
Re: Class works much slower in main project than in test project
« Reply #3 on: March 07, 2019, 09:32:19 am »
Indeed without (example) code we can´t tell

Are any gui controls associated with the string lists? They may be triggered to also update as the content of the lists changes / grows. Try adding beginupdate and endupdate for the lists and do the parse inbetween... Also check if the same memory manager is used (cmem or not).

Thaddy

  • Hero Member
  • *****
  • Posts: 14213
  • Probably until I exterminate Putin.
Re: Class works much slower in main project than in test project
« Reply #4 on: March 07, 2019, 09:34:24 am »
Without code we can't do much, but it is likely that you forgot a lot a lot of string parameters (or other refcounted types) to be declared as const parameters, which will lead to an infinite amount of copying, especially when nested. But that's still just a guess. We really need a code example for sound advice. The guessed scenario about const passing will not lead to memory leaks shown by heaptrc, but will lead to resource exhaustion and so to the use of OS swap use. In that case it is the swapping in and out of physical memory that would cause a significant slow-down of your application and while there are still no real memory leaks.
E.g.:
Code: Pascal  [Select][+][-]
  1. {$mode objfpc}{$H+}
  2. // not this
  3. function SomeFunction(s:string):string;
  4. //but like this
  5. function SomeFuntion(const s:string):string;
The first will easily lead to resource exhaustion when nested, the second won't...Both will not cause real memory leaks, but the second example is much more resource efficient.
A single function that gets called very often will show a significant slow down because what I described above. So examine your code and correct where appropriate.

But it remains just a guess by an old hand...Old hands are often right, but not always. Without seeing actual code, that is.

Note it does not matter what kind of function or procedure: they may be class members too. If possible adjust your code to use the second pattern.
« Last Edit: March 07, 2019, 10:04:02 am by Thaddy »
Specialize a type, not a var.

kr1d

  • New Member
  • *
  • Posts: 11
Re: Class works much slower in main project than in test project
« Reply #5 on: March 07, 2019, 10:03:28 am »
Details about function: I'll check it. But again as I said, iteration count, data and sll that stuff is identical in both projects.
Gui problrm: no, because I need to parse datavfirst and then I will use different class to show it.
So its not gui for dure as thrre is zero gui functions inside this class.
Its purely parsing class.

On different platform there is suggestion that I should use lists not arrays, because I reallocate array with every new setlength(a, length(a)+1); and I use huge amount of data, and memory can be fragmented, thats why I get this degradation in speed.

Currently Im trying to rewrite my code with list usage to test that suggestion.

Thaddy

  • Hero Member
  • *****
  • Posts: 14213
  • Probably until I exterminate Putin.
Re: Class works much slower in main project than in test project
« Reply #6 on: March 07, 2019, 10:06:27 am »
The fact that it is a parsing class made me aware of the possible issue I described...... :D ;D
A simple example would not show your problem immediately, but it will show in a more complex application.
I suggest to monitor memory use. But now I am even more convinced that my guess is right. O:-)
Reallocating dynamic arrays on iteration is, frankly, really really bad programming....and can also show a slowdown because of memory fragmentation indeed.(a manifestation of resource exhaustion, btw)
A list might be an option but only when you manage its capacity correctly. Growing a list - when not managed properly - can be just as expensive as growing a dynamic array, so set the capacity beforehand to a sufficient size.
But indeed this
Quote
I reallocate array with every new setlength(a, length(a)+1);
is not very smart...
At least lists grow in blocks so that should show a speed-up.
« Last Edit: March 07, 2019, 10:20:41 am by Thaddy »
Specialize a type, not a var.

kr1d

  • New Member
  • *
  • Posts: 11
Re: Class works much slower in main project than in test project
« Reply #7 on: March 07, 2019, 11:36:56 am »
Ok, test results:

Main project:
Setlength(a, 5000); // minus 5-8 secs
function param const // minus 3 secs

Test project:
Whole parsing in 1 sec. // 2-5 secs before

So difference between test project and main project is still huge: ~ 1 min 10 secs vs 1 sec

Testing lists now.
« Last Edit: March 07, 2019, 12:34:14 pm by kr1d »

Handoko

  • Hero Member
  • *****
  • Posts: 5131
  • My goal: build my own game engine using Lazarus
Re: Class works much slower in main project than in test project
« Reply #8 on: March 07, 2019, 11:43:47 am »
You mentioned lists. Did you mean TList?

If it is possible use TFPList. It is a bit faster. I ever performed tests on a game I wrote, it is about 3% faster than TList.

Martin_fr

  • Administrator
  • Hero Member
  • *
  • Posts: 9794
  • Debugger - SynEdit - and more
    • wiki
Re: Class works much slower in main project than in test project
« Reply #9 on: March 07, 2019, 01:20:23 pm »
I reallocate array with every new setlength(a, length(a)+1); and I use huge amount of data, and memory can be fragmented, thats why I get this degradation in speed.

Every "setlengh" may copy the entire content of the array. The bigger the array, the longer it takes. So with growing data this becomes notably slower.

For all else, some guesses:

Also overall growth of your app may affect speed.
More code run inside the loop, may mean less efficient use of registers. (Even more code outside your loop but in the same procedure, can lead to this).

And last, even processing the same data by the same code, can depend on the size of data. If the data you access is close together in memory, it will read by the cpu with very few cache fetches.
If the same data is all over the place, the cpu needs to do more cache fetches.

kr1d

  • New Member
  • *
  • Posts: 11
Re: Class works much slower in main project than in test project
« Reply #10 on: March 11, 2019, 08:48:03 am »
It seems I miss some basics, what's wrong with this code?
In WorkA method A array has 0 length.
While when I execute FillA it shows that array is filled.
//

Myclass=class
  A:array of classb;
  procedure FillA;
  procedure WorkA(const ii: integer);
End;

procedure Myclass.FillA;
Var ii: integer;
Begin
  Setlength(A,5000);
  For ii:=0 to 5000-1 do
  A[ii]:=classb.create;
End;

procedure Myclass.WorkA(const ii:integer);
Begin
  If ii<5000 then
  A:[ii].classbField:=calcSomething;
End;

« Last Edit: March 11, 2019, 11:32:18 am by kr1d »

Thaddy

  • Hero Member
  • *****
  • Posts: 14213
  • Probably until I exterminate Putin.
Re: Class works much slower in main project than in test project
« Reply #11 on: March 11, 2019, 11:18:36 am »
You miss the point that local variables are not global variables.
Specialize a type, not a var.

kr1d

  • New Member
  • *
  • Posts: 11
Re: Class works much slower in main project than in test project
« Reply #12 on: March 11, 2019, 11:34:18 am »
You miss the point that local variables are not global variables.
I'm on mobile, edited. FillA  and WorkA are methods of MyClass.
A is field of class(array that stores data).
« Last Edit: March 11, 2019, 11:42:53 am by kr1d »

 

TinyPortal © 2005-2018