Recent

Author Topic: Profiling tool for Lazarus applications  (Read 1023 times)

Mobius1

  • New Member
  • *
  • Posts: 10
Profiling tool for Lazarus applications
« on: November 09, 2024, 08:46:50 am »
Hi everyone,

I need to know if the tool exists, where can I find it. I need a profiling tool that works on Lazarus applications in the latest stable version of both Lazarus and FPC.

LazProfile, fpProfile doesn't work, the first one is 5yo and doesn't compile, the second one is 14yo and has a few package errors.
I tried running the application with valgrind on Linux, also allowing the -gv on the console app but the valgrind throws an error like it couldn't run the application.

We're developing a component package that is being slow compared to its ideal speed margin, so I need a tool that helps me detect where in the code the slowness is happening.

Thanks in advance

zamtmn

  • Hero Member
  • *****
  • Posts: 639
Re: Profiling tool for Lazarus applications
« Reply #1 on: November 09, 2024, 10:10:00 am »
Pay attention here https://forum.lazarus.freepascal.org/index.php/topic,68479.0.html, these are not finished yet, but very powerful tools

Martin_fr

  • Administrator
  • Hero Member
  • *
  • Posts: 10542
  • Debugger - SynEdit - and more
    • wiki
Re: Profiling tool for Lazarus applications
« Reply #2 on: November 09, 2024, 11:47:52 am »
If you can run your app on Linux: Use valgrind --tool=callgrind  and the kcachegrind to view the results.


ALLIGATOR

  • New Member
  • *
  • Posts: 24
Re: Profiling tool for Lazarus applications
« Reply #3 on: November 09, 2024, 02:20:23 pm »
If needed, I can help you build this profiler (that @zamtmn suggested) under Windows (so far it is only Windows compatible) with FPCUpDeluxe & TortoiseGit (just these tools I use) and show you how to use it.

Mobius1

  • New Member
  • *
  • Posts: 10
Re: Profiling tool for Lazarus applications
« Reply #4 on: November 09, 2024, 11:10:16 pm »
If you can run your app on Linux: Use valgrind --tool=callgrind  and the kcachegrind to view the results.

Like I wrote on my post: I tried using that approach but valgrind throws an error and won't run the application:
https://prnt.sc/ZLOsSjKmWWJV
I don't know how to make it work with valgrind, my application is a console app that requires some user inputs before starting, after it starts is where I wanna find the slowness

If needed, I can help you build this profiler (that @zamtmn suggested) under Windows (so far it is only Windows compatible) with FPCUpDeluxe & TortoiseGit (just these tools I use) and show you how to use it.

Thanks, but the specific slowness happens within Linux, more specifically Ubuntu 20.04 LTS and 22.04 LTS versions which are the ones I have to test, I could help you test that tool if you somehow make it work with Linux, just lemme know.

Martin_fr

  • Administrator
  • Hero Member
  • *
  • Posts: 10542
  • Debugger - SynEdit - and more
    • wiki
Re: Profiling tool for Lazarus applications
« Reply #5 on: November 09, 2024, 11:51:16 pm »
Ok, sorry I missed that at first.
I have never had that, and I valgrinded a ton of fpc compiled stuff.

Which version of fpc do you use, which optimization level (any specific cpu/fpc choices)?

It also looks strange to me, because I wouldn't expect a SigKill to cause an access violation.

Is the address always the same? Have you tried to load it in any debugger and find out what code (if any) is at that location? And that it is at the begin of an asm instruction? I.e. if you find the start of the function and disassemble, it has a statement starting at this address?

That might also help googling if the given statement might be an issue for valgrind.

----

About Optimization.

Since you are looking for a slow down, you may have the code compiled with the settings for performance (e.g. O2 or above). And valgrind usually has no issue with that.
But, fpc 3.2.2 older 3.2.3 (and some revisions of 3.3.1) may generate invalid code in very rare cases when optimizing.

Keep in mind, that if there is a bug in your code like dangling/uninitialised pointer, then it may not trigger in any issues in normal runs off the app. Valgrind may however change memory layout (as it keeps freed memory to detect invalid access / at least with memcheck). So at least under memcheck such a bug may suddenly cause issues.

I don't know if this can happen with callgrind or cachegrind.

However given speed differences, if you have multi threading, thread related errors might suddenly manifest.

So there is a chance that something is wrong in your code (or the compiler introduced an error).
Have you tried a valgrind memcheck?

Mobius1

  • New Member
  • *
  • Posts: 10
Re: Profiling tool for Lazarus applications
« Reply #6 on: November 10, 2024, 02:43:24 am »
Which version of fpc do you use, which optimization level (any specific cpu/fpc choices)?
FPC fixes (3.2.3) and Lazarus fixes (3.7) The default Runtime mode for Linux x86_64 target

1) Is the address always the same?
2) Have you tried to load it in any debugger and find out what code (if any) is at that location?
3) And that it is at the begin of an asm instruction?
4) I.e. if you find the start of the function and disassemble, it has a statement starting at this address?
1) If you mean memory address I have no idea, I don't mess with memory address allocation.
2) This I have no idea on how to do or what it means, sorry
3) I have no asm code in my app
4) I don't know

----
About Optimization.

Since you are looking for a slow down, you may have the code compiled with the settings for performance (e.g. O2 or above). And valgrind usually has no issue with that.
But, fpc 3.2.2 older 3.2.3 (and some revisions of 3.3.1) may generate invalid code in very rare cases when optimizing.

Keep in mind, that if there is a bug in your code like dangling/uninitialised pointer, then it may not trigger in any issues in normal runs off the app. Valgrind may however change memory layout (as it keeps freed memory to detect invalid access / at least with memcheck). So at least under memcheck such a bug may suddenly cause issues.

I don't know if this can happen with callgrind or cachegrind.

However given speed differences, if you have multi threading, thread related errors might suddenly manifest.

So there is a chance that something is wrong in your code (or the compiler introduced an error).
Have you tried a valgrind memcheck?
The code might be bugged, yes, but it doesn't show on the compiler and the stresstest doesn't throw errors, using jMeter it shows 0.00% errors.

The "slowness" is because on a previous version of the framework (our package is a framework) it was averaging between 0 and 4ms on average across 30s of stresstesting with maximum of 120ms and minimum of 0. Now after a few new features implemented, it was expected to have it averaging between 4ms and 10ms but its averaging between 60ms and 70ms, so I'd wanna know which part of the code is slowing things down. The same code works for Lazarus and Delphi, but on Delphi windows its running on the expected margin in some engines, but very slow on others. Lazarus Linux is slower across the board on all of them

Martin_fr

  • Administrator
  • Hero Member
  • *
  • Posts: 10542
  • Debugger - SynEdit - and more
    • wiki
Re: Profiling tool for Lazarus applications
« Reply #7 on: November 10, 2024, 12:19:21 pm »
Before I go into my response. Most of it is only valuable if either
- you are fine spending some time in the hope getting valgrind to work
- you want to pursuit this on the (probably rather very low) off chance that there is a hidden bug in your code (or - even less likely - in fpc 3.2.3 itself)

Otherwise, you may be better off investing time in one off the other tools.


1) Is the address always the same?
2) Have you tried to load it in any debugger and find out what code (if any) is at that location?
3) And that it is at the begin of an asm instruction?
4) I.e. if you find the start of the function and disassemble, it has a statement starting at this address?
1) If you mean memory address I have no idea, I don't mess with memory address allocation.
2) This I have no idea on how to do or what it means, sorry
3) I have no asm code in my app
4) I don't know

1)
Your image shows an address: 40ee8c

I wonder, if there is another. Google indicates there should be (before the text from your screenshot)
" valgrind: Unrecognised instruction at address: xxxxx"

2)
If you use the same exe (no recompile) running in the debugger and you pause it, then you can enter that address in the asm view and see where it points too.
Alternatively you can copy the callstack from the image, open View > Leaks and traces => paste it, and press "Resolve" and find the executable of your project (the same that threw the error, no recompile)

3)
I mean the asm generated by the compiler (as machine code / not readable asm / but then disassembled)
if you have that function, then you disassemble (using the debugger) from the begin of that function.

If disassemble from the begin of the function then 40ee8c may be at the begin of one of the statements shown (good).

Or there may be a statement at 40ee8B (or before) and the next statement is after the wanted address (bad / 99.99% likely something wrong in your executable)


4)
if there is a statement at this address, then it will have a name (e.g. "lea"). That name can be googled with the term "valgrind".
If it is an issues in valgrind, other may have had it too.




I understand it passed all your tests. And I will assume at this point that your tests are extensive, and have excellent coverage and all.

I would still keep open to the idea that there might be a bug in there. Even if it may not be likely.
I have seen very elusive bugs.

On the other hand, I don't know how up to date valgrind is on the Ubuntu 20. Since it is LTS, I would expect them to keep at least bugfixes for installed software...
There are release notes here https://valgrind.org/docs/manual/dist.news.html  (and the latest release is very new, matching the kind of error you got)
Search for "unhandled". There are quite a few fixed in the latest release.
If you get the asm instruction (and hex bytes) from the disassembler, then you can compare. (assuming that the given address is actually the one at which valgrind stopped)




Last not least, if valgrind does not understand a valid instruction, and if it either isn't fixed or you can't update, then you could try to get FPC to avoid that instruction.
For example (from the fixes in the release notes) " vmovq %xmm9,%xmm1", you may be able to disable fpu or sse usage or something.

Or compile with -Cp____ / Get the list of names with: fpc -ic

Mobius1

  • New Member
  • *
  • Posts: 10
Re: Profiling tool for Lazarus applications
« Reply #8 on: November 26, 2024, 01:54:30 am »
Sorry for the late response, being busy these weeks. I was able to find part of the slowness and fixed it but I still need to work with a profiler to see if there are any more areas that needs improvements.

I'd like to make valgrind work, I just don't have enough time right now, I read what @Martin_fr posted but I still don't know why the error pops up.

The application that I'm trying to profile is a console app that expects user to input a code from 1 to 4 on the first option and then choose a port for the server to run. After that I throw some requests with jMeter and wanna see which parts of the code are taking longer to run.

cdbc

  • Hero Member
  • *****
  • Posts: 1644
    • http://www.cdbc.dk
Re: Profiling tool for Lazarus applications
« Reply #9 on: November 26, 2024, 07:49:18 am »
Hi
Quote
The application that I'm trying to profile is a console app that expects user to input a code from 1 to 4 on the first option and then choose a port for the server to run.
Just for testing, hardcode these and get on with it...
Just my 2 cents worth.
Regards Benny
If it ain't broke, don't fix it ;)
PCLinuxOS(rolling release) 64bit -> KDE5 -> FPC 3.2.2 -> Lazarus 2.2.6 up until Jan 2024 from then on it's: KDE5/QT5 -> FPC 3.3.1 -> Lazarus 3.0

 

TinyPortal © 2005-2018