What's the best way to understand execution flow?

[EganSolo]

Say you're looking at a method in a class and you're wondering what the full execution flow looks like starting at that method. By Execution Flow, I mean the tree of method calls, limited to the code in the project. For instance, if I'm calling IntToStr, I don't want to dip inside that code.

That's not quite like profiling, which is focused on performance. That's focused on logic and semantics.

Suggestions?

Thanks!

[440bx]

That's not quite like profiling, which is focused on performance. That's focused on logic and semantics.

Suggestions?

Your mention of profiling shows that you're aware of the similarities between what you're asking for and how a profiler does its job.

The only suggestion that comes to mind and, it involves a fair initial amount of work on your part, is to _modify_ a profiler such as LazProfiler to log the calls instead of measuring time elapsed in individual methods.  if interested in that idea, you can find information about LazProfiler at:
https://wiki.freepascal.org/LazProfiler

That's the only thing that comes to mind.

Static analysis of the code does not work because of polymorphism, therefore execution flow can only be determined at runtime.

HTH.

[ALLIGATOR]

There's also this
https://github.com/Alligator-1/cpuprofiler

[n7800]

Say you're looking at a method in a class and you're wondering what the full execution flow looks like starting at that method. By Execution Flow, I mean the tree of method calls, limited to the code in the project. For instance, if I'm calling IntToStr, I don't want to dip inside that code.

That's not quite like profiling, which is focused on performance. That's focused on logic and semantics.

I'm not sure what exactly you mean:
* the call chain from which the current method is called. To do this, you can simply set a breakpoint in it and look at the call stack.
* the call chain that this method makes. Each line of the method can make a large call tree.

In any case, both chains may not be unique (not the only ones). For example, due to conditions:

Code: Pascal  [Select][+][-]

1. if i > 5 then
2.   Test;

If i is greater than 5, the Test method will be called; otherwise, it won't. But it could be called on a different line or from a different method.

All this creates a huge number of possible combinations leading to the same function call. Sometimes, the conditions may even be weakly deterministic, depending not only on the input data but also on time, execution latency, network data, or random variables.

And of course, there are nuances with recursive calls (including indirect ones), function pointers and many others.

In practice, of course, there may not be many, but theoretically...

[440bx]

@ALLIGATOR,

Your profiler looks interesting. 

Do you plan on adding a short description of how to use it ?... that would be nice

[LeP]

That's not quite like profiling, which is focused on performance. That's focused on logic and semantics.
Suggestions?

Look at this: https://wiki.freepascal.org/UML,_modelling_tools,_code_generators_and_code_analysis_tools

[Martin_fr]

Static analysis of the code does not work because of polymorphism, therefore execution flow can only be determined at runtime.

That depends... Execution may not have 100% code coverage. Especially not in some single test run... So execution charts will be incomplete. But they will show details for a specific case.
The complexness of static analysis with regards to OOP depends on how many virtual methods this particular method calls (including indirect in nested callees).

And if e.g. you know what (minimum) sub-class the code is dealing with in any case you try to analyze => then you can edit the declaration(s) of those variables to be of that class, and codetools will do the correct lookup.
If it is self (or some var, where you do not want to edit the declaration) then use syncro edit (to change all occurrences in the function at once), and temporarily change the identifier from "self" to "TFooClass(self)". Then codetool will work on "TFooClass(self).SomeVirtualMethod".


There is another profiler which can help with code flow. But it only works if you can run on Linux:
  callgrind from the valgrind suite

And then: kcachegrind.
https://kcachegrind.github.io/html/CallGraph.html

It has a nice tree view of who called who. Albeit, it does not include the order inside each method in which each callee was invoked.


As a different approach you can add logging. Its a bit of work. But using code template and/or editor-macros, it becomes reasonable effort. If you know some script language with really good reg-ex (needs recursive matching), then you can even write a small script, that changes an entire unit at once.

Just add at the start/end of each unit

Code: Pascal  [Select][+][-]

1. uses LazLogger;
2. procedure Foo;
3. begin
4.   debuglnEnter(['> Foo ',dbgsName(Self),dbgs(self),' ' ]); try
5.   //...
6.   finally debuglnExit(['< Foo '| ]); end;
7. end;

The debuglnEnter/Exit pair will nicely indent any log entries between them. So the log will be easy to read

From the lazarus.dci file (in the primary config path)

Code: Text  [Select][+][-]

1. [[det | debuglnEnter try]
2. $(AttributesStart)
3. EnableMakros=true
4. $(AttributesEnd)
5. debuglnEnter(['> $ProcedureName() ',dbgsName(Self),dbgs(self),' '| ]); try
6.  
7. [fdx | DebuglExit finally]
8. $(AttributesStart)
9. EnableMakros=true
10. $(AttributesEnd)
11. finally debuglnExit(['< $ProcedureName() '| ]); end;
12.  

[ALLIGATOR]

Your profiler looks interesting

Yes, it's quite interesting for some applications... for example, to try to understand where everything is slowing down

Or to understand the structure of the program, or as the topic starter says, the execution flow

Regarding profiling, since this is not a sampling profiler but an instrumenting profiler, it introduces some overhead into the executing code, into each function, and this needs to be understood

For some tasks, I recently switched to Intel VTune, I didn't know it had become free - it's a sampling profiler, and it also has hardware support

For AMD CPUs, there is a profiler similar to Intel VTune

And for Linux, there is perf

VTune has a problem - it doesn't understand the DWARF debugging information generated by FPC very well (and maybe DWARF in general, not just the one generated by FPC, I haven't checked anything other than FPC) - so the profiling results often only show addresses, not function names and line numbers.

... how to use it?

I tried to write instructions, but I may have missed something or written something that is not entirely clear

https://github.com/Alligator-1/gists/blob/main/CPUProfiler%20QuicStartGuide%20En/readme.md

Feel free to ask questions if you encounter any problems - I will try to answer them

[cdbc]

Hi
Have a look in THIS thread, at reply #123 from @Janasoft, he uses @Martin_fr's suggestion of using 'LazLogger', to get an overview of the flow in an application... Maybe that's what you're after... 
Regards Benny

[440bx]

That depends... Execution may not have 100% code coverage. Especially not in some single test run... So execution charts will be incomplete. But they will show details for a specific case.
The complexness of static analysis with regards to OOP depends on how many virtual methods this particular method calls (including indirect in nested callees).

Function pointers really complicate execution tracing.  If function pointers are not used then it is possible to determine every possible execution path and graph that, but... it can still be complicated but, at least it is deterministic and possible.


@ALLIGATOR,

Thank you.  That additional information is very helpful.  I was looking for it in the original link you posted.  I think you should make that information visible in that link or put the link there.


@EganSolo

I forgot that I needed a way to trace calls not that long ago and coded a very basic facility that served my needs.  Just in case you can use it, it is located at:
https://forum.lazarus.freepascal.org/index.php/topic,68636.msg531493.html#msg531493

It's very basic but it's extremely easy to use.  It does take a little bit of work to get the desired information because 2 lines of code have to be manually added to the functions to trace (note that the process of adding the instructions can be automated.)  One of the nice things is the code doesn't need to be manually removed from the source to no longer be part of the executable when tracing is no longer desired.

HTH.

[EganSolo]

Thanks, everyone, for your thoughtful and helpful answers. That helped me sharpen what I was looking for. Let's see if this resonates.
The app I'm working on has, as expected, loads of units and some of the use cases traverse several units and involve many classes and methods.
Instead of chasing the flow all over the place, I'd like a view that starts with a method of interest and provides a collapsed view of the calling sequences. Note: I don't want the view to be semantically smart, trying to figure out which overloaded method to call. I want that view to collate all the methods that could be involved and present them in a collapsible view. An example will help explain what I'm after:

Code: Pascal  [Select][+][-]

1. Type
2.   Class1 = Class
3.     Procedure MethodA1;
4.     function MethodB1: Boolean;
5.     Procedure MethodC1;
6.   end;
7.  
8.   Class2 = Class
9.   private
10.     FObect1 : Class1;
11.   public
12.    Procedure MethodA2;
13.    Procedure MethodB2;
14.   end;
15.  
16. Procedure Class1.MethodA1;
17. begin
18.   //Code not involving methods from Class1 or Class2
19. end;
20.  
21. function Class1.MethodB1: Boolean;
22. begin
23.   //Code not involving methods from Class1 or Class2
24. end;
25.  
26. Procedure Class1.MethodC1;
27. begin
28.   //Code not involving methods from Class1 or Class2
29. end;
30.  
31. Procedure Class2.MethodA2;
32. begin
33.    If X > Y
34.    then begin
35.        //Code block not involving methods from Class1 or Class2
36.       fObject1.MethodA1
37.       //More code not involving methods from Class1 or Class2
38.    end
39.    else MethodB2;
40. end;
41.  
42. Procedure Class2.MethodB2;
43. begin
44.   If fObject1.MethodB1
45.   then begin
46.     // Code block here that doesn't call any method of Class1 or 2.
47.     MethodA2;
48.   end
49.   else fObject1.MethodA1;
50. end;
51.  

Imagine that Class1 and Class2 are in different units. I'd point the (imaginary) code explorer to Class2.MethodB2. I'd configure it to ignore all calls except for those involving methods from Class1 and Class2. It would produce the following view:

Procedure Class2.MethodB2
   If [ + ] fObject1.MethodB1 (it copies the conditional without understanding it)
   then [ + ] MethodA2 (it ignores code that doesn't involve methods from Class1 and Class 2
   else  [ + ] fObject.MethodA1

If it encounters an expression involving a method call from Class1 and Class2, it offers the option to include the expression verbatim or strip it down to the methods of Class1 and Class2.

Notice the [ + ]: If I click on them, they expand the methods by showing me the same structure, so I can follow the calling sequence all in one place, without having to jump around. It would also be nice to associate that summarized view with a commentary to help me keep track of the flow.

Does this make sense?