* * *

Author Topic: Freezing debugger when stepping because of a large object  (Read 1444 times)

JernejL

  • New member
  • *
  • Posts: 17
Freezing debugger when stepping because of a large object
« on: May 08, 2018, 10:13:49 am »
At one point i'm passing an object with a lot of data over stack, here is part of the debug log to show what i mean (the whole original line is 2289340 characters long):

Code: [Select]
<-stack-list-arguments 1 0 7>
^done,stack-args=[frame={level="0",args=[{name="MAP",value="{_vptr$TTOPDOWNCITYCUBEMAP = 0x0, CITYMAP = {{{{TERRAIN = 0, SIDES = {0, 0, 0, 0, 0}, WEST = 0, EAST = 0, NORTH = 0, SOUTH = 0, LID = 0} <repeats 39 times>, {TERRAIN = 0, SIDES = {0, 0, 0, 185, 0}, WEST = 0, EAST = 0, NORTH = 0, SOUTH = 185, LID = 0} <repeats 67 times>, {TERRAIN = 0, SIDES = {0, 0, 0, 0, 0}, WEST = 0, EAST = 0, NORTH = 0, SOUTH = 0, LID = 0} <repeats 51 times>, {TERRAIN = 32768, SIDES = {0, 0, 0, 0, 239}, WEST = 0, EAST = 0, NORTH = 0, SOUTH = 0, LID = 239} <repeats 99 times>}, {{TERRAIN = 0, SIDES = {0, 185, 0, 0, 0}, WEST = 0, EAST = 185, NORTH = 0, SOUTH = 0, LID = 0}, {TERRAIN = 0, SIDES = {0, 0, 0, 0, 0}, WEST = 0, EAST = 0, NORTH = 0, SOUTH = 0, LID = 0} <repeats 38 times>, {TERRAIN = 32768, SIDES = {0, 0, 0, 0, 0}, WEST = 0, EAST = 0, NORTH = 0, SOUTH = 0, LID = 0} <repeats 53 times>, {TERRAIN = 0, SIDES = {0, 0, 0, 0, 0}, WEST = 0, EAST = 0, NORTH = 0, SOUTH = 0, LID = 0} <repeats 65 times>, {TERRAIN = 32768, SIDES = {0, 0, 0, 0, 239}, WEST = 0, EAST = 0, NORTH = 0, SOUTH = 0, LID = 239} <repeats 99 times>}, {{TERRAIN = 0, SIDES = {0, 185, 0, 0, 0}, WEST = 0, EAST = 185, NORTH = 0, SOUTH = 0, LID = 0}, {TERRAIN = 0, SIDES = {0, 0, 0, 0, 0}, WEST = 0, EAST = 0, NORTH = 0, SOUTH = 0, LID = 0} <repeats 38 times>, {TERRAIN = 32768, SIDES = {0, 0, 0, 0, 0}, WEST = 0, EAST = 0, NORTH = 0, SOUTH = 0, LID = 0} <repeats 53 times>, {TERRAIN = 0, SIDES = {0, 0, 0, 0, 0}, WEST = 0, EAST = 0, NORTH = 0, SOUTH = 0, LID = 0} <repeats 65 times>, {TERRAIN = 32768, SIDES = {0, 0, 0, 0, 239}, WEST = 0, EAST = 0, NORTH = 0, SOUTH = 0, LID = 239} <repeats 99 times>}, {{TERRAIN = 0, SIDES = {0, 185, 0, 0, 0}, WEST = 0, EAST = 185, NORTH = 0, SOUTH = 0, LID = 0}, {TERRAIN = 0, SIDES = {0, 0, 0, 0, 0}, WEST = 0, EAST = 0, NORTH = 0, SOUTH = 0, LID = 0} <repeats 38 times>, {TERRAIN = 32768, SIDES = {0, 0, 0, 0, 0}, WEST = 0, EAST = 0, NORTH = 0, SOUTH = 0, LID = 0} <repeats 53 times>, {TERRAIN = 0, SIDES = {0, 0, 0, 0, 0}, WEST = 0, EAST = 0, NORTH = 0, SOUTH = 0, LID = 0} <repeats 65 times>, {TERRAIN = 32768, SIDES = {0, 0, 0, 0, 239}, WEST = 0, EAST = 0, NORTH = 0, SOUTH = 0, LID = 239} <repeats 99 times>}, {{TERRAIN = 0, SIDES = {0, 185, 0, 0, 0}, WEST = 0, EAST = 185, NORTH = 0, SOUTH = 0, LID = 0}, {TERRAIN = 0, SIDES = {0, 0, 0, 0, 0}, WEST = 0, EAST = 0, NORTH = 0, SOUTH = 0, LID = 0} <repeats 38 times>, {TERRAIN = 32768, SIDES = {0, 0, 0, 0, 0}, WEST = 0, EAST = 0, NORTH = 0, SOUTH = 0, LID = 0}

Whenever the object is in parameter list, whole debugger takes long time to step thru breakpoints because of object's "citymap" property and debug output window becomes totally unresponsive.
 
Code: Pascal  [Select]
  1.  
  2. type..
  3.  
  4. Tblock = packed record = record with a few arrays and structures
  5.  
  6. class object..
  7.  
  8. citymap:                                array[0..mapheight, 0..mapsize, 0..mapsize] of Tblock;     // the map (z, y, x)
  9.  
  10.  

Is there a way to improve, avoid or fix this? how could i avoid this, i really do need a large 3d array in the object to hold my map data.
 
« Last Edit: May 08, 2018, 10:16:09 am by JernejL »

Martin_fr

  • Administrator
  • Hero Member
  • *
  • Posts: 4666
    • wiki
Re: Freezing debugger when stepping because of a large object
« Reply #1 on: May 08, 2018, 10:27:39 am »
In Lazarus svn trunk you can try setting
   GdbValueMemLimit
(in the property grid, in Tools > Options > Debugger)
to a lower value.


You can also try to set the field "debugger_startup_options"
    --eval-command="set print elements 10"
(or whatever number you prefer)

marcov

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 6328
Re: Freezing debugger when stepping because of a large object
« Reply #2 on: May 08, 2018, 04:43:10 pm »
Don't allocate it on the stack, allocate it on the heap.

JernejL

  • New member
  • *
  • Posts: 17
Re: Freezing debugger when stepping because of a large object
« Reply #3 on: May 08, 2018, 08:20:50 pm »
Don't allocate it on the stack, allocate it on the heap.

 
I'm not sure what you mean, the object is a global variable in a unit.
 
also, this would not solve problem with gdb choking lazarus ide with too much data.
 

JernejL

  • New member
  • *
  • Posts: 17
Re: Freezing debugger when stepping because of a large object
« Reply #4 on: May 20, 2018, 05:28:08 pm »
I suggest we put a warning / notice into debugger log if it takes too much time to get debugger data, it would help a lot to fix these freezing if it happens to other people.

Martin_fr

  • Administrator
  • Hero Member
  • *
  • Posts: 4666
    • wiki
Re: Freezing debugger when stepping because of a large object
« Reply #5 on: May 20, 2018, 07:40:09 pm »
Displaying a warning helps only in the sense that people know they still need to wait for an unknown amount of time.
Since the command is already running in gdb, afaik there is no way to stop it, so gdb is still blocked until it finishes.

Did you try any of my above suggestions? Trying to limit the amount of work done by gdb?

JernejL

  • New member
  • *
  • Posts: 17
Re: Freezing debugger when stepping because of a large object
« Reply #6 on: May 20, 2018, 08:02:05 pm »
yeah, it helped to set the limit, but since the gui freezing is pretty serious, a warning would explain people what the issue is and how to resolve it.

Just a check if it took a lot of time and a lot of data was transferred over from gdb, then show a warning about what is the cause.
« Last Edit: May 20, 2018, 08:04:41 pm by JernejL »

Martin_fr

  • Administrator
  • Hero Member
  • *
  • Posts: 4666
    • wiki
Re: Freezing debugger when stepping because of a large object
« Reply #7 on: May 20, 2018, 11:25:35 pm »
You used
  GdbValueMemLimit
or
  --eval-command="set print elements 10"
?

If this helps, then maybe it should be set by default to a lower value, so the problem doesnt happen at first. (Ideally, but I am not sure if that is easy to do, if the limit is hit, it should show a message...)

I will need to do some testing of my own, to see what is possible.
Also maybe set a lower value for getting the stack. Then explicit watches will still work (even if slow / or at a medium value)
« Last Edit: May 20, 2018, 11:28:06 pm by Martin_fr »

JernejL

  • New member
  • *
  • Posts: 17
Re: Freezing debugger when stepping because of a large object
« Reply #8 on: May 21, 2018, 07:55:51 am »
I used "set print elements"
 
Here is example of what causes massive slowdowns:
 
Code: Pascal  [Select]
  1.  
  2. const
  3.     mapheight = 7;
  4.         mapsize = 255;
  5.  
  6. type
  7.  
  8. TTerrainType = word;
  9.  
  10.     Tblock = packed record
  11.         Terrain: TTerrainType;
  12.         case boolean of
  13.             True:
  14.                 (Sides: array[0..4] of TBlockFace);
  15.                         False:
  16.                 (West, East, North, South, Lid: TBlockFace);
  17.     end;
  18.  
  19.         TCityMapGrid = array[0..mapheight, 0..mapsize, 0..mapsize] of Tblock;
  20.  
  21.         TTopDownCityCubeMap = object
  22.         public
  23.                 citymap:                                TCityMapGrid;     // the map (z, y, x)
  24.         end;
  25.  
  26.  

Create an instance of TTopDownCityCubeMap in a unit global variable, then pass it as a parameter to one function, and have it breakpoint in that function, it will lockup for up to half a minute before debugger becomes responsive.
 

Martin_fr

  • Administrator
  • Hero Member
  • *
  • Posts: 4666
    • wiki
Re: Freezing debugger when stepping because of a large object
« Reply #9 on: June 09, 2018, 07:49:25 pm »
I implemented a limit in rev 58212.

The limit applies to
- stack window
- thread window (though this did not show large data anyway
- locals window

The watches windows is not protected (well partly). You add your structure there, you have to wait. (But then the lesson should be learned and you do not add it again).

=============
The problem with the max array len is that gdb applies it for each recursion level.
On a flat array, a limit of 10 (locals window) is to low. So the default limit is 25, but in your case that is 25*25*25 which already takes 2 or 3 seconds for gdb.

So for stack/local there is also a separate memory limit (for each object). This is set to ~30k. In your case it leads to the object not show, but instead an error.

You can adjust both values in the options dialog:
    MaxLocalsLengthForStaticArray
    LocalsValueMemLimit

The limits can also be set for watches, but with diff values
    MaxDisplayLengthForStaticArray
    GdbValueMemLimit





JernejL

  • New member
  • *
  • Posts: 17
Re: Freezing debugger when stepping because of a large object
« Reply #10 on: July 16, 2018, 10:01:04 am »
This is awesome, i didn't expect a patch for this - but i'm really happy that this was made, i'll switch to trunk build soon, as i also need trunk stuff to install some profiler addon package.
 

 

Recent

Get Lazarus at SourceForge.net. Fast, secure and Free Open Source software downloads Open Hub project report for Lazarus