Recent

Author Topic: Recompile the RTL to debug  (Read 27330 times)

dbannon

  • Hero Member
  • *****
  • Posts: 2778
    • tomboy-ng, a rewrite of the classic Tomboy
Recompile the RTL to debug
« on: December 11, 2017, 11:42:00 am »
Hi Folks, I was advised http://forum.lazarus.freepascal.org/index.php/topic,38917.0.html that I need to recompile the RTL and enable debugging so that heaptrc in Lazarus will show me source file names and line numbers of a memory leak in TTrayIcon, an LCL component.

On a Mac. Sigh ....

I am unsure just how this should be done so hope you don't mind if I re-ask the question I asked in the above mentioned discussion.

I do have fpcsrc installed, its in /usr/local/share/ so, I need recompile with -gl, find the problem (yeah, sure) and then recompile again without that switch so that my normally resulting binaries are not bloated. Or should I compile a different tree, one with debug symbols and switch between ?

https://freepascal.org/docs-html/current/prog/progsu219.html#x331-347000F.4.1 says I need to compile -

System Unit - /usr/local/share/fpcsrc/rtl/macos/system.pp
Strings Unit - /usr/local/share/fpcsrc/rtl/inc/strings.pp
Doc unit - /local/share/fpcsrc/rtl/macos/dos.pp
Objects Unit - /usr/local/share/fpcsrc/packages/rtl-extra/src/inc/objects.pp

Same document suggests a command line using ppc386, as I’m using cocoa, thats all I’d need. eg

Code: [Select]
ppc386 -gl -Tdarwin -b- -Fi../inc -Fi../i386 -FE. -di386 -Us -Sg system.ppI have added -gl in there and changed target to darwin. Just where would I run this command ?

Or should I use -
/local/share/fpcsrc/rtl/darwin/Makefile.fpc and define DEBUG as per
https://freepascal.org/docs-html/current/prog/progsu206.html#x311-327000E.6.2

Davo

Lazarus 2, Linux (and reluctantly Win10, OSX)
My Project - https://github.com/tomboy-notes/tomboy-ng and my github - https://github.com/davidbannon

Pascal

  • Hero Member
  • *****
  • Posts: 932
Re: Recompile the RTL to debug
« Reply #1 on: December 11, 2017, 12:25:06 pm »
You can add OPT=-g to make in your fpc build script and build fpc again.

Here is my (windows) script for debug. The normal one is the same without OPT=-g.

Code: Text  [Select][+][-]
  1. @echo on
  2. set myroot=c:\freepascal
  3. set myFPC=%myroot%\fpc\trunk
  4. set mybinutils=%myroot%\binutils_3.0.4
  5. set PATH=%mybinutils%\i386-win32;%myFPC%\bin\i386-win32;%PATH%
  6. cd %myFPC%
  7. rd /s /q  %myfpc%\examples
  8. svn cleanup . --remove-unversioned  --remove-ignored
  9. make distclean all install OPT=-g INSTALL_PREFIX=%myFPC% PP=%mybinutils%\ppc386.exe DATA2INC=%myFPC%\utils\data2inc.exe
  10. make all OPT=-g OS_TARGET=win64 CPU_TARGET=x86_64 INSTALL_PREFIX=%myFPC% PP=%mybinutils%\ppc386.exe DATA2INC=%myFPC%\utils\data2inc.exe
  11. make crossinstall OPT=-g OS_TARGET=win64 CPU_TARGET=x86_64 INSTALL_PREFIX=%myFPC% PP=%mybinutils%\ppc386.exe DATA2INC=%myFPC%\utils\data2inc.exe
  12. cd /d %myFPC%\bin\i386-win32
  13. del fpc.cfg
  14. fpcmkcfg -d basepath=%myFPC% -o .\fpc.cfg
  15. copy fpc.exe %mybinutils%\i386-win32
laz trunk x64 - fpc trunk i386 (cross x64) - Windows 10 Pro x64 (21H2)

dbannon

  • Hero Member
  • *****
  • Posts: 2778
    • tomboy-ng, a rewrite of the classic Tomboy
Re: Recompile the RTL to debug
« Reply #2 on: December 11, 2017, 12:42:40 pm »
Thanks Pascal (you must have been in early to get that name!).

That looks like something I can work with, very useful. A touch of editing...

If I can get to where I need to, I think this could be usefully added to the wiki if you don't mind ?

David
Lazarus 2, Linux (and reluctantly Win10, OSX)
My Project - https://github.com/tomboy-notes/tomboy-ng and my github - https://github.com/davidbannon

molly

  • Hero Member
  • *****
  • Posts: 2330
Re: Recompile the RTL to debug
« Reply #3 on: December 11, 2017, 12:54:05 pm »
the problem with placing such things in the wiki is that these kind of scripts tend to be very specific to the setup of the user. For instance it wont work on a linux shell, simply because the script is written to work for windows.

I also see that the script is using trunk and svn, and that isn't necessary as you can download/install the sources separately. Script is also modifying path (which is a no go on certain setups). For cross-compiling the distclean does not always work as expected and afaik requires a 2 time call to make clean between builds, etc. etc.

The script is a good starting point though.  :)

ps: perhaps this thread might be of help as well. It has some additional points to note. Also have a look here.
« Last Edit: December 11, 2017, 01:45:11 pm by molly »

Pascal

  • Hero Member
  • *****
  • Posts: 932
Re: Recompile the RTL to debug
« Reply #4 on: December 11, 2017, 02:38:56 pm »
For cross-compiling the distclean does not always work as expected and afaik requires a 2 time call to make clean between builds, etc. etc.

That's why i added:
Code: Text  [Select][+][-]
  1. svn cleanup . --remove-unversioned  --remove-ignored


The script is based on wiki: http://wiki.freepascal.org/Installing_Lazarus#Compiling.2Finstalling_FPC_and_Lazarus_from_Sources_of_SVN_.28Win32.29
laz trunk x64 - fpc trunk i386 (cross x64) - Windows 10 Pro x64 (21H2)

molly

  • Hero Member
  • *****
  • Posts: 2330
Re: Recompile the RTL to debug
« Reply #5 on: December 11, 2017, 02:54:30 pm »
Ah, my bad Pascal. You are of course correct. I was not aware that the script you posted was part of a complete step-by-step instruction. In that case it makes a lot more sense  :)

dbannon

  • Hero Member
  • *****
  • Posts: 2778
    • tomboy-ng, a rewrite of the classic Tomboy
Re: Recompile the RTL to debug
« Reply #6 on: December 11, 2017, 10:15:08 pm »
I have not got started yet, have to head down to the "big smoke" today. I think the underlying commands, and even, heaven forbid, and explanation of what we are doing would be better than a "just copy, paste and run" script. That way, people can adapt it.

I'd assume things are where the install kits you read about on 'Announcements" put them.

It would help people like me who simply don't know where to start (and I am still far from sure). If Lazarus has a problem, its the lack of such information. To miss quote the recursion rule -

To get to know Lazarus, first you need to know Lazarus.

Good thing we have this brilliant forum and such great people populating it !

David
Lazarus 2, Linux (and reluctantly Win10, OSX)
My Project - https://github.com/tomboy-notes/tomboy-ng and my github - https://github.com/davidbannon

dbannon

  • Hero Member
  • *****
  • Posts: 2778
    • tomboy-ng, a rewrite of the classic Tomboy
Re: Recompile the RTL to debug
« Reply #7 on: December 12, 2017, 08:54:09 am »
Pascal, should that have been "Opt=-gl" ?  Did you leave off the 'l' ?

I note that in either case, it still produces 'RELEASE' code. I suspect thats a bad thing ?

Molly's suggested a link showing -

Code: [Select]
make clean all LINKSMART=1  CREATESMART=1  DEBUG=1 OPTIMIZE=0  OPT="-gl -gw -godwarfsets -O-1"
Apparently necessary to ensure a real debug build.   I'd love to see a list of what these and other options mean ....

Anyway, I found that 3.0.4 says its not capable of building itself, I had to install 3.0.2 and think I can build 3.0.4 with that. Right now, I tuck that away off to the side because if I let it override my install of 3.0.2, I cannot go back ?  I'm thinking I don't quite get it .....

Should 3.0.4 be able to build itself ?

Now, in my particular case, I don't actually need to rebuild the compiler (I think) only the RTL.  Pascal's script has three 'make' lines, the third obviously for cross compiling. So I assume the other two are for RTL and then Compiler ?   I don't see anything in the command lines that indicate which is what ? The first make line is the one that says it cannot build itself.

Please ?

David
Lazarus 2, Linux (and reluctantly Win10, OSX)
My Project - https://github.com/tomboy-notes/tomboy-ng and my github - https://github.com/davidbannon

molly

  • Hero Member
  • *****
  • Posts: 2330
Re: Recompile the RTL to debug
« Reply #8 on: December 12, 2017, 09:20:47 am »
Code: [Select]
make clean all LINKSMART=1  CREATESMART=1  DEBUG=1 OPTIMIZE=0  OPT="-gl -gw -godwarfsets -O-1"I'd love to see a list of what these and other options mean ....
make -> command that is named make
clean -> make option clean, simply makes things clean
all -> make option all

Going to link to some fpcmake topics (which i know is not entirely correct, but seems to have easiest explanation). For real make see here (original gnu document here and gnu make manual )
clean
all
For LINKSMART, CREATESMART, DEBUG, OPTIMIZE and OPT options see here.

Now what do the OPT parameters do ? see here

You also might want to have a look at the build-faq.

It is all there... you just don't seem to be able to find it (yet). Try harder  ;D


Anyway, I found that 3.0.4 says its not capable of building itself,
That is correct. The official path is to use the previous (latest) release to build your compiler.

Quote
I had to install 3.0.2 and think I can build 3.0.4 with that.
That is correct.

Quote
Right now, I tuck that away off to the side because if I let it override my install of 3.0.2, I cannot go back ?  I'm thinking I don't quite get it .....
You can have multiple version of FPC compiler installed. In case you are worried to override your 3.0.4 install then create that compiler by using make zipinstall , as that compresses the compiled files into an archive for you (and doesn't do an actual install). That way you can neatly tuck that archive somewhere safe to be used on a later date. Simply copy out the unit directory to your existing 3.0.4 installation to replace the non-debug build with your freshly compiled (with debug) units. Or put them alongside your existing installed units, f.e. into a director named unitsdbg (you would have to modify fpc.cfg in that case so that when you build in debug-mode the corret units directory is used.

Quote
Should 3.0.4 be able to build itself ?
No, although that would make life easier for a lot of people  :)

Quote
Now, in my particular case, I don't actually need to rebuild the compiler (I think) only the RTL.
That is correct. Building the compiler is just easier to accomplish (although it takes a bit more time).
« Last Edit: December 12, 2017, 10:23:20 am by molly »

Pascal

  • Hero Member
  • *****
  • Posts: 932
Re: Recompile the RTL to debug
« Reply #9 on: December 12, 2017, 09:28:43 am »
The shown script is to build a trunk version of fpc from an svn checkout but
it can also be adapted to fixes_3_0 (https://svn.freepascal.org/svn/fpc/branches/fixes_3_0/), which
can be build with 3.0.4 and 3.0.2.
If you have a look a the folder structure of my script you could also make a new subfolder
under fpc for fixes_3_0.
Maybe you should have a look at the mentioned wiki to get a more detailed view of how to
do you own setup.

The three make calls:
First make ist to build the default i386 compiler and all packages and libraries.
Second make is to build x86_64 packages an libraries
and the third make builds the crosscompiler for x86_64 (it's an i386 executable).
laz trunk x64 - fpc trunk i386 (cross x64) - Windows 10 Pro x64 (21H2)

molly

  • Hero Member
  • *****
  • Posts: 2330
Re: Recompile the RTL to debug
« Reply #10 on: December 12, 2017, 10:30:43 am »
Pascal's script has three 'make' lines, the third obviously for cross compiling. So I assume the other two are for RTL and then Compiler ?
Nope. As explained by Pascal.

Quote
I don't see anything in the command lines that indicate which is what ?
What do you mean by that ? The build-faq explains it all. All other stuff around it is just decoration to make things work for a specific setup/following a particular procedure.

Quote
The first make line is the one that says it cannot build itself.
You are required to build the 3.0.4 compiler with a 3.0.2 as starting compiler. Simple as that.

dbannon

  • Hero Member
  • *****
  • Posts: 2778
    • tomboy-ng, a rewrite of the classic Tomboy
Re: Recompile the RTL to debug
« Reply #11 on: December 12, 2017, 01:05:20 pm »
Yes Molly, thanks. I'm making some progress now.

I'm working on a VM on my Linux box (because its a lot more familiar) and with what I find, should be able to translate that to the Mac. Hopefully.

Here is what I have so far. I install the FPC3.0.2 and the 3.0.4 source. This builds the 3.0.4 compiler and RTL in a separate tree from normal. I make a matching fpc.cfg file and by putting that in place and calling the 'separate' binaries, I can build stuff with 3.0.4.  My next step will be to point Lazarus at that separate tree ...

 
Code: [Select]
#!/usr/bin/bash

FPCVERSION=3.0.4
DESTINATION=/usr-fake
SRC=/usr/share/fpcsrc/$FPCVERSION                       
COMPILER=/usr/lib/fpc/3.0.2/ppcx64 # not what we are building, we use to build $FPCVERSION
cd "$SRC"

make distclean all install LINKSMART=1 DEBUG=1 OPTIMIZE=1 OPT="-gl -gw" INSTALL_PREFIX=$DESTINATION PP=$COMPILER
# dont include CREATESMART=1 as it causes build to fail

# make distclean all install RELEASE=0 LINKSMART=1 CREATESMART=1 DEBUG=1 OPTIMIZE=0 OPT="-gl -gw -godwarfsets -O-1" INSTALL_PREFIX=$DESTINATION PP=$COMPILER

cd /etc
fpcmkcfg -d basepath="$DESTINATION/lib/fpc/$FPCVERSION" -o ./fpc.cfg-"$FPCVERSION"

One thing I have noticed so far, none of this gives me line numbers with a heaptrc report, thats a surprise, perhaps thats only available with Lazarus ?  Or am I still not getting enough debug code into my build .....

Thanks Pascal and Molly, appreciate your help.

Davo

Lazarus 2, Linux (and reluctantly Win10, OSX)
My Project - https://github.com/tomboy-notes/tomboy-ng and my github - https://github.com/davidbannon

Thaddy

  • Hero Member
  • *****
  • Posts: 14157
  • Probably until I exterminate Putin.
Re: Recompile the RTL to debug
« Reply #12 on: December 12, 2017, 01:22:00 pm »
Compiled with a debug version of fpc and lazarus and -glh, the -glh option will give you the line numbers......
Of course NOT for third party libraries written in other languages. Such libraries are probably compiled in release mode.

I have no real clue why that doesn't work for you where it works for everybody else?
Specialize a type, not a var.

molly

  • Hero Member
  • *****
  • Posts: 2330
Re: Recompile the RTL to debug
« Reply #13 on: December 12, 2017, 01:23:55 pm »
One thing I have noticed so far, none of this gives me line numbers with a heaptrc report, thats a surprise, perhaps thats only available with Lazarus ?  Or am I still not getting enough debug code into my build .....
Things can sometimes be very tricky  :)

First thing that comes to mind is your used fpc.cfg to compile the compiler with. Add -va to your OPT parameter and analyze the output. I'm guessing that debug information is stripped by default (option -Xs in fpc.cfg).

edit:
second thing that comes to mind, compiling the compiler went ok (e.g. debug was added) but then:
Code: [Select]
fpcmkcfg -d basepath="$DESTINATION/lib/fpc/$FPCVERSION" -o ./fpc.cfg-"$FPCVERSION"
Which afaik for sure adds -Xs as default option.
« Last Edit: December 12, 2017, 02:17:06 pm by molly »

dbannon

  • Hero Member
  • *****
  • Posts: 2778
    • tomboy-ng, a rewrite of the classic Tomboy
Re: Recompile the RTL to debug
« Reply #14 on: December 12, 2017, 11:46:14 pm »
Hmm, I'm going from bad to worse.  Molly, I ended up editing out the -Xs from fpc.cfg (as only way I could turn it off). No difference.

So I went back to my PC (as opposed to the VM I'm experimenting on). Here I have been developing a medium sized app and had lost of memory leaks, all easy found since Thaddy told me about -glh

Using just FPC, and new(Ptr); new(Ptr) is flagged as memory leak (good) but no line numbers (bad)!

So, still on real PC, a simple Lazarus form with the same new(Ptr); new(Ptr); under a button. "Display line numbers in run time errors" ON and "Use Heaptrc ON".  I get line numbers BUT they are wrong. Heaptrc is blaming LCL inc files, Its my memory leak, Line 37 of Project1.pas !

Maybe the overwriting of a longint pointer is too simple a tast for Heaptrc - all my past leaks were more complicated.

Sigh ...
Lazarus 2, Linux (and reluctantly Win10, OSX)
My Project - https://github.com/tomboy-notes/tomboy-ng and my github - https://github.com/davidbannon

 

TinyPortal © 2005-2018