Recent

Author Topic: Recompile the RTL to debug  (Read 27356 times)

dbannon

  • Hero Member
  • *****
  • Posts: 2786
    • tomboy-ng, a rewrite of the classic Tomboy
Re: Recompile the RTL to debug
« Reply #15 on: December 13, 2017, 02:33:07 am »
Just to clarify Thaddy
Code: Pascal  [Select][+][-]
  1. program test;
  2. {$mode objfpc}
  3. var
  4.         Ptr : ^longint;
  5. begin
  6.         new(Ptr);
  7.         new(Ptr);
  8.         writeln('hello world');
  9. end.
  10.  

Compiles like this -
Code: [Select]
fpc -ghl test.pas
Gives me this -
Code: [Select]
dbannon@Aspire:~/Desktop/VirtualBoxShared$ ./test
hello world
Heap dump by heaptrc unit
2 memory blocks allocated : 8/16
0 memory blocks freed     : 0/0
2 unfreed memory blocks : 8
True heap size : 32768
True free heap : 32448
Should be : 32496
Call trace for block $00007F45173EF160 size 4
Call trace for block $00007F45173EF0C0 size 4

This on my "un-fiddled" Linux box, using FPC 3.0.3 as bundled as a Deb in 1.8RC3.  With -Xs manually commented out of fpc.cfg. Now, Lazarus gives me a stack of line numbers, wrong but at least some line numbers. It adds something to heaptrc that we don't get using -glh and fpc

I see the same on my VM with 3.0.4 but I have fiddled extensivly with that so it can be excused ...
Davo
Lazarus 3, Linux (and reluctantly Win10/11, OSX Monterey)
My Project - https://github.com/tomboy-notes/tomboy-ng and my github - https://github.com/davidbannon

dbannon

  • Hero Member
  • *****
  • Posts: 2786
    • tomboy-ng, a rewrite of the classic Tomboy
Re: Recompile the RTL to debug
« Reply #16 on: December 13, 2017, 07:21:48 am »
OK,  so easy to go off chasing shadows and forget where you are really heading ...

Right now, I'm learning how to rebuild  compiler/rtl to include debugging so I can find a memory leak on a Mac. Remember that David.

 The immediate issue is, even with a newly build compiler/rtl with [DEBUG, -glh] I don't see references to offending lines of code. Sometimes.

Its not a compiler issue, I suspect HeapTrc does not bother to tell you a line number if it thinks you are doing it deliberately. It feels insulted !     :)

For example, I declare a pointer to a record, new(Ptr), create a list of such records and don't Free anything, I get told I have a memory leak but not where.

But if I add that record to the list, I get a sensible line number !

This behaviour is not, in any way, dependent on what compiler/RTL I use !  Thats the shadow I've been chasing.

As my memory leak is a sincere and probably subtle one, when I move back to the Mac (yek) I'll probably get a fully cooperative HeapTrc.  :-\

Anyone want to see the  61 line FPC code to demonstrate what I just said, please ask !

David
Lazarus 3, Linux (and reluctantly Win10/11, OSX Monterey)
My Project - https://github.com/tomboy-notes/tomboy-ng and my github - https://github.com/davidbannon

dbannon

  • Hero Member
  • *****
  • Posts: 2786
    • tomboy-ng, a rewrite of the classic Tomboy
Re: Recompile the RTL to debug
« Reply #17 on: December 14, 2017, 12:08:10 pm »
OK, just to help someone who reads this thread.

On Linux at least, the missing line numbers I (don't) see are not a Heaptrc  personality issue. Heaptrc cannot report on leaks that occur in the main function (sorry, old C programmer) or whatever Pascal folk call the final begin...end. Yep, my simple test was too simple !

Valgrind also does not report the line numbers (from main() ) but does refer to main_stub which started me thinking. It probably makes sense, these tools may use function calls as intercept points ....

I'll head back to my Mac and see if the same rules apply there, I suspect they don't and this whole exercise has been non productive but certainly enlightening.

David
Lazarus 3, Linux (and reluctantly Win10/11, OSX Monterey)
My Project - https://github.com/tomboy-notes/tomboy-ng and my github - https://github.com/davidbannon

Thaddy

  • Hero Member
  • *****
  • Posts: 14197
  • Probably until I exterminate Putin.
Re: Recompile the RTL to debug
« Reply #18 on: December 14, 2017, 12:43:35 pm »
compile fpc with make option OPT='-dDEBUG -glh -O-' no need to exclude Xs because it is not in the default debug section in fpc.cfg

It will give you the proper and correct line number information.
« Last Edit: December 14, 2017, 12:46:53 pm by Thaddy »
Specialize a type, not a var.

molly

  • Hero Member
  • *****
  • Posts: 2330
Re: Recompile the RTL to debug
« Reply #19 on: December 15, 2017, 02:41:17 am »
... no need to exclude Xs because it is not in the default debug section in fpc.cfg
Although i'm note sure about fpc.cfg during build-process, for sure it is default for any newly created fpc.cfg

Skipping boring parts
Code: [Select]
#
# Config file generated by fpcmkcfg on 15-12-2017 - 2:27:55
# Example fpc.cfg for Free Pascal Compiler
#

# ----------------------
# Defines (preprocessor)
# ----------------------

#
# nested #IFNDEF, #IFDEF, #ENDIF, #ELSE, #DEFINE, #UNDEF are allowed
#
# -d is the same as #DEFINE
# -u is the same as #UNDEF
#

#
# Some examples (for switches see below, and the -? helppages)
#
# Try compiling with the -dRELEASE or -dDEBUG on the commandline
#

# For a release compile with optimizes and strip debuginfo
#IFDEF RELEASE
  -O2
  -Xs
  #WRITE Compiling Release Version
#ENDIF

# For a debug version compile with debuginfo and all codegeneration checks on
#IFDEF DEBUG
  -gl
  -Crtoi
  #WRITE Compiling Debug Version
#ENDIF
ok, so debug says no stripping... then we move on to line 244 of default fpc.cfg
Code: [Select]
# Always strip debuginfo from the executable
-Xs

# Always use smartlinking on i8086, because the system unit exceeds the 64kb
# code limit
#ifdef cpui8086
-CX
-XX
#endif

# -------------
# Miscellaneous
# -------------

So, that means stripping is on by default, also when debug is defined. At least for fpc. No idea about what Lazarus does behind the scenes.

oversight perhaps ?

Thaddy

  • Hero Member
  • *****
  • Posts: 14197
  • Probably until I exterminate Putin.
Re: Recompile the RTL to debug
« Reply #20 on: December 15, 2017, 06:50:23 am »
molly that seems a bug in the default config generated by fpcmkcfg Debug mode should of course NEVER strip anything.
I think this is a regression for the default. I filed a bug report (0032831) in Mantis.
I also attached a patch that solves the issue.
« Last Edit: December 15, 2017, 07:22:30 am by Thaddy »
Specialize a type, not a var.

dbannon

  • Hero Member
  • *****
  • Posts: 2786
    • tomboy-ng, a rewrite of the classic Tomboy
Re: Recompile the RTL to debug
« Reply #21 on: December 15, 2017, 08:27:24 am »
OK, on the mac, I do see -Xs on the command lines.

With all the switches mentioned so far.  I'll have a look at Thaddy's patch and see if I can manually apply it.

eg
Code: [Select]
/usr/local/lib/fpc/3.0.2/ppcx64 -dNOMOUSE -Ur -dFPC_USE_LIBC -XX -gl -Ur -Xs -O2 -n -O2 -Fi../inc -Fi../i386 -Fi../unix -Fi../bsd -Fi../bsd/i386 -Fi../darwin/i386 -FE. -FU/usr/local/share/fpcsrc/3.0.4/rtl/units/i386-darwin -glh -O- -gw -di386 -dDEBUG -dRELEASE -Us -Sg ../bsd/system.pp
David
Lazarus 3, Linux (and reluctantly Win10/11, OSX Monterey)
My Project - https://github.com/tomboy-notes/tomboy-ng and my github - https://github.com/davidbannon

dbannon

  • Hero Member
  • *****
  • Posts: 2786
    • tomboy-ng, a rewrite of the classic Tomboy
Re: Recompile the RTL to debug
« Reply #22 on: December 15, 2017, 09:11:11 am »
But in fact, its not building on the Mac anyway.  I'm asking for i386 of course because its cocoa but the RTL/system.inc seems to have x86_64 defined and includes x86_64.inc.

I have no idea where that 64 bit define is coming from. My script looks like this -
Code: [Select]
#!/usr/bin/bash

FPCVERSION=3.0.4
OLDFPCVERSION=3.0.2
DESTINATION=/fpc-usr/3.0.4debug
TARGET="OS_TARGET=darwin CPU_TARGET=i386"
SRC=/usr/local/share/fpcsrc/$FPCVERSION                       
COMPILER=/usr/local/lib/fpc/$OLDFPCVERSION/ppcx64 # not what we are building, we use to build $FPCVERSION

mkdir $DESTINATION
cd "$SRC"

make distclean all install LINKSMART=1 DEBUG=1 OPTIMIZE=1 OPT="-glh -O- -gw" $TARGET INSTALL_PREFIX=$DESTINATION PP=$COMPILER
# dont include CREATESMART=1 as it causes build to fail

mkdir "$DESTINATION/lib/fpc/etc"
cd "$DESTINATION/lib/fpc/etc"
$DESTINATION/bin/fpcmkcfg -d basepath="$DESTINATION/lib/fpc/$FPCVERSION" -o ./fpc.cfg
# see https://www.freepascal.org/docs-html/user/usersu10.html

#cd /etc
# cp ./fpc.cfg ./fpc.cfg-$OLDFPCVERSION
# fpcmkcfg -d basepath="$DESTINATION/lib/fpc/$FPCVERSION" -o ./fpc.cfg-"$FPCVERSION"

cd $DESTINATION/bin
ln -s ../lib/fpc/$FPCVERSION/ppcx64
echo "OK, FPC files are in $DESTINATION along with their own fpc.cfg"

I see this as soon as Make has finished cleaning up -

Code: [Select]
...
/bin/mkdir -p /usr/local/share/fpcsrc/3.0.4/rtl/units/i386-darwin
/usr/local/lib/fpc/3.0.2/ppcx64 -dNOMOUSE -Ur -dFPC_USE_LIBC -XX -gl -Ur -Xs -O2 -n -O2 -Fi../inc -Fi../i386 -Fi../unix -Fi../bsd -Fi../bsd/i386 -Fi../darwin/i386 -FE. -FU/usr/local/share/fpcsrc/3.0.4/rtl/units/i386-darwin -glh -O- -gw -di386 -dDEBUG -dRELEASE -Us -Sg ../bsd/system.pp
system.inc(181,4) Fatal: Cannot open include file "x86_64.inc"
Fatal: Compilation aborted
Lazarus 3, Linux (and reluctantly Win10/11, OSX Monterey)
My Project - https://github.com/tomboy-notes/tomboy-ng and my github - https://github.com/davidbannon

Lutz Mändle

  • Jr. Member
  • **
  • Posts: 65
Re: Recompile the RTL to debug
« Reply #23 on: December 15, 2017, 09:37:44 am »
Try to export the variables OS_TARGET and CPU_TARGET, define them separately, like this:

...

OS_TARGET=darwin
CPU_TARGET=i386
export OS_TARGET CPU_TARGET

...




Thaddy

  • Hero Member
  • *****
  • Posts: 14197
  • Probably until I exterminate Putin.
Re: Recompile the RTL to debug
« Reply #24 on: December 15, 2017, 09:51:15 am »
Please note I forgot to run data2inc on my clean room, so I had to provide a third patch.
That one is the one you need. It is rather big, but that's because that wasn't run for a long time, so all kind of formatting issues are solved at the same time.

So you will need  fpcmkcfg3.diff , ignore the others. Not running data2inc wasn't an obvious oversight to me, but now it is correct.

Now, when you specify the make option OPT='-dDEBUG -glh'' you can debug with the correct line info with a default fpc.cfg too.

Nasty bug. No wonder it leads to so much confusion. Thanks to molly for spotting it.
« Last Edit: December 15, 2017, 09:54:45 am by Thaddy »
Specialize a type, not a var.

molly

  • Hero Member
  • *****
  • Posts: 2330
Re: Recompile the RTL to debug
« Reply #25 on: December 15, 2017, 10:10:24 am »
@Thaddy:
Thank you for the bug-report and patch(es)  8-)

@dbannon:
I see you are still attempting to make things difficult for yourself. Ok, then. I'll cook something that is (hopefully) easier to grasp. No need for an older compiler. Still need to run make though  ;D Just gimme a mom.

molly

  • Hero Member
  • *****
  • Posts: 2330
Re: Recompile the RTL to debug
« Reply #26 on: December 15, 2017, 10:49:59 am »
Creating and setting up debug units for fpc 3.0.4 compiler.

( This is not (officially) supported. Might break at any time at any point, hurt your setup or start ww3 )
Code: [Select]
Notes:
a) applies for fpc 3.0.4
b) Substitute X: with whatever is your working location.
c) assuming you have all required binutils in place. You have them when you have a(ny) working FPC installation. The only thing that isn't covered 100% here is cross-compiling (but then you know which additional binutils to install).

1) Make sure your setup is clean. No nasty (hidden) configuration stored around.
2) download bootstrap compiler for your setup [ftp://ftp.freepascal.org/pub/fpc/dist/3.0.4/bootstrap/]. In case it isn't listed then read further.
3) download fpc 3.0.4 sources [ftp://ftp.freepascal.org/pub/fpc/dist/3.0.4/source/fpc-3.0.4.source.zip]
4) Extract (or copy from your existing fpc 3.0.4 installation) the bootstrap compiler to X: [in my case ppc386.exe]
5) Extract from fpc 3.0.4 zip archive the directories [rtl] and [packages] and files from root directory of the zip-file into directory X:fpcsrc
6) create a new batch file in X: [build.bat] with contents from script below (or rewrite for working with bash, in which case name is [build.sh]).
7) run batch-file
8) drink some coffee
9) done.... almost. See next topic.


Batch for windows (so you'll need to turn that it into something that bash understands when linux):
Code: [Select]
@ECHO OFF
@ECHO Building started ....
@SET CURRENT_DIR=%CD%
@SET SOURCE_DIR=%CURRENT_DIR%\fpcsrc
@SET START_COMPILER=%CURRENT_DIR%\ppc386.exe
@SET INSTALL_DIR=%CURRENT_DIR%\fpcbin
@SET LOG_FILE=%CURRENT_DIR%\logfile.txt

@CD %SOURCE_DIR%
@ECHO Start 1> %LOG_FILE% 2>&1

@REM  Compile RTL with make
@ECHO Building RTL
@ECHO Building RTL 1>> %LOG_FILE% 2>&1
MAKE -C rtl install INSTALL_PREFIX=%INSTALL_DIR% CPU_TARGET=i386 OS_TARGET=win32 PP=%START_COMPILER% OPT="-dDEBUG -glh -O-" 1>> %LOG_FILE% 2>&1

@REM  Compile packages with make
@ECHO Building Packages
@ECHO Building Packages 1>> %LOG_FILE% 2>&1
MAKE -C packages install INSTALL_PREFIX=%INSTALL_DIR% CPU_TARGET=i386 OS_TARGET=win32 PP=%START_COMPILER% OPT="-dDEBUG -glh -O-" 1>> %LOG_FILE% 2>&1

@ECHO ........ Done !
Make sure you change the "CPU_TARGET=i386" and "OS_TARGET=win32" parts to match your host/target.

OPT options based on Thaddy's post. (nice output in logfile also  :) )



Installing debug units for FPC 3.0.4 correctly

After the build has finished correctly then you have a new directory named fpcbin in x: with some stuff inside it (if not correctly, then view log for errors)
Code: [Select]
1) we are only interested in the directory named "units".
2) copy this directory to your existing fpc 3.0.4 installation. But, we want to preserve original units so we place these newly created units in a directory named units_dbg
3) edit your fpc.cfg
4) make sure there is a #IFDEF DEBUG section exactly as is in default fpc.cfg
5) make sure nothing else is interfering with this "debug' Mode. Spot and remove the line where it says: "-Xs" (or comment this line) and/or any other optimization options.

Now, inside your fpc.cfg you should be able to read one or several lines where it says something like:
-FuC:\FPC\$FPCVERSION\units\$FPCTARGET\*

Point is, we need to make an exception for debug build to point to the correct units_dbg directory

For this test case that looks something like:
#IFDEF DEBUG
-FuC:\FPC\$FPCVERSION\units_dbg\$FPCTARGET\*
#ELSE
-FuD:\FPC\$FPCVERSION\units\$FPCTARGET\*
#ENDIF

Make sure this works for every unit path that you've added to your fpc.cfg file.

Now, _build_ your test code with:
Code: [Select]
fpc -B -glh -dDEBUG testcode.pas
And run it. You should now have a back trace with line numbers in case heaptrace has anything sensible to report. Results might not always be satisfactory because heaptrace is restricted by fpc RTL and/or users stupidity  ;). Debugging OS library calls is something else  :)

Note that a back trace is based on X:, so might work confusing if you did not expected that.

dbannon

  • Hero Member
  • *****
  • Posts: 2786
    • tomboy-ng, a rewrite of the classic Tomboy
Re: Recompile the RTL to debug
« Reply #27 on: December 15, 2017, 12:20:41 pm »
Wow Molly, thats just about a publication ! I am seriously grateful !

However, I'm being hard to get along with ! 

Firstly, I'm on a Mac now, thats where the problem exists, I was using Linux while feeling my way around but have to make it work on the Mac. Your ftp site does mention freeBSD bootstrap compilers, honestly, I know so little about Mac that I don't know....

Secondly, I now have both the 3.0.4 and the 3.0.2 compilers running on my Mac. So, much of its done. I have them tucked out of the way so I can reconfigure or butcher without causing any real problems. I hope :-)

The 32/64 bit issue is all about a stupid typo in my scripts, compile is running now, as we speak. But I'm still getting a -Xs after I commented out both -Xs in the 'in use' fpc.cfg.  Looks like its coming in from the Makefile, going to have a read.....
Lazarus 3, Linux (and reluctantly Win10/11, OSX Monterey)
My Project - https://github.com/tomboy-notes/tomboy-ng and my github - https://github.com/davidbannon

dbannon

  • Hero Member
  • *****
  • Posts: 2786
    • tomboy-ng, a rewrite of the classic Tomboy
Re: Recompile the RTL to debug
« Reply #28 on: December 15, 2017, 12:33:59 pm »
Nope !  Removed the -Xs from two places in Makefile but build still includes the infamous -Xs

I note -dRELEASE too, got to be related.

Sigh.

Too late, going to bed.

Davo

Lazarus 3, Linux (and reluctantly Win10/11, OSX Monterey)
My Project - https://github.com/tomboy-notes/tomboy-ng and my github - https://github.com/davidbannon

molly

  • Hero Member
  • *****
  • Posts: 2330
Re: Recompile the RTL to debug
« Reply #29 on: December 15, 2017, 12:54:34 pm »
If you specifically use the 3.0.4 bootstrap compiler (ppc386.exe for windows 32-bit) running in it's own location then you should be good to go.

Meaning, that in case you receive such messages about using -Xs that there is probably something else bugging you. You should _not_ modify the makefiles. They should be good to go, as they are clean. With every (failed) make attempt, you should run "make clean" twice (or better yet in order to make sure, delete the lot and extract again).

For every other error(s) that might pop-up: there is a log-file so use it wisely :). Additional (verbose) options are allowed to produce even more extensive information.

Do note however, that i have no experience running the same script on a mac. Your mileage may vary... but in case of errors, they should be clear reported errors.

In case the same error is also happening for you when you use the 3.0.2 compiler in order to create a complete 3.0.4 compiler with debug/linenumbers added to the build files then that would be a clear indication something is crossing your setup. If not, then there is something seriously wrong with mac version: the 3.0.2 compiler _must_ be able to compile the 3.0.4 compiler, no matter what (exotic host/targets perhaps excluded). If not, then that is to be considered a bug.

A good nights rest, sometimes does wonders though  :D

PS: I just had a thought. Are you perhaps able to post a dump of your shell environment variables ? There are a few that overrule/add to your compiler settings. Those should be cleared. For me personally i once had (if i remember correctly) env compiler_options set, which wasted quite an extensive amount time. No option added and still those options where used for compilation. A bit like a fata-morgana in a cardboard box :)
« Last Edit: December 15, 2017, 01:05:17 pm by molly »

 

TinyPortal © 2005-2018