Recent

Author Topic: Using C object files with Pascal with heaptrc  (Read 468 times)

LongTimePascaler

  • New Member
  • *
  • Posts: 20
Using C object files with Pascal with heaptrc
« on: July 01, 2022, 06:52:35 pm »
I have been using heaptrc to locate memory leaks as I develop a fairly large suite of software.  I now need to link to some software written in C (compiled with gcc); the C is fairly old, tightly written and riddled with dynamic memory allocation.  Normally I would just translate the C to Pascal to keep everything simple, but reading through this C code I decided that life was too short ...
For convenience I have written a unit as a wrapper, and this unit LINKLIBs to the 'c' and 'm' C libraries and LINKs to the C object file.  In test programs this works well without any use of CMem.
What is passed to the C code is a Pascal record that has a number of dynamically allocated arrays as fields, and a couple of similar records which the C code will use by dynamically allocating (malloc) memory as the fields' arrays and which are then passed back to the Pascal code to use.  The C links happily with the Pascal and for simple cases runs fine, but it doesn't take much complexity before the whole thing collapses at random points with memory errors. For example:
Code: Text  [Select][+][-]
  1. Marked memory at $00007FD0F03BFF30 invalid
  2. Wrong signature $1F4AE121 instead of 5998D445
  3.  
I assume (given the warning on the Wiki page https://wiki.freepascal.org/heaptrc) that there is a clash between heaptrc (as memory manager) and the C memory manager used by the C code.  I also assume that heaptrc is incompatible with the unit CMem.
Are my assumptions correct?  Is there any way for heaptrc and C code to cohabit, or are they completely incompatible so that it is either use heaptrc or link to C code (but not both)?

MarkMLl

  • Hero Member
  • *****
  • Posts: 4753
Re: Using C object files with Pascal with heaptrc
« Reply #1 on: July 01, 2022, 07:11:51 pm »
I assume (given the warning on the Wiki page https://wiki.freepascal.org/heaptrc) that there is a clash between heaptrc (as memory manager) and the C memory manager used by the C code.  I also assume that heaptrc is incompatible with the unit CMem.
Are my assumptions correct?  Is there any way for heaptrc and C code to cohabit, or are they completely incompatible so that it is either use heaptrc or link to C code (but not both)?

I'm working from memory here since I'm without my principal development system (destructive mains glitch). I think there's ways... see https://forum.lazarus.freepascal.org/index.php?topic=46121.0 as an example (via Google, forum search might turn up others).

MarkMLl
MT+86 & Turbo Pascal v1 on CCP/M-86, multitasking with LAN & graphics in 128Kb.
Pet hate: people who boast about the size and sophistication of their computer.
GitHub repositories: https://github.com/MarkMLl?tab=repositories

marcov

  • Administrator
  • Hero Member
  • *
  • Posts: 10175
  • FPC developer.
Re: Using C object files with Pascal with heaptrc
« Reply #2 on: July 01, 2022, 07:24:06 pm »
I think so. CMem doesn't replace the C memory allocation routines, so all allocations made by C (and possibly also FPC allocations deallocated by C) are not tracked.

You might want to look at valgrind.

PascalDragon

  • Hero Member
  • *****
  • Posts: 4303
  • Compiler Developer
Re: Using C object files with Pascal with heaptrc
« Reply #3 on: July 02, 2022, 04:40:53 pm »
What is passed to the C code is a Pascal record that has a number of dynamically allocated arrays as fields, and a couple of similar records which the C code will use by dynamically allocating (malloc) memory as the fields' arrays and which are then passed back to the Pascal code to use.

In addition to what marcov wrote: I hope you're not using Pascal dynamic arrays here.

LongTimePascaler

  • New Member
  • *
  • Posts: 20
Re: Using C object files with Pascal with heaptrc
« Reply #4 on: July 04, 2022, 10:00:55 am »
@MarkMLl My searching didn't come up with that reference; perhaps my search words weren't well enough chosen. I will study that thread.

@marcov I have used valgrind in the past to isolate bottlenecks, but not for finding memory leaks.  I will have to investigate that.

@PascalDragon Yes, The Pascal record mimics the C struct which uses dynamically allocated arrays.  The C code sometimes traverses its arrays using indices and sometimes using pointer arithmetic.  None of the arrays allocated in the Pascal code is freed by the C code, they are just read/traversed and that seems to work. 
The structs/records returned (through the parameters) by the C code do have array fields that are dynamically allocated by the C code.  These are treated as dynamically allocated arrays by the Pascal code.  The Pascal code takes the data out of those arrays and puts them into the Pascal program's data structures.  Then another C function is called to free those C-allocated dynamic arrays; that function simply uses C's free function to dealloacate those arrays.  If I do this just once this works, but if the Pascal code happens to do it more than once the program crashes with the kind of memory error given above.  Probably (I haven't checked) if I didn't try to free that memory the software wouldn't crash, but then I would have a persistent memory leak.
There are some other complications involved in getting anything to work, such as setting up the arguments to the principal C function (the one that does the work) in the Pascal calling code, but since I have found a way to make that work so that the C stuff runs and returns sensible values I am concentrating on the memory management issue here.

I have other things I must do today but I'll return to this tomorrow and follow up what @MarkMLl and @marcov have suggested.  @PascalDragon if there is an alternative to using dynamically allocated arrays then please advise.

440bx

  • Hero Member
  • *****
  • Posts: 2933
Re: Using C object files with Pascal with heaptrc
« Reply #5 on: July 04, 2022, 10:29:41 am »
What is passed to the C code is a Pascal record that has a number of dynamically allocated arrays as fields, and a couple of similar records which the C code will use by dynamically allocating (malloc) memory as the fields' arrays and which are then passed back to the Pascal code to use.
Mixing different compiler's memory managers is always problematic.  As the saying goes: too many cooks in the memory management kitchen.

If I were you, I'd change both, the Pascal code and the C code to directly use the O/S's memory management services.  This way, there is only _one_ memory manager involved and that makes life a lot easier.

You didn't mention what O/S you're targeting but, if it's Windows only, Windows offers a good number of very convenient methods to solve that problem.

HTH.
FPC v3.0.4 and Lazarus 1.8.2 on Windows 7 64bit.

MarkMLl

  • Hero Member
  • *****
  • Posts: 4753
Re: Using C object files with Pascal with heaptrc
« Reply #6 on: July 04, 2022, 10:50:05 am »
If I were you, I'd change both, the Pascal code and the C code to directly use the O/S's memory management services.  This way, there is only _one_ memory manager involved and that makes life a lot easier.

I'm not sure that is viable if e.g. Pascal strings (i.e. on the heap, as distinct from short strings) are being used.

MarkMLl
MT+86 & Turbo Pascal v1 on CCP/M-86, multitasking with LAN & graphics in 128Kb.
Pet hate: people who boast about the size and sophistication of their computer.
GitHub repositories: https://github.com/MarkMLl?tab=repositories

440bx

  • Hero Member
  • *****
  • Posts: 2933
Re: Using C object files with Pascal with heaptrc
« Reply #7 on: July 04, 2022, 11:25:00 am »
I'm not sure that is viable if e.g. Pascal strings (i.e. on the heap, as distinct from short strings) are being used.
Only the OP knows if that's a viable alternative.  Depends on the code.  It's a possibility/option he may want to look into.
FPC v3.0.4 and Lazarus 1.8.2 on Windows 7 64bit.

PascalDragon

  • Hero Member
  • *****
  • Posts: 4303
  • Compiler Developer
Re: Using C object files with Pascal with heaptrc
« Reply #8 on: July 04, 2022, 01:25:48 pm »
@PascalDragon Yes, The Pascal record mimics the C struct which uses dynamically allocated arrays.  The C code sometimes traverses its arrays using indices and sometimes using pointer arithmetic.  None of the arrays allocated in the Pascal code is freed by the C code, they are just read/traversed and that seems to work. 
The structs/records returned (through the parameters) by the C code do have array fields that are dynamically allocated by the C code.  These are treated as dynamically allocated arrays by the Pascal code.  The Pascal code takes the data out of those arrays and puts them into the Pascal program's data structures.  Then another C function is called to free those C-allocated dynamic arrays; that function simply uses C's free function to dealloacate those arrays.  If I do this just once this works, but if the Pascal code happens to do it more than once the program crashes with the kind of memory error given above.  Probably (I haven't checked) if I didn't try to free that memory the software wouldn't crash, but then I would have a persistent memory leak.
There are some other complications involved in getting anything to work, such as setting up the arguments to the principal C function (the one that does the work) in the Pascal calling code, but since I have found a way to make that work so that the C stuff runs and returns sensible values I am concentrating on the memory management issue here.

I have other things I must do today but I'll return to this tomorrow and follow up what @MarkMLl and @marcov have suggested.  @PascalDragon if there is an alternative to using dynamically allocated arrays then please advise.

Because your message doesn't clearly state this: do you or do you not use array of XXX fields in the record/struct that is shared between the C and the Pascal code? If you do so: DON'T. A Pascal dynamic array is a much different beast than what passes as an array in C. If you want to have your C and Pascal code communicate you need to use the lowest common denominator, which in this case is C, and thus you need to use pointer variables and manually allocate/free them like you do in C to represent arrays.

 

TinyPortal © 2005-2018