Recent

Author Topic: Windows 7 x64 and possible code limit causing error 203 ?  (Read 1329 times)

Malkomax

  • New member
  • *
  • Posts: 7
Windows 7 x64 and possible code limit causing error 203 ?
« on: November 25, 2020, 07:40:57 am »
Hello,

I've started to run into a heap 203 error when running my project on Windows 7 x64, and it seems I've hit some code limit on Free Pascal(Lazarus) project code size.

The project will successfully build with a Clean Build but when running with F9 option (debugging ON) it gives error 203.
It will run with debugging off.

The Pascal commands aren't related to arrays, they are ANY lines of code:
   - simple assignments,
   - IF THEN
   - CASE
   - etc

The problem isn't just related to one module, but adding code to another module also causes the 203 error.
If I remove the newly added lines then the project runs again.
If I comment out working lines of code from somewhere else and then add the new lines, the project will then run again with no 203 error.


My project is a conversion of HyperNext, a GUI compiler app builder, from REALbasic to Free Pascal.
Currently my FP project has over 34k lines of code, with lots of arrays for compiler tables etc, but none of the arrays are very large.
When fully finished there will be over 100k lines of code.


The same project runs fine on Ubuntu 18.04 and Raspberry Pi Buster, and both allow more lines of code to be added.

I'm using FPC v3.2 and Lazarus 2.0.10 on Window 7 Ultimate, with 16GB RAM and i5-2500. 


Is there some way to increase the FPC heap size or some limit flag on Windows FPC?

Thank you in advance,
Malkom

Thaddy

  • Hero Member
  • *****
  • Posts: 10705
Re: Windows 7 x64 and possible code limit causing error 203 ?
« Reply #1 on: November 25, 2020, 08:13:39 am »
I suspect programmer error, but anyway:
 -Ch<n>[,m] <n> bytes min heap size (between 1023 and 67107840) and optionally [m] max heap size
Usually and as standard the max heap size is simply available resources. Hence I think your code is not sound.
Do you properly release memory after use? AFAIK RealBasic has automatic memory management (and therefor sometimes slow), but in FPC you have to release the memory yourself.
Basically, when you try to port from FreeBasic to FreePascal (or C/C++ and others) you have to be aware of this: you need to control memory release yourself.
FreePascal should use less heap compared to RealBasic. Not more. The reason it works on some nixes is simply the behavior of swap files extending the heap. Windows supports that too, but not for heap without intervention. Anyway, that would be hiding coding errors.
« Last Edit: November 25, 2020, 08:44:14 am by Thaddy »

Malkomax

  • New member
  • *
  • Posts: 7
Re: Windows 7 x64 and possible code limit causing error 203 ?
« Reply #2 on: November 25, 2020, 10:12:17 am »
I suspect programmer error, but anyway:
 -Ch<n>[,m] <n> bytes min heap size (between 1023 and 67107840) and optionally [m] max heap size
Usually and as standard the max heap size is simply available resources. Hence I think your code is not sound.
Do you properly release memory after use? AFAIK RealBasic has automatic memory management (and therefor sometimes slow), but in FPC you have to release the memory yourself.
Basically, when you try to port from FreeBasic to FreePascal (or C/C++ and others) you have to be aware of this: you need to control memory release yourself.
FreePascal should use less heap compared to RealBasic. Not more. The reason it works on some nixes is simply the behavior of swap files extending the heap. Windows supports that too, but not for heap without intervention. Anyway, that would be hiding coding errors.

Thank you for such a prompt reply and details of the heap etc.

I've spent 2 weeks on this because in my experience errors are almost ALWAYS caused by me and rarely by the development system itself.
Hopefully its the case here and I've overlooked something, it might help others.

My problem - the program never even reaches its splash screen before the GUI should then appear.
Once the GUI appears then the user can compile their project - this is when massive use of the heap would occur.
The way it works, the arrays are defined but left unpopulated, but once compilation/building of a user's project is underway, then the arrays massively use the heap as objects are created etc.
However, it never reaches this stage, by failing before the splash screen even loads.

Note, about adding code
For instance, if I add the code line "x=2" to my FPC project, the 203 error occurs, if I then delete the "x=2" line, the 203 error does not occur.

By the way, with the "x=2" removed, no 203 error. The project runs normally and can then can compile a users project, which implies that the FPC code and heap memory are separate.
Unfortunately I don't yet understand how FPC uses memory.

Ah - just occurred to me - it could be the FPC Debugger - it might need more space allocating for code symbols etc - i'll look into that.
 
This might be relevant for me: - https://wiki.freepascal.org/Size_Matters
« Last Edit: November 25, 2020, 10:18:12 am by Malkomax »

PascalDragon

  • Hero Member
  • *****
  • Posts: 2758
  • Compiler Developer
Re: Windows 7 x64 and possible code limit causing error 203 ?
« Reply #3 on: November 25, 2020, 10:38:40 pm »
-Ch<n>[,m] <n> bytes min heap size (between 1023 and 67107840) and optionally [m] max heap size
Usually and as standard the max heap size is simply available resources. Hence I think your code is not sound.

The -Ch option is only used for targets using the OMF format (namely the i8086 targets).

Ah - just occurred to me - it could be the FPC Debugger - it might need more space allocating for code symbols etc - i'll look into that.

Can you try to run it without debugger?

Also can you check what Windows task manager reports for the memory? Are you maybe trying to do some really large allocations?

Do you indeed compile for x86_64-win64 and not i386-win32?

marcov

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 9089
  • FPC developer.
Re: Windows 7 x64 and possible code limit causing error 203 ?
« Reply #4 on: November 25, 2020, 11:08:48 pm »
First, lazarus and the compiler itself are hundred thousands of lines.

My largest private apps are in the single hundred thousand lines (100-200kloc) ranges, and I never encountered such error.

heap exhaustion comes from some typical cases:
  • some loop or (unintended?) recursion goes awry, and a certain allocation is done much more often than you think
  • You have a few very large allocations (e.g. concatting strings or working with tstringlist ) and somehow you exhaust or fragment the 2-4 GB 32-bit address space, so that some large block can't be allocated. Think hundreds of megabytes here
  • The calculation of the size of a block goes wrong, and you try to allocate a humongous block like ptruint(-1)

If that is not the case, it might be some pointer mistake that corrupts the heap and gives bad errors. Try to turn on runtime checks (including -gtt and -CRriot) and try to work through the issues that popup and see if that makes you any wiser.

Random putting halts in the startup process and using the heap dump ( -gh) to check for unexpected amounts of allocations can also help.

If all fails, trying to run under valgrind on Linux can be illuminating, because it has a better chance of reacting already to the origin of the problem, rather than the consequences.

If running the whole application under Linux is a bridge too far, if you have a good gui-logic separation, running unit tests and other partial bits might help to get them validated.

Malkomax

  • New member
  • *
  • Posts: 7
Re: Windows 7 x64 and possible code limit causing error 203 ?
« Reply #5 on: November 25, 2020, 11:30:25 pm »
-Ch<n>[,m] <n> bytes min heap size (between 1023 and 67107840) and optionally [m] max heap size
Usually and as standard the max heap size is simply available resources. Hence I think your code is not sound.

The -Ch option is only used for targets using the OMF format (namely the i8086 targets).


Can you try to run it without debugger?

Also can you check what Windows task manager reports for the memory? Are you maybe trying to do some really large allocations?

Do you indeed compile for x86_64-win64 and not i386-win32?

Yes it runs fine without the debugger, and even its internal compiler runs which does use a lot of memory.

Running without debugger
 - when just after splash screen and doing nothing the Task Manager reports 4904k
 - when the HyperNext compiler is compiling a HyperNext project it goes up to a max of 10272k
 - after compilation is finished it settles down to 6516k

Actually this version is building for i386-win32 as many users need the 32bit version.
However, I will certainly need a x64 version.

I've just tried to set for x86_64-win64 but it won't allow as naturally is states that 
  c:\lazarus\fpc\3.04\bin\i386-win32\fpc.exe does not support target x86_64-win32

I've just set to using the 3.2.0 fpc.exe which is for x86_64-win64. However, I must need to set something else up as all my project pascal files etc are not showing up.
I'ts getting late now, so I'll spend tomorrow(Thursday) on this.

As you can see I know very little about FPC  and consider myself a relative newbie.

Thank you for your help, its cleared a lot up for me.

Malkomax

  • New member
  • *
  • Posts: 7
Re: Windows 7 x64 and possible code limit causing error 203 ?
« Reply #6 on: November 26, 2020, 07:03:08 am »
The project is now set to x86_64-win64 and it runs both with and without the debugger.

There are also now no problems adding extra code.

However, there is an issue with the Win32 widget on x64 set but I'll sort that later.

Now just need to solve the problem for the i386-win32 version as detailed in my initial post because most users will use 32bit version.
To me it seems that FPC is generating good code but there is a problem associated with GDB (debugger).
Perhaps I've got some options incorrectly set, FPC links incorrectly or GDB itself has a problem.

Thank you for helping me make progress.
« Last Edit: November 26, 2020, 07:21:29 am by Malkomax »

PascalDragon

  • Hero Member
  • *****
  • Posts: 2758
  • Compiler Developer
Re: Windows 7 x64 and possible code limit causing error 203 ?
« Reply #7 on: November 26, 2020, 09:17:23 am »
Now just need to solve the problem for the i386-win32 version as detailed in my initial post because most users will use 32bit version.
To me it seems that FPC is generating good code but there is a problem associated with GDB (debugger).
Perhaps I've got some options incorrectly set, FPC links incorrectly or GDB itself has a problem.

Can you try to use FpDebug instead of GDB? Just in case that's really a problem in GDB.

Malkomax

  • New member
  • *
  • Posts: 7
Re: Windows 7 x64 and possible code limit causing error 203 ?
« Reply #8 on: November 27, 2020, 07:38:06 am »
Now just need to solve the problem for the i386-win32 version as detailed in my initial post because most users will use 32bit version.
To me it seems that FPC is generating good code but there is a problem associated with GDB (debugger).
Perhaps I've got some options incorrectly set, FPC links incorrectly or GDB itself has a problem.

Can you try to use FpDebug instead of GDB? Just in case that's really a problem in GDB.

Thank you for this idea, but sorry for my late reply as I'm still struggling to get the FpDebug working.

I have set the debugger type as LLDB debugger(with fpdebug)(Beta)
but cannot select the FpDebug.exe as there doesn't seem to be one.

I'm trying to compile/build the FpDebug.exe but compiler is giving error "Identifier not found TDbgBreakpoint".
I'll keep reading the docs etc and won't give up on this as would be really nice to see FpDebug working.


Malkomax

  • New member
  • *
  • Posts: 7
Re: Windows 7 x64 and possible code limit causing error 203 ?
« Reply #9 on: November 27, 2020, 01:50:09 pm »
Now just need to solve the problem for the i386-win32 version as detailed in my initial post because most users will use 32bit version.
To me it seems that FPC is generating good code but there is a problem associated with GDB (debugger).
Perhaps I've got some options incorrectly set, FPC links incorrectly or GDB itself has a problem.

Can you try to use FpDebug instead of GDB? Just in case that's really a problem in GDB.

I've now got FpDebug working by installing LazDebuggerFP as suggested by the documentation that you gave me the link to.
I should have read it more closely, my fault :(

Now the error dialog gives:

   Project HN_Pascal raised exception class 'EOutOfMemory' with message: Out of memory

   At address 40FE49



The assembler top line is:

   0040FE49 8B45EC                   mov eax,[ebp-$14]



By the way. I've just run this x32 HyperNext exe built without debug info on ReactOS and it actually works.

« Last Edit: November 27, 2020, 02:13:27 pm by Malkomax »

Seenkao

  • Full Member
  • ***
  • Posts: 118
Re: Windows 7 x64 and possible code limit causing error 203 ?
« Reply #10 on: December 03, 2020, 11:34:35 pm »
The -Ch option is only used for targets using the OMF format (namely the i8086 targets).
I've started to run into a heap 203 error when running my project on Windows 7 x64, and it seems I've hit some code limit on Free Pascal(Lazarus) project code size.

PascalDragon, я вас предупреждал об ошибках отладчика, и что он не показывает ошибку, но будет генерировать ошибки во время работы программы (если флаги включены) или на стадии компиляции. Неужели так и должно быть? Или стоит посмотреть на устранение проблем с отладкой, чтоб Lazarus выдавал соответствующие ошибки, а не генерировал их во время отладки программы? Тогда пользователь будет понимать что происходит.
google translate:
PascalDragon, I warned you about the debugger errors, and that it does not show an error, but it will generate errors during the program run (if the flags are enabled) or at the compilation stage. Is that how it should be? Or is it worth looking at fixing debugging problems so that Lazarus throws appropriate errors rather than generating them while debugging a program? Then the user will understand what is happening.

Peter H

  • Full Member
  • ***
  • Posts: 195
Re: Windows 7 x64 and possible code limit causing error 203 ?
« Reply #11 on: December 04, 2020, 05:16:05 am »
If there is a heap overflow this doesnt necessarily mean the heap overflow caused an error, this heap overflow can result from another error.

I think more sensitive than heapsize is stacksize.
Try to tick all checkboxes in debug options - including stackcheck - for your project,
Find out the stacksize and double it, if the error changes or vanishes, there is probably a stack problem.

Stack overflow can easily happen if you use large arrays as procedure arguments without the var or const or constref attribute together with recursion.
In turn this can cause a lot of other problems, which are difficult or impossible to debug.

FpDebug is beta, I would use gdb instead.
Be sure to use dwarf with sets as debugger format, not "default".
« Last Edit: December 04, 2020, 07:46:03 am by Peter H »

Thaddy

  • Hero Member
  • *****
  • Posts: 10705
Re: Windows 7 x64 and possible code limit causing error 203 ?
« Reply #12 on: December 04, 2020, 11:58:27 am »
As I wrote before, you can not simply blindly translate FreeBasic to Pascal constructs. You need to manually release the memory after use in Pascal. E.g, with classes you need to call free and with GetMem and AllocMem you need to call FreeMem.
Without that you will sooner or later get an EOutOfMemory.  And I am almost sure that is the cause.

Malkomax

  • New member
  • *
  • Posts: 7
Re: Windows 7 x64 and possible code limit causing error 203 ?
« Reply #13 on: December 15, 2020, 12:35:06 pm »
The -Ch option is only used for targets using the OMF format (namely the i8086 targets).
I've started to run into a heap 203 error when running my project on Windows 7 x64, and it seems I've hit some code limit on Free Pascal(Lazarus) project code size.

PascalDragon, я вас предупреждал об ошибках отладчика, и что он не показывает ошибку, но будет генерировать ошибки во время работы программы (если флаги включены) или на стадии компиляции. Неужели так и должно быть? Или стоит посмотреть на устранение проблем с отладкой, чтоб Lazarus выдавал соответствующие ошибки, а не генерировал их во время отладки программы? Тогда пользователь будет понимать что происходит.
google translate:
PascalDragon, I warned you about the debugger errors, and that it does not show an error, but it will generate errors during the program run (if the flags are enabled) or at the compilation stage. Is that how it should be? Or is it worth looking at fixing debugging problems so that Lazarus throws appropriate errors rather than generating them while debugging a program? Then the user will understand what is happening.

Thank you for your comment :)

 

TinyPortal © 2005-2018