Compiling existing 32bit FPC Programm for 64Bit

[TKoecher]

Hey guys,

I got a short (and probably very stupid) question.

We have a couple of FPC programs, made quite simple in FPC (not using Lazerus yet) - ending up in having 32Bit Programs.

Now, we would need the advantage of 64bit of being able to address more Memory (read: extend the clip of ~2Gb to >12GB).
It makes me feel kinda stupid, but I totally fail at this (I'm not the one who made the code either, nor I'm really into programming FPC).

Is there an easy Way to do so, or is this more complicated than I think?
Can you please give me a quick explanation how to archive this?

Thanks!

[Bart]

In compiler optiosn set th eappropriate target and architecture.
This may require you hvae cross-compilers on your system (32->64 bit)

On Windows it as easy as installing the win32->win64 cross compiler, then setting target to Win64, architecture to x86_64 and build.

You need to be aware that in the 32-bit era people used to abuse integers for pointer arithmetic etc.
However on 64-bit Integer is still 32-bit (use Int64/UInt64 for 64-bit), but pointers are 64-bit. You need to use PtrInt or PtrUInt if you want to cast a pointer to an integer-type (not only on 64-bit, but you should do that on all platforms to keep it cross-platform).

So, unfortunately you must study the source code to see if there are possible bugs.

Bart

[TKoecher]

Thank you!
Sounds good.

The Problem is just one (or a few) really big arrays.
Ar=Array[0..100000,0..100000] of byte;
for example.

Of course 32bit FPC fails on this, as it just overflows its usable ~2GB RAM (32GB Ram Available on the PC, Win7 64bit). Should consume ~10GB.

I'll give it a shoot and see if I get it running.

Btw, you speak about using Lazerus for this, right?

[Bart]

Btw, you speak about using Lazerus for this, right?

Yes, i did, when referring to how easy it was to set it up.
Using plain fpc, you will have to set target and architecture using commandline parameters (which is what Lazarus does when it calls the compiler).

Bart

[TKoecher]

Finally, I got a valid chance to test all this.
First of all, Lazarus runs great; it creates a 64Bit Task (seen in the Task Manager).
However, I struggle with making use of it.

I have a some kinda dummy program, doing in fact nothing but creating a huge array and making some stupid calculations than (to give me time to observe). If I make the array size to flood ~1,2GB RAM, it runs (but it already did that in FPC 32Bit).
If I try to extend the array by 10 times one directions (so ~12GB RAM), it stops with SIGSEGV error.
Okay, maybe the code is a bit messy for just compile it as 64Bit. I think I need to dig in more deep on here.

But then I tried to create 2 arrays, size of ~1,2GB each. And now the strange things begin. Now the debugger crashes and it tells me: ^error,msg="Error creating process Z:/Tobias/Test64Bit.exe, (error 193)."
I tried to use latest lazarus version (2.6.4) as well as 2.8 RC3. Same message.  I also tried to use gdb_77, but still the same.
And this is where I can’t find any solution. Google told me, the problems happens when Lazarus is not happy with the file path (containing spaces or whatever), but the path is pretty clean. I already tried to move to a different location, still the same. The compiling works, but if I try to start the .exe manually, it ends up on windows telling me, its not a valid win32 application…
Anybody got an idea?

[avra]

Anybody got an idea?

Can you start executable from command line? Also temporary switch off antivirus and try again.

[Thaddy]

The Problem is just one (or a few) really big arrays.
Ar=Array[0..100000,0..100000] of byte;

Well in a procedure or function this will probably bomb out the stack, but you can declare it as a pointer type and allocate on the heap.
Then you can get as much memory as is physically available.

[marcov]

Note that many 64-bit OSes only support very large structures on the heap.

Normal variables are limited to  2GB or 4GB in total for cheaper addressing (shorter instructions)

[Thaddy]

True.

[TKoecher]

Can you start executable from command line? Also temporary switch off antivirus and try again.

Nope, if I try to start form command line, I get the same error.
I cant switch of AV that easy... everything under control of the holy IT department

Well in a procedure or function this will probably bomb out the stack, but you can declare it as a pointer type and allocate on the heap.
Then you can get as much memory as is physically available.

I'm not sure if I did it any correct way, but I got something running. And it crashes the debugger the same way the 2 independent arrays do...
(still, you might got a link to good tutorial? Google overfloods me with information on this)

[josh]

Hi

Just done a test in laz compiled for x86_64 and using dynamic arrays and longint for indexing, no issue, can create the array and also assign values to every byte of the 2 dimensional array.

Form has just a tbutton and a tlabel on it, with the button onclick event doing the work.

Code below; it is probably not the best, as I am adjusting the length of the array many times when you click the button, I done this so you can visually see the application is doing something, as if I just set to max value the application appears to stop; it has not stopped just creating the 12GB of data.

Hope it helps, obviously the code can be improved in areas.

Code: Pascal  [Select]

1. unit Unit1;
2.  
3. {$mode objfpc}{$H+}
4.  
5. interface
6.  
7. uses
8.   Classes, SysUtils, FileUtil, Forms, Controls, Graphics, Dialogs, StdCtrls;
9.  
10. const max_size=100000;
11.  
12. type
13.  
14.   { TForm1 }
15.  
16.   TForm1 = class(TForm)
17.     Button1: TButton;
18.     Label1: TLabel;
19.     procedure Button1Click(Sender: TObject);
20.   private
21.  
22.   public
23.  
24.   end;
25.  
26. var
27.   Form1: TForm1;
28.  
29.   Ar:array of array of byte;
30.   a,b:longint;  // use longint to index the array not Integer
31.  
32. implementation
33.  
34. {$R *.lfm}
35.  
36. { TForm1 }
37.  
38. procedure TForm1.Button1Click(Sender: TObject);
39. const rat=100;
40. begin
41.   // Diable button to stop you pressing it twice
42.   button1.enabled:=false;
43.   application.ProcessMessages;
44.   //for loop below is to show visual indication the program is working, as without some indication user may assume application has crashed
45.   //if you check application processes you should see the memory used increasing
46.   for a:=1 to max_size div rat do
47.   begin
48.     label1.caption:='setting length of array to :'+inttostr(a*rat);
49.     SetLength(ar, a*rat, max_size);//a*rat);
50.     application.ProcessMessages;
51.   end;
52.   SetLength(ar, max_size+1, max_size+1);
53.   // Setting length to max+1 so that you can address all elements of array, as without I was getting sigsev error when indexing elements at max_size
54.  
55.   // Loop through array and set all elements of 2 D array to 255
56.   for a:=0 to max_size do
57.   begin
58.     for b:=0 to max_size do
59.     begin
60.       ar[a,b]:=255;
61.     end;
62.     label1.caption:=inttostr(a);
63.     application.ProcessMessages;
64.   end;
65.   button1.Enabled:=true; // Re-enable button
66.   application.ProcessMessages;
67. end;
68.  
69. end.
70.  
71.  

Additional.
After doing some timing test, a substantial increase in speed in creating the array with setlength is achieved my using cmem memory manager.
On my test system, without cmem creating array took over 110 seconds, and with cmem same routine completed in just under 10 seconds.
However the disadvantage was that the assigning of all values in the 2D array was slower by about 12% when using cmem.

[TKoecher]

Thank you for your effort josh,

I've tryed your code, but it gives me instant SIGSEGV (in include\control.inc line 3271) on first klick.

I also tryed to take my test-prog and compile it by command line and FPC ppcrossx64.exe; same behavior as Lazarus. (with 2 arrays, 1GB each, it gives me "no valid win32 application") 

[josh]

Hi

I have attached a zip of project, this is compiled with lazarus 32 bit on windows, and target set for 64 in project options.

Note this is created with lazarus trunk. Hopefully your lazarus should open the project.

You can un comment the cmem in the project lpr file.