Brand new iMac i5 27" retina OS10.10.6 cannot create array

[summerleas]

Have I bought a useless computer?

   BallotData = record
      BallotNumber : longint;
      NumberOfPreferences : smallint;
      Preferences : array[1..256] of byte;
      CurrentPreference : byte;
      CurrentValue : double;
      end;

var
   BallotsGroup1 : array[1 .. 1000000] of BallotData;
   BallotsGroup2 : array[1 .. 1000000] of BallotData;
   BallotsGroup3 : array[1 .. 1000000] of BallotData;
   BallotsGroup4 : array[1 .. 1000000] of BallotData;
   BallotsGroup5 : array[1 .. 1000000] of BallotData;
   BallotsGroup6 : array[1 .. 1000000] of BallotData;


I compiled this with:

fpc -Px86_64 -FeDesktop/Senate-preferences/SenateDebug Desktop/Senate-preferences/SenateTestExhaustedVotes.pas

The SenateDebug file reported:

Target OS: Darwin for x86_64
Compiling Desktop/Senate-preferences/SenateTestExhaustedVotes.pas
Assembling (pipe) Desktop/Senate-preferences/SenateTestExhaustedVotes.s
Linking Desktop/Senate-preferences/SenateTestExhaustedVotes
SenateTestExhaustedVotes.pas(132,49) Error: Error while linking
SenateTestExhaustedVotes.pas(132,49) Fatal: There were 1 errors compiling module, stopping
Fatal: Compilation aborted

Terminal reported:

    __TEXT/__text addr=0x1000015F0, size=0x00045FCE, fileOffset=0x000015F0, type=1
    __TEXT/__stubs addr=0x1000475BE, size=0x0000018C, fileOffset=0x000475BE, type=28
    __TEXT/__stub_helper addr=0x10004774C, size=0x00000318, fileOffset=0x0004774C, type=32
    __TEXT/.fpc addr=0x100047A68, size=0x0000002A, fileOffset=0x00047A68, type=0
    __TEXT/__const addr=0x100047AA0, size=0x00005513, fileOffset=0x00047AA0, type=0
    __TEXT/__unwind_info addr=0x10004CFB4, size=0x00000048, fileOffset=0x0004CFB4, type=22
    __DATA/__dyld addr=0x10004D000, size=0x00000038, fileOffset=0x0004D000, type=30
    __DATA/__la_symbol_ptr addr=0x10004D038, size=0x00000210, fileOffset=0x0004D038, type=27
    __DATA/__const addr=0x10004D250, size=0x0001158E, fileOffset=0x0004D250, type=0
    __DATA/__data addr=0x10005E7E0, size=0x00005468, fileOffset=0x0005E7E0, type=0
    __DATA/__common addr=0x100063C48, size=0x00001414, fileOffset=0x00000000, type=25
    __DATA/__bss addr=0x100065060, size=0x00001C44, fileOffset=0x00000000, type=25
    __DATA/__huge addr=0x100066CB0, size=0xBCE41084, fileOffset=0x00000000, type=25
ld: 32-bit RIP relative reference out of range (2640414934 max is +/-4GB): from _P$COUNTEXHAUSTEDBALLOTS_$$_PUTBALLOT$LONGINT$BALLOTDATA (0x100001650) to _U_$P$COUNTEXHAUSTEDBALLOTS_$$_BALLOTSGROUP6@0x9D5B3C40 (0x19D61A0B0) in '_P$COUNTEXHAUSTEDBALLOTS_$$_PUTBALLOT$LONGINT$BALLOTDATA' from Desktop/Senate-preferences/SenateTestExhaustedVotes.o for architecture x86_64
An error occurred while linking
Error: /usr/local/bin/ppcx64 returned an error exitcode


Does this mean (as I suspect) the iMac is useless because it can only access 32 bit memory blocks?

Or have I missed something?

[Phil]

Have I bought a useless computer?

More likely useless code. Whenever I see someone using very large arrays, the "bad idea" alarm goes off in my head.

Can you explain why you can't write whatever it is you're trying to accomplish so you don't need such large arrays?

[molly]

rinse and repeat

[summerleas]

The data must be accessed repeatedly out of order. The alternative is to repeatedly read the data file maybe 2000 times. There is nothing wrong with large arrays per se, although I would be happy to avoid them.

Anyway, the problem appears to be with the iMac.

[Phil]

The data must be accessed repeatedly out of order. The alternative is to repeatedly read the data file maybe 2000 times. There is nothing wrong with large arrays per se, although I would be happy to avoid them.

Anyway, the problem appears to be with the iMac.

Put the data into a file in the "out of order" order that you'll need it in.

Or put the data into a binary random-access file. Then your "array" can be as big as your disk.

[summerleas]

The former won't work, the latter might.

As a first step I was hoping that someone might know how Macs access RAM.

I am assuming that the problem is not in free Pascal, but maybe it is ...

[summerleas]

On another forum I was directed to:

https://developer.apple.com/library/content/documentation/Performance/Conceptual/ManagingMemory/Articles/AboutMemory.html

It states that "OS X provides approximately 18 exabytes of addressable space for 64-bit processes."

In which case, my program should compile and run because it requires only about 4 or 5 GB. So the problem is with free Pascal or the way I am using it.

Any suggestions?

[summerleas]

In the Free Pascal Reference guide (page 24)

"With the exception of int64, qword and Real types, all base types are ordinal types."

And on page 39 an array is defined as: array[ordinal type, ...]

So can the index of an array be of type int64?

[Thaddy]

Most likely the problem is NOT the Mac.
Can you give us a compilable example to reproduce the error.I dusted off my mac recently, so I can debug things again.

The problem is that if you yourself don't provide a compilable example, we start to "assume" the probably correct things and then it suddenly works.
Better have you give a small example that can be compiled.

BTW: I suspect a documentation bug: With the exception of int64, qword and real types"
By now int64 and qword are native ordinal types on most platforms, afaik.
And even real is an alias for double, which has never been an ordinal type anyway.

Anyway some proof:

Code: Pascal  [Select][+][-]

1. program untitled;
2. var
3.   a:int64 = 0;
4.   b:qword = 0;
5.   c:real := 0.0;
6. begin
7.   inc(a);
8.   inc(b);
9.   a := pred(a);
10.   b := pred(b);
11. end.

I filed a bug report against the documentation. Mantis #0031280

Don't let this distract you: we will still need a compilable example that replicates your issue.

[mischi]

Whether int64 and qword are taken as ordinal seems to depend on 32 bit / 64 bit.
The following test program compiles ok with 64 bit but gives the error message below with 32 bit.

Code: Pascal  [Select][+][-]

1. program int4Test;
2. var index: int64;
3. begin
4.   for index := 1 to 10 do
5.     writeln (index)
6. end.

Excerpt of the error message on 32 bit (Darwin for i386):
...
Compiling int4Test.pas
int4Test.pas(4,7) Error: Ordinal expression expected

[Cyrax]

Use dynamic arrays, not static ones.

Also make sure that you have enough swap space, too.

[Jonas Maebe]

OS X does not support more than 4GB of statically allocated data, even on 64 bit platforms (probably for efficiency reasons). You have to allocate your data dynamically if you want more than that.

[Noodly]

The Preferences array is taking up most of the memory.

For each BallotData record do you really need 256 preferences, each with 256 potential values? Obviously I don't know what your use cases are, but have you considered if there might be a different way of handling preferences with a smaller data structure?

[Handoko]

I think the TS's main intention is not focus on the use case, he/she just wanted to push FPC to its limits to see how far it can handle large memory or at least to make sure it can handle large memory.

Several 'smarter' solutions have already been suggested, but one interesting question still remain:
Why 64-bit iMac OS 10 with 8 GB RAM can't compile program that uses 4 GB memory. Is it FPC problem or OS issue?

[Thaddy]

Is it FPC problem or OS issue?

Well. Jonas is the expert: OS issue.