Slow speed reading a binary file

[marcov]

Hi - This is all very interesting. Just for fun I ran Thaddy's code (reply #6) with a counter added (just to give me some return). I used a 7.5MB text file. As before, the pascal version took about 32 seconds, while the c version was almost instantaneous (~5 sec). I am comparing older-style pascal on purpose, as I am simply interested as to why it is slower, as I see no reason why it should be (as stated by others)...

The default buffer size of FPC is a TP compatible 128 bytes. Use SetTextbuf to allocate a large ones as mentioned elsewhere in the thread.

[Laur]

I hope this is the right area to ask this. I have noticed that reading binary files is very slow compared to other languages, at least using older pascal verbage. As an example this:

while not EOF(subject_file) do
    begin
      read(subject_file, b);
      TotalCount := TotalCount + 1;
    end;

is more than 10x slower than equivalent C language. I have wondered about this for years, and finally decided to ask for some input. Does anyone have an ideas about this?

Try use file mapping - something like this (i do not know fp but it should work - for any system):

p = createMapOfFile(filename);

and now the p i just a pointer to data stored on a file.

if p is char type then:
p[7] is a 8 byte of the file...

This method is the fastest possible for file reading, and in any op. system.

[DMStevenson]

Thanks! After your suggestions and some reading I think I understand what's behind the slowness, and yes, increasing the buffer makes all the difference. Thank's everyone!

[Thaddy]

Hi - This is all very interesting. Just for fun I ran Thaddy's code (reply #6) with a counter added (just to give me some return). I used a 7.5MB text file. As before, the pascal version took about 32 seconds, while the c version was almost instantaneous (~5 sec). I am comparing older-style pascal on purpose, as I am simply interested as to why it is slower, as I see no reason why it should be (as stated by others)...

that can only be caused by buffering and your original code HAS no buffering. When used on par, the compilers are on par.
btw I don't believe you tested with -O1 or you forgot about {$I-} or you forgot about not including sysutils. (because I did test it)

[PascalDragon]

This method is the fastest possible for file reading, and in any op. system.

Not in every operating system. There can be operating systems that don't support that concept at all either because they don't support virtual memory (e.g. MS-DOS) or because a concept like that is an unnecessary complication.

[DMStevenson]

Hi - Yes, Thaddy, you are right. When I 'fixed' the buffering the execution between pascal and C were identical. Thanks very much for the help!!!