Recent

Author Topic: Even more problems with untyped files - blockwrite causes crashes.  (Read 3969 times)

Stiivais

  • Newbie
  • Posts: 2
Disregard the following, after using the debugger i found out that my parsing is all wrong. Nevermind.

After finishing the program i started yesterday to convert binary data into ascii, i decided to make something that does the opposite: takes a file with byte values in ascii, separated by a delimiter character and converts them into a sequence of binary byte values.

Now, initially i had problems with speed, because i tried to write everything to an array, which lead to some rather grim memory usage (7.6 million values), but i worked it out by doing the interpretation, coversion and writing to the output file in one loop. I am not entirely sure if this would work (not really familiar with blockread and blockwrite), but at least i got somewhere.

The problems, however, start when i try to write the actual values to the file, using blockwrite - the application crashes. If i comment out the lines responsible for converting the data, the rest of the application works, so at least i have seemingly isolated the problem.
Here's the lines at fault:
Code: [Select]
buffer3[0]:=byte(strtoint(buffer2[0]));
    blockwrite(outputfile,buffer3,length(buffer3));
    result:=result+1;
If i comment out the first line (#76 in full source), then the application works.
buffer3 is an array of bytes, therefore it accepts only byte data type, which i get through the byte() command, but this in turn requires an integer inside of the brackets, because of which i use strtoint() command on the value of buffer2 array, which is an array of shortstrings. I am not certain why it is crashing. Could anyone explain to me what i am doing wrong?

Here's the full source code:
Code: [Select]

program fpc_template;

uses CRT,fileutil,sysutils;

var outputfile:file; console:text; inputfile: file of char;
    filename,filename2:string;
    n,count:longword; result:longint; clength,n2:byte;
    buffer,buffer3: array of byte; buffer2: array of shortstring;
    wait:boolean;

begin
clrscr;

//sets filenames and delimiter
wait:=false;
filename:='output.txt';
filename2:='output.vxl';

//checks command line parameters
if (ParamCount >0) and (ParamStr(1)<>'0') then filename:=ParamStr(1);
if (ParamCount >1) and (ParamStr(2)<>'0') then filename2:=ParamStr(2);
if (ParamCount >2) and (ParamStr(3)='wait') then wait:=true;

assign(inputfile,filename);
assign(outputfile,filename2);
assigncrt(console);
rewrite(console);

//outputs parameters to console
writeln(console,paramstr(0));
writeln('input - ',filename, ' (',fileutil.filesize(filename)/1000000:0:2,'MB)   output - ',filename2);
writeln;

//reads values to memory
count:=0; result:=0;
reset(inputfile,1);
count:=system.filesize(inputfile);  //gets the amount of characters
writeln(console,count,' characters found, reading...');
setlength(buffer,count);  //sets array length to fit the characters
blockread(inputfile,buffer[0],count,result); //  fills the array
writeln(console,result,' characters read, parsing...');
close(inputfile);

/////////////////////////////////////////////////////////////////////////

rewrite(outputfile,1);
n:=0; clength:=1; result:=0;
setlength(buffer3,1);
setlength(buffer2,0); setlength(buffer2,1);

/////////////////////////////////////////////////////////////////////////

for n:=0 to length(buffer)-1 do
begin
if (ord(buffer[n])>47) and (ord(buffer[n])< 58) and (n<length(buffer)-1) then //checks if current number is a number, not the last
begin  //is a number
clength:=clength+1;
    end
else
begin  //is a delimiter
if (n < length(buffer) - 1) then
    for n2:=clength-1 downto 1 do
    begin  //for each character in current selection (if not the last number)
buffer2[0]:=buffer2[0]+chr(buffer[n-n2]);
        end
else for n2:=clength-1 downto 0 do
    begin  //for each character in current selection (if last number)
    buffer2[0]:=buffer2[0]+chr(buffer[n-n2]);
        end;
clength:=1;
end;

    //WRITES VALUES
    buffer3[0]:=byte(strtoint(buffer2[0]));
    blockwrite(outputfile,buffer3,length(buffer3));
    result:=result+1;

//(optional) updates info text
if (n mod trunc(length(buffer)/500) = 0) then
    begin
        write(console,trunc(trunc(n/length(buffer)*100)),' %','  -  ',n,' out of ',length(buffer));
        gotoxy(1,wherey);
        end;
end;
close(outputfile);

reset(outputfile);
gotoxy(1,wherey);
writeln(console,result,' values written to ',filename2,' (',fileutil.filesize(filename2)/1000000:0:2,'MB)');
close(outputfile);

//optional info if wait is true
if wait = true then
begin
writeln(console,'program finished, press any key to quit.');
readkey;
end;

close(console);

end.

Also, the file produced, even without specifying the values to be saved, takes up as much space as the delimited text file, but the file from which the .vxl file was generated was almost 3 times smaller in size. Am i converting the data wrong by using blockread/blockwrite?
The idea behind these programs is this conversion of files in the following order:

.vxl (bytes stored in binary) --> .txt (ascii, delimited with a character)
.txt (the one in previous step) --> .vxl (the same format as previously)
more specifically: input.vxl - output.txt - output.vxl

Here is the source for the fixed version of the first program (to generate the .txt) - http://pastebin.com/cDV6y9kt
Here are 2 example files - the source .vxl and a generated .txt -
https://www.dropbox.com/s/kb7vgging0cbhtv/input.vxl?dl=0
https://www.dropbox.com/s/vtpfs53i55v7kvg/output.txt?dl=0

I have been reading the Wiki for about 6 hours now and can't find a solution to this problem. Learning pascal isn't quite as efortless as i imagined, but i am really hoping for your help, because even google doesn't turn up much.  :-\

Original post:
Quote
Hello everyone, i am making a program to read a binary file of bytes and to output it as a text file with the byte values, separated by a delimiter. Everything seemed to be working fine until the point where i tried to use blockread - i a crash no matter how i try to use the parameters.

Here's my code, crashes as soon as i remove the comment at line 34:
Code: [Select]
program fpc_template;

uses CRT,fileutil,sysutils;

var inputfile:file; outputfile,console:text;
    filename,filename2,delimiter:string;
    n,count:longword; result:longint;
    buffer: array of byte;

begin
clrscr;

//sets filenames and delimiter
delimiter:=',';
filename:='map.vxl';
filename2:='map.txt';

assign(inputfile,filename);
assign(outputfile,filename2);
assigncrt(console);
rewrite(console);

//outputs parameters to console
writeln(console,paramstr(0));
writeln('input - ',filename, ' (',fileutil.filesize(filename)/1000000:0:2,'MB)   output - ',filename2);
writeln;

//reads values to memory
count:=0; result:=0;
reset(inputfile);
count:=system.filesize(inputfile);  //gets the amount of values
writeln(console,count,' bytes found, reading...');
setlength(buffer,count);  //sets array length to fit amount of values found
//blockread(inputfile, buffer, count, result);  //fills the array
writeln(console,result,' bytes read, outputting to file...');
close(inputfile);

{
//writes values to file
rewrite(outputfile);
n:=0;
while n<=count do
begin
write(inttostr(buffer[n])+delimiter);  //writes count-1 values
n:=n+1;
end;
write(buffer[count]);  //writes the last value, so there is no delimiter at end

writeln(console,count,' values written to ',filename2,' (',fileutil.filesize(filename)/1000000:0:2,'MB)');
close(outputfile);
}

readkey;
end.

Here's an example file, just put it in the directory of the compiled .exe: https://www.dropbox.com/s/chasbqqeipc7c6b/map.vxl?dl=0
Any help will be appreciated.
« Last Edit: October 28, 2014, 07:15:40 pm by Stiivais »

howardpc

  • Hero Member
  • *****
  • Posts: 4144
Re: blockread causes crash
« Reply #1 on: October 27, 2014, 11:43:17 pm »
Buffer is a dynamic array, so you want to pass buffer[0] - where the data should start to be read from.
You also need to set the file record size to 1 to ensure the sizes correspond. So something like
Code: [Select]
//reads values to memory

count:=0; result:=0;
reset(inputfile,1);
count:=system.filesize(inputfile);  //gets the amount of values
writeln(console,count,' bytes found, reading...');
setlength(buffer,count);  //sets array length to fit amount of values found
blockread(inputfile, buffer[0], count, result);  //fills the array
writeln(console,result,' bytes read, outputting to file...');
close(inputfile);   

Stiivais

  • Newbie
  • Posts: 2
Re: blockread causes crash
« Reply #2 on: October 27, 2014, 11:59:03 pm »
Worked like a charm, thanks!
The wiki just wasn't all that clear on this...

Edit:
Now have new problems, main post updated.
« Last Edit: October 28, 2014, 05:10:46 pm by Stiivais »

 

TinyPortal © 2005-2018