Lazarus

Free Pascal => General => Topic started by: dbannon on August 05, 2017, 04:15:02 am

Title: [SOLVED] Using TFileStream to write plan old text files
Post by: dbannon on August 05, 2017, 04:15:02 am
TFileStream does not behave as I expect when I send it a series of strings, have tried with both shortstring and ANSIstring.  fsOut.Write(S1, length(S1)) makes what appears as a binary or data file. Using S1[1] does what I need. Here is an example -
Code: Pascal  [Select][+][-]
  1. program SaveStringToPathDemo;
  2. {$mode objfpc}
  3. uses
  4.   Classes, sysutils;
  5. var
  6.   fsOut : TFileStream;
  7.   S1, S2, S3 : ANSIstring;
  8.  
  9. begin
  10.     S1 := 'This is some text '#10;
  11.     S2 := 'And this is some more ';
  12.     S3 := 'This is probably enough.'#10;
  13.     fsOut := TFileStream.Create('TestFile1.txt', fmCreate);
  14.     fsOut.Write(S1, length(S1));
  15.     fsOut.Write(S2, length(S2));
  16.     fsOut.Write(S3, length(S3));
  17.     fsOut.Free;
  18.     fsOut := TFileStream.Create('TestFile2.txt', fmCreate);
  19.     fsOut.Write(S1[1], length(S1));
  20.     fsOut.Write(S2[1], length(S2));
  21.     fsOut.Write(S3[1], length(S3));
  22.     fsOut.Free;
  23. end.

Running this writes two file, only the second is a clean text file.

Code: Pascal  [Select][+][-]
  1. db:~/ObjectPascal$ file Test*
  2. TestFile1.txt: data
  3. TestFile2.txt: ASCII text

Now, finally, my question. Is this a bug or is that an "undocumented Feature" ?  If the latter, I'll document it on eg http://wiki.freepascal.org/File_Handling_In_Pascal

My searches indicate that this is the behaviour since, at least 2009 so "fixing" it would be pretty brave I'd expect. But newcomers like me need to be told (I say newcomer in that last time I used Pascal/Delphi, the streams approach did not exist).

David



Title: Re: Using TFileStream to write plan old text files
Post by: taazz on August 05, 2017, 04:44:02 am
Quote from: dbannon on August 05, 2017, 04:15:02 am
Now, finally, my question. Is this a bug or is that an "undocumented Feature" ?  If the latter, I'll document it on eg http://wiki.freepascal.org/File_Handling_In_Pascal (http://wiki.freepascal.org/File_Handling_In_Pascal)
Neither, its the result of untyped constants and misunderstanding what the modern string variable holds. You can think of a string as a pointer to an array of characters. The variable
Code: Pascal  [Select][+][-]
  1. var
  2.   vStr:String;
  3. begin
  4.   vStr := 'This is a string';
  5. end;
  6.  
holds the memory address of the first character of the string in the heap. passing a string in an untyped paramater you pass the address of the pointer. It has been like this from the delphi 2 days that huge strings introduced circa 1995.
Title: Re: Using TFileStream to write plan old text files
Post by: RAW on August 05, 2017, 04:49:59 am
This works too...

Code: Pascal  [Select][+][-]
  1. Procedure TForm1.FormClick(Sender: TObject);
  2.   Var
  3.    fs : TFileStream;
  4.    str: String;
  5.  Begin
  6.   str:= 'Hello Hello Hello';
  7.  
  8.   fs:= TFileStream.Create('Hello.txt', fmCreate);
  9.    Try
  10.     fs.WriteBuffer(Pointer(str)^, Length(str));
  11.    Finally
  12.     fs.Free;
  13.    End;
  14.  End;
Title: Re: Using TFileStream to write plan old text files
Post by: dbannon on August 05, 2017, 04:56:28 am
Hi taazz, thanks for that. Yes, aware of how it happens now but my point is its not what an uninformed user would expect. The function accepts a string, other functions that accept a string know not to use the first char (string) or the management block (ANSIstring).

I sure agree that changing the behaviour is totally off the agenda but am asking why I should not document what I found. I spent, maybe, 2 hours looking at this, I'd like to prevent other people from making the same assumptions.

But new enough to want to show my plan to the more experienced before editing the wiki ....

So, streams been there since Delphi 2 ?  I used only Delphi 1, long time ago !

Thanks mate !

David
Title: Re: Using TFileStream to write plan old text files
Post by: howardpc on August 05, 2017, 05:01:37 am
You could use the filestream methods dedicated to strings, and not worry about the file format itself.
e.g.
Code: Pascal  [Select][+][-]
  1. program SaveStringToPathDemo;
  2.  
  3. {$mode objfpc}{$H+}
  4.  
  5. uses
  6.   Classes, sysutils;
  7.  
  8. const
  9.   File_Name = 'TestFile1.txt';
  10.  
  11. var
  12.   fs: TFileStream;
  13.   S1, S2, S3 : AnsiString;
  14.  
  15. begin
  16.     S1 := 'This is some text '#10;
  17.     S2 := 'And this is some more ';
  18.     S3 := 'This is probably enough.'#10;
  19.     fs:=TFileStream.Create(File_Name, fmCreate);
  20.     try
  21.       fs.WriteAnsiString(S1);
  22.       fs.WriteAnsiString(S2);
  23.       fs.WriteAnsiString(S3);
  24.     finally
  25.       fs.Free;
  26.     end;
  27.  
  28.     s1:=''; s2:=''; s3:='';
  29.     fs:=TFileStream.Create(File_Name, fmOpenRead);
  30.     try
  31.       s1:=fs.ReadAnsiString;
  32.       s2:=fs.ReadAnsiString;
  33.       s3:=fs.ReadAnsiString;
  34.     finally
  35.       fs.Free;
  36.     end;
  37.     WriteLn(s1);
  38.     WriteLn(s2);
  39.     WriteLn(s3);
  40.     ReadLn;
  41. end.
Title: Re: Using TFileStream to write plan old text files
Post by: RAW on August 05, 2017, 05:02:02 am
Quote
...uninformed user would expect.
Maybe uninformed users would take a look at the wiki...
It takes seconds to find several examples with str[1]...  :P

No harm meant ...  :D
Title: Re: Using TFileStream to write plan old text files
Post by: taazz on August 05, 2017, 05:05:28 am
Quote from: dbannon on August 05, 2017, 04:56:28 am
Hi taazz, thanks for that. Yes, aware of how it happens now but my point is its not what an uninformed user would expect. The function accepts a string, other functions that accept a string know not to use the first char (string) or the management block (ANSIstring).
No it does not accept a string, it accepts a buffer any buffer. If it was of type string you would not be able to pass anything but strings.
Quote from: dbannon on August 05, 2017, 04:56:28 am
I sure agree that changing the behaviour is totally off the agenda but am asking why I should not document what I found. I spent, maybe, 2 hours looking at this, I'd like to prevent other people from making the same assumptions.

But new enough to want to show my plan to the more experienced before editing the wiki ....

So, streams been there since Delphi 2 ?  I used only Delphi 1, long time ago !

Thanks mate !

David
It is valuable to have the behavior documented as a caution for the new comers I agree, the exact wording or presentation of the behavior I leave it to more experienced members.
Title: Re: Using TFileStream to write plan old text files
Post by: taazz on August 05, 2017, 05:07:27 am
Quote from: howardpc on August 05, 2017, 05:01:37 am
You could use the filestream methods dedicated to strings, and not worry about the file format itself.
Have you looked at those method your self? they inject the length of the string infront of every string written. that might be acceptable and might not.
Title: Re: Using TFileStream to write plan old text files
Post by: dbannon on August 05, 2017, 05:45:07 am
Quote from: RAW on August 05, 2017, 05:02:02 am
Maybe uninformed users would take a look at the wiki...
It takes seconds to find several examples with str[1]...  :P

Well, no, not really. It took me (and I can be as uninformed as anyone) over two hours to find  http://free-pascal-general.1045716.n5.nabble.com/Creating-text-files-with-TFileStream-td2824859.html   And thats obviously not on THE wiki. There is one example on http://wiki.freepascal.org/File_Handling_In_Pascal, is slipped into the code without any explanation or attempt to draw a reader's attention. Yep, I missed it after going over that page several times, sigh ...

Anyway, I think RAW's approach, Pointer(str)^ is better in that its more obvious in what its doing. Wordier but clearer I think. I also think the wiki needs a few words added and I'll do unless someone says 'no'.
Title: Re: Using TFileStream to write plan old text files
Post by: RAW on August 05, 2017, 05:50:11 am
 :)
EDIT: The second hit has got 2 examples... that's not bad...  :)
Title: Re: Using TFileStream to write plan old text files
Post by: dbannon on August 05, 2017, 05:56:31 am
Quote from: howardpc on August 05, 2017, 05:01:37 am
You could use the filestream methods dedicated to strings, and not worry about the file format itself.
e.g.
       fs.WriteAnsiString(S1);
 

Actually howardpc, I tried that before posting. It does not produce a clean ascii text file. Did you try it yourself  ? Be interesting to know if you cat'ed the file at the console or relied on your programme being able to re-read it ? I suspect some (eg) editors, trying to be helpful, filter out some things they think, quite rightly, don't belong.
Title: Re: Using TFileStream to write plan old text files
Post by: Handoko on August 05, 2017, 06:38:51 am
TFileStream.WriteAnsiString works as it should be. It doesn't mean to produce text file. It actually write the string's length first before the string's content, and it does not write #13 after the end of the string.
Title: Re: Using TFileStream to write plan old text files
Post by: RAW on August 05, 2017, 07:05:17 am
This works for me... (Read file...)

Code: Pascal  [Select][+][-]
  1. Procedure TForm1.FormClick(Sender: TObject);
  2.   Var
  3.    fs : TFileStream;
  4.    str: String;
  5.  Begin
  6.   str:= Application.Location+'LAZARUS.txt';
  7.  
  8.   If FileExists(str)
  9.   Then
  10.    Begin
  11.     fs:= TFileStream.Create(str, fmOpenRead);
  12.      Try
  13.       SetLength(str, fs.Size);
  14.       fs.ReadBuffer(Pointer(str)^, Length(str)); // or fs.Read(str[1], fs.Size);
  15.      Finally
  16.       fs.Free;
  17.      End;
  18.     ShowMessage(str);
  19.    End;
  20.  End;
Title: Re: Using TFileStream to write plan old text files
Post by: howardpc on August 05, 2017, 09:25:48 am
Quote from: dbannon on August 05, 2017, 05:56:31 am
Quote from: howardpc on August 05, 2017, 05:01:37 am
You could use the filestream methods dedicated to strings, and not worry about the file format itself.
e.g.
       fs.WriteAnsiString(S1);
 

Actually howardpc, I tried that before posting. It does not produce a clean ascii text file. Did you try it yourself  ? Be interesting to know if you cat'ed the file at the console or relied on your programme being able to re-read it ? I suspect some (eg) editors, trying to be helpful, filter out some things they think, quite rightly, don't belong.

My comment to not worry about the file format itself was intended to indicate that TFileStream ReadAnsiString/WriteAnsiString are not designed to create traditional .txt files (structured only by CR/LF) but binary files that can be read and written consistently using streaming, just as Readln/Writeln can be used for reading and writing traditional .txt files using classic Pascal routines. Their advantage is the increased speed of reading this gives since extra string length information is stored in the file (so does not have to be calculated by parsing as the file is read as with a .txt file). The disadvantage is the exotic format, and the slightly larger file size.
Title: Re: Using TFileStream to write plan old text files
Post by: dbannon on August 05, 2017, 10:50:07 am
Quote from: howardpc on August 05, 2017, 09:25:48 am
My comment to not worry about the file format itself was intended to indicate that TFileStream ReadAnsiString/WriteAnsiString are not designed to create traditional .txt files (structured only by CR/LF) but binary files that can be read and written consistently using streaming, just as Readln/Writeln can be used for reading and writing traditional .txt files using classic Pascal routines. Their advantage is the increased speed of reading this gives since extra string length information is stored in the file (so does not have to be calculated by parsing as the file is read as with a .txt file). The disadvantage is the exotic format, and the slightly larger file size.

Ah, I see what you mean. Might suit some others but certainly not me. I'm writing out some xml to be read in another application so must be plain old text file I'm afraid.

Anyway, thanks everyone for you input, great to get such a response. Think we are done here.
Title: Re: Using TFileStream to write plan old text files
Post by: Thaddy on August 05, 2017, 10:59:58 am
Quote from: dbannon on August 05, 2017, 10:50:07 am
Ah, I see what you mean. Might suit some others but certainly not me. I'm writing out some xml to be read in another application so must be plain old text file I'm afraid.
If you are writing xml, just use a TFilestream and  write buffered, do not use the string functions in that case.
Point is: XML doesn't care about line endings or zero termination at all, so provided your other application expects valid XML that should simply work.
You will loose some text formatting (not within the XML tags, though), but that formatting is not part of XML itself..... It is just to make it easy on the eyes.
TinyPortal © 2005-2018