Recent

Author Topic: [SOLVED] Using TFileStream to write plan old text files  (Read 20424 times)

dbannon

  • Hero Member
  • *****
  • Posts: 2786
    • tomboy-ng, a rewrite of the classic Tomboy
[SOLVED] Using TFileStream to write plan old text files
« on: August 05, 2017, 04:15:02 am »
TFileStream does not behave as I expect when I send it a series of strings, have tried with both shortstring and ANSIstring.  fsOut.Write(S1, length(S1)) makes what appears as a binary or data file. Using S1[1] does what I need. Here is an example -
Code: Pascal  [Select][+][-]
  1. program SaveStringToPathDemo;
  2. {$mode objfpc}
  3. uses
  4.   Classes, sysutils;
  5. var
  6.   fsOut : TFileStream;
  7.   S1, S2, S3 : ANSIstring;
  8.  
  9. begin
  10.     S1 := 'This is some text '#10;
  11.     S2 := 'And this is some more ';
  12.     S3 := 'This is probably enough.'#10;
  13.     fsOut := TFileStream.Create('TestFile1.txt', fmCreate);
  14.     fsOut.Write(S1, length(S1));
  15.     fsOut.Write(S2, length(S2));
  16.     fsOut.Write(S3, length(S3));
  17.     fsOut.Free;
  18.     fsOut := TFileStream.Create('TestFile2.txt', fmCreate);
  19.     fsOut.Write(S1[1], length(S1));
  20.     fsOut.Write(S2[1], length(S2));
  21.     fsOut.Write(S3[1], length(S3));
  22.     fsOut.Free;
  23. end.

Running this writes two file, only the second is a clean text file.

Code: Pascal  [Select][+][-]
  1. db:~/ObjectPascal$ file Test*
  2. TestFile1.txt: data
  3. TestFile2.txt: ASCII text

Now, finally, my question. Is this a bug or is that an "undocumented Feature" ?  If the latter, I'll document it on eg http://wiki.freepascal.org/File_Handling_In_Pascal

My searches indicate that this is the behaviour since, at least 2009 so "fixing" it would be pretty brave I'd expect. But newcomers like me need to be told (I say newcomer in that last time I used Pascal/Delphi, the streams approach did not exist).

David



« Last Edit: August 05, 2017, 10:51:15 am by dbannon »
Lazarus 3, Linux (and reluctantly Win10/11, OSX Monterey)
My Project - https://github.com/tomboy-notes/tomboy-ng and my github - https://github.com/davidbannon

taazz

  • Hero Member
  • *****
  • Posts: 5368
Re: Using TFileStream to write plan old text files
« Reply #1 on: August 05, 2017, 04:44:02 am »
Now, finally, my question. Is this a bug or is that an "undocumented Feature" ?  If the latter, I'll document it on eg http://wiki.freepascal.org/File_Handling_In_Pascal
Neither, its the result of untyped constants and misunderstanding what the modern string variable holds. You can think of a string as a pointer to an array of characters. The variable
Code: Pascal  [Select][+][-]
  1. var
  2.   vStr:String;
  3. begin
  4.   vStr := 'This is a string';
  5. end;
  6.  
holds the memory address of the first character of the string in the heap. passing a string in an untyped paramater you pass the address of the pointer. It has been like this from the delphi 2 days that huge strings introduced circa 1995.
Good judgement is the result of experience … Experience is the result of bad judgement.

OS : Windows 7 64 bit
Laz: Lazarus 1.4.4 FPC 2.6.4 i386-win32-win32/win64

RAW

  • Hero Member
  • *****
  • Posts: 868
Re: Using TFileStream to write plan old text files
« Reply #2 on: August 05, 2017, 04:49:59 am »
This works too...

Code: Pascal  [Select][+][-]
  1. Procedure TForm1.FormClick(Sender: TObject);
  2.   Var
  3.    fs : TFileStream;
  4.    str: String;
  5.  Begin
  6.   str:= 'Hello Hello Hello';
  7.  
  8.   fs:= TFileStream.Create('Hello.txt', fmCreate);
  9.    Try
  10.     fs.WriteBuffer(Pointer(str)^, Length(str));
  11.    Finally
  12.     fs.Free;
  13.    End;
  14.  End;
Windows 7 Pro (x64 Sp1) & Windows XP Pro (x86 Sp3).

dbannon

  • Hero Member
  • *****
  • Posts: 2786
    • tomboy-ng, a rewrite of the classic Tomboy
Re: Using TFileStream to write plan old text files
« Reply #3 on: August 05, 2017, 04:56:28 am »
Hi taazz, thanks for that. Yes, aware of how it happens now but my point is its not what an uninformed user would expect. The function accepts a string, other functions that accept a string know not to use the first char (string) or the management block (ANSIstring).

I sure agree that changing the behaviour is totally off the agenda but am asking why I should not document what I found. I spent, maybe, 2 hours looking at this, I'd like to prevent other people from making the same assumptions.

But new enough to want to show my plan to the more experienced before editing the wiki ....

So, streams been there since Delphi 2 ?  I used only Delphi 1, long time ago !

Thanks mate !

David
Lazarus 3, Linux (and reluctantly Win10/11, OSX Monterey)
My Project - https://github.com/tomboy-notes/tomboy-ng and my github - https://github.com/davidbannon

howardpc

  • Hero Member
  • *****
  • Posts: 4144
Re: Using TFileStream to write plan old text files
« Reply #4 on: August 05, 2017, 05:01:37 am »
You could use the filestream methods dedicated to strings, and not worry about the file format itself.
e.g.
Code: Pascal  [Select][+][-]
  1. program SaveStringToPathDemo;
  2.  
  3. {$mode objfpc}{$H+}
  4.  
  5. uses
  6.   Classes, sysutils;
  7.  
  8. const
  9.   File_Name = 'TestFile1.txt';
  10.  
  11. var
  12.   fs: TFileStream;
  13.   S1, S2, S3 : AnsiString;
  14.  
  15. begin
  16.     S1 := 'This is some text '#10;
  17.     S2 := 'And this is some more ';
  18.     S3 := 'This is probably enough.'#10;
  19.     fs:=TFileStream.Create(File_Name, fmCreate);
  20.     try
  21.       fs.WriteAnsiString(S1);
  22.       fs.WriteAnsiString(S2);
  23.       fs.WriteAnsiString(S3);
  24.     finally
  25.       fs.Free;
  26.     end;
  27.  
  28.     s1:=''; s2:=''; s3:='';
  29.     fs:=TFileStream.Create(File_Name, fmOpenRead);
  30.     try
  31.       s1:=fs.ReadAnsiString;
  32.       s2:=fs.ReadAnsiString;
  33.       s3:=fs.ReadAnsiString;
  34.     finally
  35.       fs.Free;
  36.     end;
  37.     WriteLn(s1);
  38.     WriteLn(s2);
  39.     WriteLn(s3);
  40.     ReadLn;
  41. end.

RAW

  • Hero Member
  • *****
  • Posts: 868
Re: Using TFileStream to write plan old text files
« Reply #5 on: August 05, 2017, 05:02:02 am »
Quote
...uninformed user would expect.
Maybe uninformed users would take a look at the wiki...
It takes seconds to find several examples with str[1]...  :P

No harm meant ...  :D
Windows 7 Pro (x64 Sp1) & Windows XP Pro (x86 Sp3).

taazz

  • Hero Member
  • *****
  • Posts: 5368
Re: Using TFileStream to write plan old text files
« Reply #6 on: August 05, 2017, 05:05:28 am »
Hi taazz, thanks for that. Yes, aware of how it happens now but my point is its not what an uninformed user would expect. The function accepts a string, other functions that accept a string know not to use the first char (string) or the management block (ANSIstring).
No it does not accept a string, it accepts a buffer any buffer. If it was of type string you would not be able to pass anything but strings.
I sure agree that changing the behaviour is totally off the agenda but am asking why I should not document what I found. I spent, maybe, 2 hours looking at this, I'd like to prevent other people from making the same assumptions.

But new enough to want to show my plan to the more experienced before editing the wiki ....

So, streams been there since Delphi 2 ?  I used only Delphi 1, long time ago !

Thanks mate !

David
It is valuable to have the behavior documented as a caution for the new comers I agree, the exact wording or presentation of the behavior I leave it to more experienced members.
Good judgement is the result of experience … Experience is the result of bad judgement.

OS : Windows 7 64 bit
Laz: Lazarus 1.4.4 FPC 2.6.4 i386-win32-win32/win64

taazz

  • Hero Member
  • *****
  • Posts: 5368
Re: Using TFileStream to write plan old text files
« Reply #7 on: August 05, 2017, 05:07:27 am »
You could use the filestream methods dedicated to strings, and not worry about the file format itself.
Have you looked at those method your self? they inject the length of the string infront of every string written. that might be acceptable and might not.
Good judgement is the result of experience … Experience is the result of bad judgement.

OS : Windows 7 64 bit
Laz: Lazarus 1.4.4 FPC 2.6.4 i386-win32-win32/win64

dbannon

  • Hero Member
  • *****
  • Posts: 2786
    • tomboy-ng, a rewrite of the classic Tomboy
Re: Using TFileStream to write plan old text files
« Reply #8 on: August 05, 2017, 05:45:07 am »
Maybe uninformed users would take a look at the wiki...
It takes seconds to find several examples with str[1]...  :P

Well, no, not really. It took me (and I can be as uninformed as anyone) over two hours to find  http://free-pascal-general.1045716.n5.nabble.com/Creating-text-files-with-TFileStream-td2824859.html   And thats obviously not on THE wiki. There is one example on http://wiki.freepascal.org/File_Handling_In_Pascal, is slipped into the code without any explanation or attempt to draw a reader's attention. Yep, I missed it after going over that page several times, sigh ...

Anyway, I think RAW's approach, Pointer(str)^ is better in that its more obvious in what its doing. Wordier but clearer I think. I also think the wiki needs a few words added and I'll do unless someone says 'no'.
Lazarus 3, Linux (and reluctantly Win10/11, OSX Monterey)
My Project - https://github.com/tomboy-notes/tomboy-ng and my github - https://github.com/davidbannon

RAW

  • Hero Member
  • *****
  • Posts: 868
Re: Using TFileStream to write plan old text files
« Reply #9 on: August 05, 2017, 05:50:11 am »
 :)
EDIT: The second hit has got 2 examples... that's not bad...  :)
« Last Edit: August 05, 2017, 05:53:31 am by RAW »
Windows 7 Pro (x64 Sp1) & Windows XP Pro (x86 Sp3).

dbannon

  • Hero Member
  • *****
  • Posts: 2786
    • tomboy-ng, a rewrite of the classic Tomboy
Re: Using TFileStream to write plan old text files
« Reply #10 on: August 05, 2017, 05:56:31 am »
You could use the filestream methods dedicated to strings, and not worry about the file format itself.
e.g.
       fs.WriteAnsiString(S1);
 

Actually howardpc, I tried that before posting. It does not produce a clean ascii text file. Did you try it yourself  ? Be interesting to know if you cat'ed the file at the console or relied on your programme being able to re-read it ? I suspect some (eg) editors, trying to be helpful, filter out some things they think, quite rightly, don't belong.
Lazarus 3, Linux (and reluctantly Win10/11, OSX Monterey)
My Project - https://github.com/tomboy-notes/tomboy-ng and my github - https://github.com/davidbannon

Handoko

  • Hero Member
  • *****
  • Posts: 5130
  • My goal: build my own game engine using Lazarus
Re: Using TFileStream to write plan old text files
« Reply #11 on: August 05, 2017, 06:38:51 am »
TFileStream.WriteAnsiString works as it should be. It doesn't mean to produce text file. It actually write the string's length first before the string's content, and it does not write #13 after the end of the string.

RAW

  • Hero Member
  • *****
  • Posts: 868
Re: Using TFileStream to write plan old text files
« Reply #12 on: August 05, 2017, 07:05:17 am »
This works for me... (Read file...)

Code: Pascal  [Select][+][-]
  1. Procedure TForm1.FormClick(Sender: TObject);
  2.   Var
  3.    fs : TFileStream;
  4.    str: String;
  5.  Begin
  6.   str:= Application.Location+'LAZARUS.txt';
  7.  
  8.   If FileExists(str)
  9.   Then
  10.    Begin
  11.     fs:= TFileStream.Create(str, fmOpenRead);
  12.      Try
  13.       SetLength(str, fs.Size);
  14.       fs.ReadBuffer(Pointer(str)^, Length(str)); // or fs.Read(str[1], fs.Size);
  15.      Finally
  16.       fs.Free;
  17.      End;
  18.     ShowMessage(str);
  19.    End;
  20.  End;
« Last Edit: August 05, 2017, 11:38:09 pm by RAW »
Windows 7 Pro (x64 Sp1) & Windows XP Pro (x86 Sp3).

howardpc

  • Hero Member
  • *****
  • Posts: 4144
Re: Using TFileStream to write plan old text files
« Reply #13 on: August 05, 2017, 09:25:48 am »
You could use the filestream methods dedicated to strings, and not worry about the file format itself.
e.g.
       fs.WriteAnsiString(S1);
 

Actually howardpc, I tried that before posting. It does not produce a clean ascii text file. Did you try it yourself  ? Be interesting to know if you cat'ed the file at the console or relied on your programme being able to re-read it ? I suspect some (eg) editors, trying to be helpful, filter out some things they think, quite rightly, don't belong.

My comment to not worry about the file format itself was intended to indicate that TFileStream ReadAnsiString/WriteAnsiString are not designed to create traditional .txt files (structured only by CR/LF) but binary files that can be read and written consistently using streaming, just as Readln/Writeln can be used for reading and writing traditional .txt files using classic Pascal routines. Their advantage is the increased speed of reading this gives since extra string length information is stored in the file (so does not have to be calculated by parsing as the file is read as with a .txt file). The disadvantage is the exotic format, and the slightly larger file size.

dbannon

  • Hero Member
  • *****
  • Posts: 2786
    • tomboy-ng, a rewrite of the classic Tomboy
Re: Using TFileStream to write plan old text files
« Reply #14 on: August 05, 2017, 10:50:07 am »
My comment to not worry about the file format itself was intended to indicate that TFileStream ReadAnsiString/WriteAnsiString are not designed to create traditional .txt files (structured only by CR/LF) but binary files that can be read and written consistently using streaming, just as Readln/Writeln can be used for reading and writing traditional .txt files using classic Pascal routines. Their advantage is the increased speed of reading this gives since extra string length information is stored in the file (so does not have to be calculated by parsing as the file is read as with a .txt file). The disadvantage is the exotic format, and the slightly larger file size.

Ah, I see what you mean. Might suit some others but certainly not me. I'm writing out some xml to be read in another application so must be plain old text file I'm afraid.

Anyway, thanks everyone for you input, great to get such a response. Think we are done here.
Lazarus 3, Linux (and reluctantly Win10/11, OSX Monterey)
My Project - https://github.com/tomboy-notes/tomboy-ng and my github - https://github.com/davidbannon

 

TinyPortal © 2005-2018