Recent

Author Topic: beginner question about streams  (Read 245 times)

TNoob

  • New member
  • *
  • Posts: 9
beginner question about streams
« on: August 27, 2019, 12:57:14 pm »
I'm having a headache trying to understand how all these streams are supposed to be used, even for a simple task like line oriented text read/write. I did solve the problem at hand but it feels clunky and probably wrong.
If there is a comprehensive 'streams in free pascal' tutorial somewhere please direct me to the proper RTFM department, I didn't find it myself.

What I'm doing right now: I want to write procedures/methods that take a TStream (any of them) and read/write text from/to it, be it a TFileStream or TStringStream or whatever (in practice, only those two, but I'm interested in the general case for learning). Text I/O is mostly line oriented (but I might want to write a line 'in parts' as opposed to building up the complete line and writing it at once, or read n characters ahead). Said procedures/methods are recursive (they deal with a hierarchical structure stored as text and expect it to be correct, no sophisticated parsing nor error detection is done apart from checking EOF). One is called 'Load' and one 'Save'. I gave up on 'Load' taking a TStream to avoid wrapping it in a TTextReader (but which one?) on every recursive call, so it's
Code: Pascal  [Select]
  1. Load(source: TTextReader)
and I call it with
Code: Pascal  [Select]
  1. Load(TStreamReader.Create(TFileStream.Create(filename,fmOpenRead)))
(but it could be a string in memory instead of a file). Writing is even more of a kludge:
Code: Pascal  [Select]
  1. Save(dest: TStream)
does take a TStream as I'd like, but I resorted to writing with
Code: Pascal  [Select]
  1. dest.Write(s[1],Length(s));
. Again, it works but feels wrong. I know I could work around the problem altogether, for instance always writing the whole text to a string or TStringList and then dumping it to file if it needs be, or testing 'if mystream is TFileStream then...else...', but I'm looking for the clean, 'right way' to do it. I looked at the docs, but they don't say much beyond repeating the inheritance of the classes and the signatures of the methods with a scanty description, in the end I have more questions than answers.
Am I correct to assume Free Pascal classes mirror Delphi ones, to the point its docs also apply here?
Why is there a TTextReader but no TTextWriter?
What is the difference between using TStringReader and instead TStreamReader.Create(TStringStream(...))?
What is the difference between a TStringStream and a TMemoryStream?
What happens if my text is unicode?
I see there is a StreamIO unit which I discovered too late and haven't tried yet. It seems like the way to go, is it?
If so, how does it handle text which might be utf8? (let's say I don't care all the other encodings beyond ascii and utf8) Or should I use TCharEncStream? The two of them together somehow?
Also I'm not sure I understand how I'm supposed to use the StreamIO (ok, this will perhaps become clear once I actually try it in code): it seems to me I would declare a 'dummy' file variable, call StreamIO's 'Assign' on it, then read/write the file variable knowing that actually StreamIO will take over and do its thing, is that correct?
Thanks to all and sorry for the long post.

Leledumbo

  • Hero Member
  • *****
  • Posts: 8108
  • Programming + Glam Metal + Tae Kwon Do = Me
Re: beginner question about streams
« Reply #1 on: August 27, 2019, 09:41:27 pm »
If there is a comprehensive 'streams in free pascal' tutorial somewhere please direct me to the proper RTFM department, I didn't find it myself.
FPC streams are Delphi compatible, so you can use Delphi materials as well. Not that there are many of them, since it's not really a big topic. Here's a simple yet to the point explanation: https://www.thoughtco.com/tstream-class-in-delphi-4077896
What I'm doing right now: I want to write procedures/methods that take a TStream (any of them) and read/write text from/to it, be it a TFileStream or TStringStream or whatever (in practice, only those two, but I'm interested in the general case for learning).
...
Again, it works but feels wrong. I know I could work around the problem altogether, for instance always writing the whole text to a string or TStringList and then dumping it to file if it needs be, or testing 'if mystream is TFileStream then...else...'
I suggest creating the TTextReader inside Load implementation instead of as parameter. Normally for this kind of API, you don't want to expose how the loading process is supposed to be done, you just know that you can load something from a given TStream. The same principle applies to Save. Other than that, unless the TStream descendant you expect to work with isn't capable of either Read (e.g. TWriteBufStream) or Write (e.g. TReadBufStream), you shouldn't do any runtime type check. After all, if you want to deal with TStream in general and not any specific descendant of it, just assume it's a TStream. i.e. just call TStream methods and properties, neither of its descendants.
Am I correct to assume Free Pascal classes mirror Delphi ones, to the point its docs also apply here?
As I said above, yes. Mostly, at least.
Why is there a TTextReader but no TTextWriter?
Because it's not as difficult or practical? Line oriented reading requires some cumbersome code to detect newlines (which could be in multiple formats) and return part of the long string until that newline. For writing, however, it's straightforward. Just write something then write a newline. Done. No cumbersome code required.
What is the difference between using TStringReader and instead TStreamReader.Create(TStringStream(...))?
The former reads directly from a string, the latter reads from a stream made from a string. The former should be a tad faster.
What is the difference between a TStringStream and a TMemoryStream?
TStringStream is a stream made from (ansi)string, while TMemoryStream is made from memory... I mean raw bytes of data in memory, it doesn't know nor care in what format the data is, you are fully responsible for it.
What happens if my text is unicode?
Unclear what you ask this in connection to.
I see there is a StreamIO unit which I discovered too late and haven't tried yet. It seems like the way to go, is it?
Arguable. As written in the overview, the unit is meant to use compiler built-in Read and Write procedures (that you normally begin Pascal programming with) on streams. Everything else is built upon those two procedures. e.g. if you have an old program that reads from standard input and writes to standard output but you want to easily change the program to read from network stream and write to text file, you can simply AssignStream a TNetworkStream (no, it doesn't exist, just to show you a possibility to Input (or StdIn) and a TFileStream to Output (or StdOut) and what you want is achieved automagically.
If so, how does it handle text which might be utf8? (let's say I don't care all the other encodings beyond ascii and utf8) Or should I use TCharEncStream? The two of them together somehow?
Read is incapable of reading types outside of Char, Integer, Real and String. So don't use StreamIO for this purpose.
Also I'm not sure I understand how I'm supposed to use the StreamIO (ok, this will perhaps become clear once I actually try it in code): it seems to me I would declare a 'dummy' file variable, call StreamIO's 'Assign' on it, then read/write the file variable knowing that actually StreamIO will take over and do its thing, is that correct?
I see this unit lacks example in its docs though there's a demo in trunk repo.
That (text)file ain't dummy, it will be the textfile representation of TStream instance you pass to AssignStream that you can pass as first argument to Read(Ln) or Write(Ln), any rule applies to both procedures also apply. e.g.: not passing the textfile variable means each procedure will read from corresponding standard stream, you can use Write's output formatting capabilities.

TNoob

  • New member
  • *
  • Posts: 9
Re: beginner question about streams
« Reply #2 on: August 28, 2019, 12:02:11 am »
Thanks for reply, I'll update 'Load' to take a plain TStream and then pass the reader around, and leave StringIO aside. About character encodings, I guess I'll look into TCharEncStream if I actually run into problems (meanwhile I wrote some more code and I introduced an 'object freed prematurely' kind of bug which now gets precedence).
« Last Edit: August 28, 2019, 01:27:28 am by TNoob »