Recent

Author Topic: About Streams  (Read 3638 times)

yamer

  • Jr. Member
  • **
  • Posts: 87
About Streams
« on: November 21, 2014, 11:34:16 am »
Hi all,

I have 2 problems with streams.

1/ I have a memory block (allocated by GetMem) that contains data. I want to read this memory block like if it is a stream. I know that I can use a TMemoryStream and fill it with the datas. This operation would be unnecessary if there was a class that implements a stream (read only) that takes is content from a pointer and a size. Is there something like that in the RTL ?

2/ I have implemented a function to read TGA file from a stream. When I read from a TFileStream it's very slow. Example :
  • Reading from TFileStream : 1171 ms
  • Filling a TMemoryStream and Reading from it : 57ms
I think the problem is that TFileStream uses no buffer. Is there somewhere in the RTL a buffered file stream or  a class that adds a buffer to another stream (like BufferedInputStream in java) ?

Thanks

PS : For both problems, if there is no existing solutions in the RTL, I can try to implement one.

howardpc

  • Hero Member
  • *****
  • Posts: 3182
Re: About Streams
« Reply #1 on: November 21, 2014, 11:39:30 am »
Re your second point:
objects.pp has TBufStream, for which the docs say

"Bufstream implements a buffered file stream. That is, all data written to the stream is written to memory first. Only when the buffer is full, or on explicit request, the data is written to disk.

Also, when reading from the stream, first the buffer is checked if there is any unread data in it. If so, this is read first. If not the buffer is filled again, and then the data is read from the buffer.

The size of the buffer is fixed and is set when constructing the file.

This is useful if you need heavy throughput for your stream, because it speeds up operations."

yamer

  • Jr. Member
  • **
  • Posts: 87
Re: About Streams
« Reply #2 on: November 21, 2014, 11:46:51 am »
Exactly what I need. Sadly TBufStream is old style object and can't be mixed with new style objects. However this is an implementation example.

Thanks

wp

  • Hero Member
  • *****
  • Posts: 6354
Re: About Streams
« Reply #3 on: November 21, 2014, 12:10:43 pm »
A newer version of a buffered stream is contained in the fpspreadsheet package (lazarus-ccr, svn trunk). There has been some request to offer it to fpc, but I am not quite sure if it is ready for general usage. Anyway, the unit, fpsstreams, is pretty much self-contained and will certainly speed up your reader - it does a great job with spreadsheet files.
Lazarus trunk / fpc 3.0.4 / all 32-bit on Win-10

Laksen

  • Hero Member
  • *****
  • Posts: 621
    • J-Software
Re: About Streams
« Reply #4 on: November 21, 2014, 03:15:45 pm »
There's TReadBufStream in the bufstream unit.

For the first question you could make your own class that inherits from TCustomMemoryStream. Take a look at the source for TBytesStream and TMemoryStream. Should be very simple to implement.

yamer

  • Jr. Member
  • **
  • Posts: 87
Re: About Streams
« Reply #5 on: November 21, 2014, 03:50:11 pm »
Thanks to all and specially to Laksen.

TReadBufStream and TWriteBufStream are exactly what I am looking for.

And for my first question I have made my own TDataStream class.