Read only memory stream, given start memory pointer and length

[Чебурашка]

Hi everybody,
I have the following situation:

1. there is a packed record type variable populated by some entity

Code: Pascal  [Select][+][-]

1.  
2. type
3.  
4.  TMyRecord = packed record
5.    f1: Int
6.    f2: Int
7.    f3: Int
8.    f4: Single
9.    f5: Single
10.    f6: Single
11.    f7: Single
12.  end
13.  
14. var
15.   x: TMyRecord
16.  
17.  

2. This variable is passed to a procedure by means of two parameters: StartPointer: Pointer, MemoryLength: Int64

Code: Pascal  [Select][+][-]

1.  
2. procedure MakeSomething(StartPointer: Pointer, MemoryLength: Int64);
3.  
4. [...]
5.  
6. MakeSomething(Addr(x), SizeOf(TMyRecord));
7.  
8.  

In this way I get into the procedure with the memory area where the variable x lives.

I would like to use a TStream, because inside the procedure there is an activity that writes to db, that would like to get a TStream as input, something like this:

Code: Pascal  [Select][+][-]

1.  
2. ParamByName('samplingData').LoadFromStream(stream, ftBlob); // <--- it would be nice if stream contains the memory area used by variable x
3.  
4.  

I need to have a sort of TStream able to read from the memory area from StartPointer to StartPointer + MemoryLength - 1 so that no extra memory is allocated because the memory area used by x will exist during the call to the procedure, and the procedure is called billion of billion times per hour.

Is there anything ready or do I need to implement a custom stream??

Thank you very much!

[Zvoni]

I remember doing a very small sample-project (more like proof of concept) with SQLite for storing and retrieving blobs
https://forum.lazarus.freepascal.org/index.php/topic,47482.msg340894.html#msg340894

[BrunoK]

Code: Pascal  [Select][+][-]

1. program Project1;
2.  
3. {$mode objfpc}{$H+}
4.  
5. uses
6.  {$IFDEF UNIX}
7.   cthreads,
8.      {$ENDIF}
9.   Classes { you can add units after this };
10.  
11. type
12.  
13.   TMyRecord = packed record
14.     f1: integer;
15.     f2: integer;
16.     f3: integer;
17.     { etc ... }
18.   end;
19.   PMyRecord = ^TMyRecord;
20.  
21.  
22. procedure MakeSomething(apMyRecord : PMyRecord);
23. var
24.   lSumOfInt : integer;
25. begin
26.   // Do something with the pointed record
27.   lSumOfInt := apMyRecord^.f1 + apMyRecord^.f2 + apMyRecord^.f3;
28. end;
29.  
30. var
31.   vInStream : TMemoryStream; // or another TStream descendent
32.  
33. begin
34.   { Whatever needed to create and load the data to vInStream
35.   ...
36.   }
37.   begin
38.     vInStream.Position := 0; // go to stream begining
39.     repeat
40.       if (vInStream.Position + SizeOf(TMyRecord)) > vInStream.Size then
41.         break;
42.       MakeSomething(PMyRecord(vInStream.Memory + vInStream.Position));
43.       vInStream.Position := vInStream.Position + SizeOf(TMyRecord); // Move forward
44.     until False;
45.   end;
46. end.

[ASerge]

I need to have a sort of TStream able to read from the memory area from StartPointer to StartPointer + MemoryLength - 1 so that no extra memory is allocated because the memory area used by x will exists during the call to the procedure, and the procedure is called billion of billion times per hour.

Code: Pascal  [Select][+][-]

1. type
2.   TStreamOnStaticMemory = class(TCustomMemoryStream)
3.   public
4.     constructor Create(AMemory: Pointer; ASize: SizeInt);
5.   end;
6.  
7. constructor TStreamOnStaticMemory.Create(AMemory: Pointer; ASize: SizeInt);
8. begin
9.   inherited Create;
10.   SetPointer(AMemory, ASize);
11. end;

[cdbc]

Hi
You could do a sliding window technique, of sorts.
TMemorystream exposes a property "Memory", it's a pointer.
PMyRecord = ^TMyRecord;
Calculate the offset and place PMyRecord there, and read the values you need.
Regards Benny

[Thaddy]

That is called a windowed stream( nothing to do with windows). There are many implementations for that. It is a very basic concept.

[Чебурашка]

Thank you all very much for the help.

Regards,
Tomaso

[Чебурашка]

I need to have a sort of TStream able to read from the memory area from StartPointer to StartPointer + MemoryLength - 1 so that no extra memory is allocated because the memory area used by x will exists during the call to the procedure, and the procedure is called billion of billion times per hour.

Code: Pascal  [Select][+][-]

1. type
2.   TStreamOnStaticMemory = class(TCustomMemoryStream)
3.   public
4.     constructor Create(AMemory: Pointer; ASize: SizeInt);
5.   end;
6.  
7. constructor TStreamOnStaticMemory.Create(AMemory: Pointer; ASize: SizeInt);
8. begin
9.   inherited Create;
10.   SetPointer(AMemory, ASize);
11. end;

Thank you ASerge.

So the answer is: there is nothing ready, I must subclass on my own.

Despite no already available stream class type exists in the standard library, the class you suggested is exactly what I was looking for.

[Чебурашка]

Have looked at TmemoryStreamEx ?

Where can I find it?

[marcov]

I can't find it with a quick grep.

I do know the principle, a few years ago I derived a version from Indy's TidMemoryStream for this purpose (pretty much like Serge's). 

[dsiders]

I need to have a sort of TStream able to read from the memory area from StartPointer to StartPointer + MemoryLength - 1 so that no extra memory is allocated because the memory area used by x will exists during the call to the procedure, and the procedure is called billion of billion times per hour.

Code: Pascal  [Select][+][-]

1. type
2.   TStreamOnStaticMemory = class(TCustomMemoryStream)
3.   public
4.     constructor Create(AMemory: Pointer; ASize: SizeInt);
5.   end;
6.  
7. constructor TStreamOnStaticMemory.Create(AMemory: Pointer; ASize: SizeInt);
8. begin
9.   inherited Create;
10.   SetPointer(AMemory, ASize);
11. end;

Thank you ASerge.

So the answer is: there is nothing ready, I must subclass on my own.

Despite no already available stream class type exists in the standard library, the class you suggested is exactly what I was looking for.

Perhaps this will help:

TWindowedStream in fpc/3.2.2/source/packages/fcl-base/src/streamex.pp

[marcov]

I don't know but that kind of defeats what the memory stream is for I guess but who am i...

In Indy a lot of streams are used as arguments. (buffer to send etc) If you have some buffer you would have to write it to a tstream first, resulting in bufferbloat. This way circumvents that.

But the Indy situation is IMHO not ideal, but I guess due to past .NET experiments.