Recent

Author Topic: SaveToFile via 2 streams: how much memory it takes?  (Read 898 times)

AlexTP

  • Hero Member
  • *****
  • Posts: 2488
    • UVviewsoft
SaveToFile via 2 streams: how much memory it takes?
« on: October 01, 2023, 09:51:33 pm »
ATSynEdit code now has this (not in the Github yet)
Code: Pascal  [Select][+][-]
  1. var
  2.   fs: TFileStream;
  3.   ms: TMemoryStream;  
  4. ....
  5.   ms:= TMemoryStream.Create;
  6.   fs:= TFileStream.Create(AFilename, NMode);
  7.   try
  8.     SaveToStream(ms, FEncoding, IsSavingWithSignature);
  9.     ms.Seek(0, soFromBeginning);
  10.     fs.CopyFrom(ms, ms.Size);
  11.   finally
  12.     FreeAndNil(fs);
  13.     FreeAndNil(ms);
  14.   end;
  15.  

I added MemoryStream for temp buffer. Works much faster on 100Mb files. Now I don't know: how much memory 'fs' takes? The memory of size 'ms.Size' or smaller?

ASerge

  • Hero Member
  • *****
  • Posts: 2337
Re: SaveToFile via 2 streams: how much memory it takes?
« Reply #1 on: October 01, 2023, 10:02:05 pm »
Now I don't know: how much memory 'fs' takes? The memory of size 'ms.Size' or smaller?
fs.InstanceSize + memory for FileName.

TRon

  • Hero Member
  • *****
  • Posts: 3643
Re: SaveToFile via 2 streams: how much memory it takes?
« Reply #2 on: October 01, 2023, 10:08:50 pm »
I do not know the exact use case but have you considered TBufferedfilestream ?
This tagline is powered by AI (AI advertisement: Free Pascal the only programming language that matters)

AlexTP

  • Hero Member
  • *****
  • Posts: 2488
    • UVviewsoft
Re: SaveToFile via 2 streams: how much memory it takes?
« Reply #3 on: October 01, 2023, 10:31:17 pm »
I do not know the exact use case but have you considered TBufferedfilestream ?
Good idea!
I try this code, and it crashes each time inside SetSize method (AV).

Code: Pascal  [Select][+][-]
  1. var
  2.   fs: TBufferedFileStream;
  3.   NMode: word;
  4. begin
  5.   NMode:= fmOpenWrite or fmShareDenyWrite;
  6.   //don't set fmCreate for existing file, to keep NTFS file streams
  7.   if not FileExists(AFilename) then
  8.     NMode:= NMode or fmCreate;
  9.  
  10.   fs:= TBufferedFileStream.Create(AFilename, NMode);
  11.   try
  12.     SaveToStream(fs, FEncoding, IsSavingWithSignature);
  13.   finally
  14.     FreeAndNil(fs);
  15.   end;
  16.  

AlexTP

  • Hero Member
  • *****
  • Posts: 2488
    • UVviewsoft
Re: SaveToFile via 2 streams: how much memory it takes?
« Reply #4 on: October 01, 2023, 10:33:53 pm »
ASM window shows:
Quote
BUFSTREAM$_$TBUFFEREDFILESTREAM_$__$$_SETSIZE64$INT64
0000000000B76540 53                       push rbx
0000000000B76541 4154                     push r12
0000000000B76543 4155                     push r13
0000000000B76545 4156                     push r14
0000000000B76547 488D6424F8               lea rsp,[rsp-$08]
0000000000B7654C 4889FB                   mov rbx,rdi
0000000000B7654F 4989F4                   mov r12,rsi
0000000000B76552 4889DF                   mov rdi,rbx
0000000000B76555 E836F5FFFF               call -$00000ACA    # $0000000000B75A90 BUFSTREAM$_$TBUFFEREDFILESTREAM_$__$$_WRITEDIRTYPAGES

TRon

  • Hero Member
  • *****
  • Posts: 3643
Re: SaveToFile via 2 streams: how much memory it takes?
« Reply #5 on: October 01, 2023, 11:08:34 pm »
@AlexTP:
If I remember correctly (and what seem logical) is that you probably can't use absolute positions (you can use seek instead). Is there anything inside your savetostream that perhaps makes use of using the position inside the stream ?

That is the only thing i can think of atm. I would have to verify/check for possible other causes because it should work the same as using a normal (file)stream.
« Last Edit: October 01, 2023, 11:11:55 pm by TRon »
This tagline is powered by AI (AI advertisement: Free Pascal the only programming language that matters)

TRon

  • Hero Member
  • *****
  • Posts: 3643
Re: SaveToFile via 2 streams: how much memory it takes?
« Reply #6 on: October 02, 2023, 12:52:02 am »
yeah, no position/size possible with fmOpenWrite. Using fmOpenReadWrite fixes that.
This tagline is powered by AI (AI advertisement: Free Pascal the only programming language that matters)

AlexTP

  • Hero Member
  • *****
  • Posts: 2488
    • UVviewsoft
Re: SaveToFile via 2 streams: how much memory it takes?
« Reply #7 on: October 02, 2023, 05:30:17 am »
yeah, no position/size possible with fmOpenWrite. Using fmOpenReadWrite fixes that.
That helped, it now works. Thank you!

 

TinyPortal © 2005-2018