Lazarus

Programming => General => Topic started by: user5 on July 13, 2020, 05:51:30 am

Title: Put text file into memory?
Post by: user5 on July 13, 2020, 05:51:30 am
    Is it possible to put a text file like newfile1.txt into memory instead of disk and read and write to it in the normal way (ex: readln(readfile,tempstr);) instead of using an array? I'd prefer not to use a ram disk. I've looked all over the Web but I couldn't find anything concerning this question. By the way, thanks for confirming/answering my previous post titled "Dynamic array question".
Title: Re: Put text file into memory?
Post by: Handoko on July 13, 2020, 06:03:51 am
As far as I know, no.

But the most similar to what you want are TMemoryStream and in-memory database. Not too hard, but you still need some time to learn them:

https://lazarus-ccr.sourceforge.io/docs/rtl/classes/tmemorystream.html
https://wiki.lazarus.freepascal.org/How_to_write_in-memory_database_applications_in_Lazarus/FPC
Title: Re: Put text file into memory?
Post by: cdbc on July 13, 2020, 07:35:15 am
Hi
Somewhat along the lines of this:
Code: Pascal  [Select][+][-]
  1. unit lfm_main;
  2.  
  3. {$mode objfpc}{$H+}
  4.  
  5. interface
  6.  
  7. uses
  8.   Classes, SysUtils, Forms, Controls, Graphics, Dialogs, StdCtrls;
  9.  
  10. type
  11.  
  12.   { TTxtFile }
  13.  
  14.   TTxtFile = class
  15.   private
  16.     fStringList: TStringList;
  17.     fCursor: ptrint;
  18.     function get_EOF: boolean;
  19.   public
  20.     constructor Create(const aTextFilename: string);
  21.     destructor Destroy;
  22.     function Read: string;
  23.     function ReadLn: string;
  24.     procedure Write(const aLine: string);
  25.     procedure WriteLn(const aLine: string);
  26.     property EOF: boolean read get_EOF;
  27.   end;
  28.  
  29.   { TForm1 }
  30.  
  31.   TForm1 = class(TForm)
  32.     Button1: TButton;
  33.     Memo1: TMemo;
  34.     OpenDialog1: TOpenDialog;
  35.     procedure Button1Click(Sender: TObject);
  36.   private
  37.  
  38.   public
  39.  
  40.   end;
  41.  
  42. var
  43.   Form1: TForm1;
  44.  
  45. implementation
  46.  
  47. {$R *.lfm}
  48.  
  49. { TTxtFile }
  50.  
  51. function TTxtFile.get_EOF: boolean;
  52. begin
  53.   Result:= false;
  54.   if fCursor = fStringList.Count-1 then Result:= true;
  55. end;
  56.  
  57. constructor TTxtFile.Create(const aTextFilename: string);
  58. begin
  59.   inherited Create;
  60.   fStringList:= TStringlist.Create;
  61.   fStringList.LoadFromFile(aTextFilename);
  62.   fCursor:= -1; // before first
  63. end;
  64.  
  65. destructor TTxtFile.Destroy;
  66. begin
  67.   FreeAndNil(fStringList);
  68.   fCursor:= -1;
  69.   inherited Destroy;
  70. end;
  71.  
  72. function TTxtFile.Read: string;
  73. begin
  74.   if fCursor >= 0 then
  75.     Result:= fStringList[fCursor];
  76. end;
  77.  
  78. function TTxtFile.ReadLn: string;
  79. begin
  80.   if fCursor >= 0 then begin
  81.     Result:= fStringList[fCursor];
  82.     inc(fCursor)
  83.   end;
  84. end;
  85.  
  86. procedure TTxtFile.Write(const aLine: string);
  87. begin
  88.   fStringList.Insert(fCursor,aLine);
  89. end;
  90.  
  91. procedure TTxtFile.WriteLn(const aLine: string);
  92. begin
  93.   fStringList.Insert(fCursor,aLine);
  94.   inc(fCursor);
  95. end;
  96.  
  97. { TForm1 }
  98.  
  99. procedure TForm1.Button1Click(Sender: TObject);
  100. begin
  101.  
  102. end;
  103.  
  104. end.
  105.  
  106.  
Regards Benny  ;)
Title: Re: Put text file into memory?
Post by: Handoko on July 13, 2020, 08:14:00 am
+1 cdbc
Nice trick.

But I still recommend OP to learn TMemoryStream. Able to use TMemoryStream will open many programming tricks.
Title: Re: Put text file into memory?
Post by: cdbc on July 13, 2020, 08:34:52 am
Hi
Yep, I love streams too  ;)
This one, was just a Quick'n'dirty... Maybe he can use it as a base, to enhance and get on with it  :D
Regards Benny
Title: Re: Put text file into memory?
Post by: PascalDragon on July 13, 2020, 09:19:11 am
    Is it possible to put a text file like newfile1.txt into memory instead of disk and read and write to it in the normal way (ex: readln(readfile,tempstr);) instead of using an array? I'd prefer not to use a ram disk. I've looked all over the Web but I couldn't find anything concerning this question. By the way, thanks for confirming/answering my previous post titled "Dynamic array question".

Once you've read a file into a TMemoryStream using a TFileStream as the others have suggested you can use the unit StreamIO (https://www.freepascal.org/docs-html/current/fcl/streamio/index.html) to create a TextFile based on a stream if you really want to use the Pascal style I/O instead of the Stream API (though I suggest you to learn about the stream API as well).
Title: Re: Put text file into memory?
Post by: Kays on July 13, 2020, 09:49:59 am
Is it possible to put a text file like newfile1.txt into memory instead of disk and read and write to it in the normal way (ex: readln(readfile,tempstr);) instead of using an array? […]
Yes-no:
Title: Re: Put text file into memory?
Post by: user5 on July 13, 2020, 10:23:47 am
    Wow. You guys have given me some cool things to really think about. cdbc, I never would have been able to come up with something as good as the example you provided.
    It also occurred to me that if necessary I could use arrays to store strings and have a simple converter procedure that would translate the custom read and write procedures below to access the strings in the arrays. Each array could simulate a text file. Each array would of course be 2-dimensional with the first array representing a line in the simulated text file and with the second array containing a string in that line. A counter would keep track of what line is current. If I do that then I will post the code.
    In any event, I think that I'm in business. If I have trouble then I may post again.
    Thank you all very much!

    readit(simfile1,tempstr,linecounter); //A line read procedure - simfile1 is an array and readit is a procedure that extracts a line from that array/simulated text file
    writeit(simfile2,tempstr,linecounter); //A line write procedure
Title: Re: Put text file into memory?
Post by: user5 on July 13, 2020, 10:36:26 am
    Correction: I think that I will use a 1-dimensional array instead of a 2-dimensional array with each element of the array containing a string.
Title: Re: Put text file into memory?
Post by: TRon on July 13, 2020, 11:00:16 am
    Correction: I think that I will use a 1-dimensional array instead of a 2-dimensional array with each element of the array containing a string.
Good of you that you were able to correct yourself there....

.. however that brings a question: why (w/sh)ould you want to do that ? Is it for a better understanding on how things work ? Because in case all you really need is quick and easy access to some lines of text that are originally stored inside some text-file, then you could perhaps consider using a TStringList.

https://wiki.freepascal.org/TStringList-TStrings_Tutorial
Title: Re: Put text file into memory?
Post by: user5 on July 13, 2020, 02:37:44 pm
    Below is the code that I came up with to write and read text files to memory. The code is much simpler and less elegant than what was suggested to me but it gets the job done. I hope that I haven't left anything out of the uses clause.  Thanks again for taking the effort and time to help me. All praise to Lazarus.

Code: Pascal  [Select][+][-]
  1. unit Unit1;
  2.  
  3. {$mode objfpc}{$H+}
  4.  
  5. interface
  6.  
  7. uses
  8.   Classes, SysUtils, FileUtil, wincrt, Process, Forms, Controls, Graphics, Dialogs, StdCtrls;
  9.  
  10. type
  11.  grouper = array of string;
  12.  
  13. { TForm1 }
  14.  
  15. TForm1 = class(TForm)
  16.   Button13: TButton;
  17.  
  18.   procedure Button13Click(Sender: TObject); //Writes to array, reads from the array and writes to a text file.
  19.   procedure readline(filename:string; var textstr:string); //Read one line from one of four simulated text files (array).
  20.   procedure writeline(filename:string;textstr:string); //Write one line to one of four simulated text files (array).
  21.  
  22. var
  23.   arraycount1,arraycout2,arraycout3,arraycout4:integer; //Each counter goes with a particular array (simulated text file).
  24.   simtextfile1,simtextfile2,simtextfile3,simtextfile4:grouper; //Four simulated text files.
  25.  
  26. implementation
  27.  
  28. {$R *.lfm}
  29.  
  30. { TForm1 }
  31.  
  32. procedure TForm1.Button13Click(Sender: TObject);
  33. var i:integer;
  34.     filename1,tempstr:string;
  35.     writefile:TextFile;
  36. begin
  37.  setlength(simtextfile1,6);
  38.  arraycount1 := 0; //arraycount1 is associated with simtextfile1 etc.
  39.  filename1 := 'simtextfile1';
  40.  //Write seven lines of text into the array. Each element of the array represents a line of string text.
  41.  writeline(filename1,'Four'); //Write a line using the procedure "writeline".
  42.  writeline(filename1,'score');
  43.  writeline(filename1,'and');
  44.  writeline(filename1,'seven');
  45.  writeline(filename1,'years');
  46.  writeline(filename1,'ago');
  47.  
  48.  //Set the counter back to zero so as to read the first line in the array "simtextfile1"..
  49.  arraycount1 := 0;
  50.  assignfile(writefile,'testfile.txt');
  51.  rewrite(writefile);
  52.  
  53.  //Read six lines from an array (simulated text file) and write those same six lines to a test text file.
  54.  readline(filename1,tempstr);
  55.  writeln(writefile,tempstr);
  56.  readline(filename1,tempstr);
  57.  writeln(writefile,tempstr);
  58.  readline(filename1,tempstr);
  59.  writeln(writefile,tempstr);
  60.  readline(filename1,tempstr);
  61.  writeln(writefile,tempstr);
  62.  readline(filename1,tempstr);
  63.  writeln(writefile,tempstr);
  64.  readline(filename1,tempstr);
  65.  writeln(writefile,tempstr);
  66.  
  67.  closefile(writefile);
  68.  setlength(simtextfile1,0); //Shut down the array and return its memory to the system.
  69.  sound(350);
  70. end;
  71.  
  72.  
  73.  
  74. procedure TForm1.readline(filename:string; var textstr:string);
  75. var tempcount:integer;
  76. begin
  77.  
  78.  if filename = 'simtextfile1' then
  79.   begin
  80.    arraycount1 := arraycount1 + 1;
  81.    tempcount := arraycount1 - 1;
  82.    textstr := simtextfile1[tempcount];
  83.   end;
  84.  
  85.  if filename = 'simtextfile2' then
  86.   begin
  87.    arraycount2 := arraycount2 + 1;
  88.    tempcount := arraycount2 - 1;
  89.    textstr := simtextfile2[tempcount];
  90.   end;
  91.  
  92.  if filename = 'simtextfile3' then
  93.   begin
  94.    arraycount3 := arraycount3 + 1;
  95.    tempcount := arraycount3 - 1;
  96.    textstr := simtextfile3[tempcount];
  97.   end;
  98.  
  99.  if filename = 'simtextfile4' then
  100.   begin
  101.    arraycount4 := arraycount4 + 1;
  102.    tempcount := arraycount4 - 1;
  103.    textstr := simtextfile4[tempcount];
  104.   end;
  105.  
  106. end;
  107.  
  108. procedure TForm1.writeline(filename:string;textstr:string);
  109. var tempcount:integer;
  110. begin
  111.  
  112.  if filename = 'simtextfile1' then
  113.   begin
  114.    arraycount1 := arraycount1 + 1;
  115.    tempcount := arraycount1 - 1;
  116.    simtextfile1[tempcount] := textstr;
  117.   end;
  118.  
  119.  if filename = 'simtextfile2' then
  120.   begin
  121.    arraycount2 := arraycount2 + 1;
  122.    tempcount := arraycount2 - 1;
  123.    simtextfile2[tempcount] := textstr;
  124.   end;
  125.  
  126.  if filename = 'simtextfile3' then
  127.   begin
  128.    arraycount3 := arraycount3 + 1;
  129.    tempcount := arraycount3 - 1;
  130.    simtextfile3[tempcount] := textstr;
  131.   end;
  132.  
  133.  if filename = 'simtextfile4' then
  134.   begin
  135.    arraycount4 := arraycount4 + 1;
  136.    tempcount := arraycount4 - 1;
  137.    simtextfile4[tempcount] := textstr;
  138.   end;
  139.  
  140.  
  141. end;
  142.  
  143. end.
Title: Re: Put text file into memory?
Post by: Warfley on July 13, 2020, 02:53:02 pm
Take a look at the TextRec (the internal datastructure used for Text files):
Code: Pascal  [Select][+][-]
  1.   TextRec = {$ifdef VER2_6} packed {$endif} Record
  2.     Handle    : THandle;
  3. {$if defined(CPU8) or defined(CPU16)}
  4.     Mode      : Word;
  5. {$else}
  6.     Mode      : longint;
  7. {$endif}
  8.     bufsize   : SizeInt;
  9.     _private  : SizeInt;
  10.     bufpos,
  11.     bufend    : SizeInt;
  12.     bufptr    : ^textbuf;
  13.     openfunc,
  14.     inoutfunc,
  15.     flushfunc,
  16.     closefunc : codepointer;
  17.     UserData  : array[1..32] of byte;
  18.     name      : array[0..textrecnamelength-1] of TFileTextRecChar;
  19.     LineEnd   : TLineEndStr;
  20.     buffer    : textbuf;
  21. {$ifdef FPC_HAS_CPSTRING}
  22.     CodePage  : TSystemCodePage;
  23. {$endif}
  24.   End;    

Here you have the inoutfunc which is a function pointer you can overwrite, which will be called by readline and writeline.
You can "simply" override it with your own functions and you can do with the file what you want. See Crt Unit for an example (assigncrt function)
Title: Re: Put text file into memory?
Post by: user5 on July 13, 2020, 02:55:37 pm
    To answer the last question posed to me about TStringList, I have reasons for wanting to use an array instead of TStringList. One of those reasons is the fact that I have some lengthy, intricate code that uses many readlns and writelns and I need to convert all that into faster statements. Reading and writing to disk is fairly fast but it's not fast enough for my purposes. The procedures readline and writeline in my new code are very similar to readln and writeln so basically all I have to do is substitue readln with readline etc. and this should make the work easier.
Title: Re: Put text file into memory?
Post by: Thaddy on July 13, 2020, 02:58:32 pm
    To answer the last question posed to me about TStringList, I have reasons for wanting to use an array instead of TStringList. One of those reasons is the fact that I have some lengthy, intricate code that uses many readlns and writelns and I need to convert all that into faster statements. Reading and writing to disk is fairly fast but it's not fast enough for my purposes. The procedures readline and writeline in my new code are very similar to readln and writeln so basically all I have to do is substitue readln with readline etc. and this should make the work easier.
In that case use memory mapped files. That will be faster and less risky.
Title: Re: Put text file into memory?
Post by: TRon on July 14, 2020, 04:58:47 am
Take a look at the TextRec (the internal datastructure used for Text files):
Sure. That is how you do it when you know what you are doing  :D

But, for mortals like me that find that a bit (over)complicated, there was the invention of streamIO as suggested by PascalDragon.

I mention it, in case you were not aware.
Title: Re: Put text file into memory?
Post by: TRon on July 14, 2020, 06:49:04 am
    To answer the last question posed to me about TStringList, I have reasons for wanting to use an array instead of TStringList. One of those reasons is the fact that I have some lengthy, intricate code that uses many readlns and writelns and I need to convert all that into faster statements. Reading and writing to disk is fairly fast but it's not fast enough for my purposes. The procedures readline and writeline in my new code are very similar to readln and writeln so basically all I have to do is substitue readln with readline etc. and this should make the work easier.
Then I still don't quite understand why you would do it the way you showed with your solution.

It is far from me to question your code, you should use whatever you are comfortable with but I think it can be done in a better way.

If your code already consist out of many readln's/writeln's then using StreamIO is perhaps the better option (I can show that code as well if you like), but the following is based on your example-code:
Code: Pascal  [Select][+][-]
  1. program bettercameupwith;
  2.  
  3. {$MODE OBJFPC}{$H+}
  4.  
  5. uses
  6.   classes;
  7.  
  8. procedure Button13Click;
  9. var
  10.   sl1 : TStringList;
  11. begin
  12.   sl1 := TStringList.Create;
  13.  
  14.   // "Write" _six_ lines of text into the list. Each element of the list represents a line of string text.
  15.   sl1.Append('Four');
  16.   sl1.Append('score');
  17.   sl1.Append('and');
  18.   sl1.Append('seven');
  19.   sl1.Append('years');
  20.   sl1.Append('ago');
  21.  
  22.   // "Read" six lines from an list (simulated text file) and write those same six lines to a test text file.
  23.   sl1.SaveToFile('testfile.txt');
  24.  
  25.   // Shut down the list and return its memory to the system.
  26.   sl1.Free;
  27.   // sound(350);
  28. end;
  29.  
  30.  
  31. begin
  32.   Button13Click;
  33. end.
  34.  
The code is shorter, does not require readln's and writeln's (other than saving or loading the stream itself from and to disk) and you can make as many StringList's as your project requires 'files in memory'. On top of that a stringlist offers indexed access to the individual strings it contains so that in that respect it can be compared to your use of array's.

PS: method looks a bit different then yours because i'm using FreePascal, not Lazarus, and you should be able to copy-paste the relevant parts into a Lazarus project without issues.
TinyPortal © 2005-2018