Recent

Author Topic: Put text file into memory?  (Read 2561 times)

user5

  • Sr. Member
  • ****
  • Posts: 357
Put text file into memory?
« on: July 13, 2020, 05:51:30 am »
    Is it possible to put a text file like newfile1.txt into memory instead of disk and read and write to it in the normal way (ex: readln(readfile,tempstr);) instead of using an array? I'd prefer not to use a ram disk. I've looked all over the Web but I couldn't find anything concerning this question. By the way, thanks for confirming/answering my previous post titled "Dynamic array question".

Handoko

  • Hero Member
  • *****
  • Posts: 5131
  • My goal: build my own game engine using Lazarus
Re: Put text file into memory?
« Reply #1 on: July 13, 2020, 06:03:51 am »
As far as I know, no.

But the most similar to what you want are TMemoryStream and in-memory database. Not too hard, but you still need some time to learn them:

https://lazarus-ccr.sourceforge.io/docs/rtl/classes/tmemorystream.html
https://wiki.lazarus.freepascal.org/How_to_write_in-memory_database_applications_in_Lazarus/FPC

cdbc

  • Hero Member
  • *****
  • Posts: 1026
    • http://www.cdbc.dk
Re: Put text file into memory?
« Reply #2 on: July 13, 2020, 07:35:15 am »
Hi
Somewhat along the lines of this:
Code: Pascal  [Select][+][-]
  1. unit lfm_main;
  2.  
  3. {$mode objfpc}{$H+}
  4.  
  5. interface
  6.  
  7. uses
  8.   Classes, SysUtils, Forms, Controls, Graphics, Dialogs, StdCtrls;
  9.  
  10. type
  11.  
  12.   { TTxtFile }
  13.  
  14.   TTxtFile = class
  15.   private
  16.     fStringList: TStringList;
  17.     fCursor: ptrint;
  18.     function get_EOF: boolean;
  19.   public
  20.     constructor Create(const aTextFilename: string);
  21.     destructor Destroy;
  22.     function Read: string;
  23.     function ReadLn: string;
  24.     procedure Write(const aLine: string);
  25.     procedure WriteLn(const aLine: string);
  26.     property EOF: boolean read get_EOF;
  27.   end;
  28.  
  29.   { TForm1 }
  30.  
  31.   TForm1 = class(TForm)
  32.     Button1: TButton;
  33.     Memo1: TMemo;
  34.     OpenDialog1: TOpenDialog;
  35.     procedure Button1Click(Sender: TObject);
  36.   private
  37.  
  38.   public
  39.  
  40.   end;
  41.  
  42. var
  43.   Form1: TForm1;
  44.  
  45. implementation
  46.  
  47. {$R *.lfm}
  48.  
  49. { TTxtFile }
  50.  
  51. function TTxtFile.get_EOF: boolean;
  52. begin
  53.   Result:= false;
  54.   if fCursor = fStringList.Count-1 then Result:= true;
  55. end;
  56.  
  57. constructor TTxtFile.Create(const aTextFilename: string);
  58. begin
  59.   inherited Create;
  60.   fStringList:= TStringlist.Create;
  61.   fStringList.LoadFromFile(aTextFilename);
  62.   fCursor:= -1; // before first
  63. end;
  64.  
  65. destructor TTxtFile.Destroy;
  66. begin
  67.   FreeAndNil(fStringList);
  68.   fCursor:= -1;
  69.   inherited Destroy;
  70. end;
  71.  
  72. function TTxtFile.Read: string;
  73. begin
  74.   if fCursor >= 0 then
  75.     Result:= fStringList[fCursor];
  76. end;
  77.  
  78. function TTxtFile.ReadLn: string;
  79. begin
  80.   if fCursor >= 0 then begin
  81.     Result:= fStringList[fCursor];
  82.     inc(fCursor)
  83.   end;
  84. end;
  85.  
  86. procedure TTxtFile.Write(const aLine: string);
  87. begin
  88.   fStringList.Insert(fCursor,aLine);
  89. end;
  90.  
  91. procedure TTxtFile.WriteLn(const aLine: string);
  92. begin
  93.   fStringList.Insert(fCursor,aLine);
  94.   inc(fCursor);
  95. end;
  96.  
  97. { TForm1 }
  98.  
  99. procedure TForm1.Button1Click(Sender: TObject);
  100. begin
  101.  
  102. end;
  103.  
  104. end.
  105.  
  106.  
Regards Benny  ;)
If it ain't broke, don't fix it ;)
PCLinuxOS(rolling release) 64bit -> KDE5 -> FPC 3.2.2 -> Lazarus 2.2.6 up until Jan 2024 from then on it's: KDE5/QT5 -> FPC 3.3.1 -> Lazarus 3.0

Handoko

  • Hero Member
  • *****
  • Posts: 5131
  • My goal: build my own game engine using Lazarus
Re: Put text file into memory?
« Reply #3 on: July 13, 2020, 08:14:00 am »
+1 cdbc
Nice trick.

But I still recommend OP to learn TMemoryStream. Able to use TMemoryStream will open many programming tricks.

cdbc

  • Hero Member
  • *****
  • Posts: 1026
    • http://www.cdbc.dk
Re: Put text file into memory?
« Reply #4 on: July 13, 2020, 08:34:52 am »
Hi
Yep, I love streams too  ;)
This one, was just a Quick'n'dirty... Maybe he can use it as a base, to enhance and get on with it  :D
Regards Benny
If it ain't broke, don't fix it ;)
PCLinuxOS(rolling release) 64bit -> KDE5 -> FPC 3.2.2 -> Lazarus 2.2.6 up until Jan 2024 from then on it's: KDE5/QT5 -> FPC 3.3.1 -> Lazarus 3.0

PascalDragon

  • Hero Member
  • *****
  • Posts: 5446
  • Compiler Developer
Re: Put text file into memory?
« Reply #5 on: July 13, 2020, 09:19:11 am »
    Is it possible to put a text file like newfile1.txt into memory instead of disk and read and write to it in the normal way (ex: readln(readfile,tempstr);) instead of using an array? I'd prefer not to use a ram disk. I've looked all over the Web but I couldn't find anything concerning this question. By the way, thanks for confirming/answering my previous post titled "Dynamic array question".

Once you've read a file into a TMemoryStream using a TFileStream as the others have suggested you can use the unit StreamIO to create a TextFile based on a stream if you really want to use the Pascal style I/O instead of the Stream API (though I suggest you to learn about the stream API as well).

Kays

  • Hero Member
  • *****
  • Posts: 569
  • Whasup!?
    • KaiBurghardt.de
Re: Put text file into memory?
« Reply #6 on: July 13, 2020, 09:49:59 am »
Is it possible to put a text file like newfile1.txt into memory instead of disk and read and write to it in the normal way (ex: readln(readfile,tempstr);) instead of using an array? […]
Yes-no:
  • Simply do not precede your rewrite call by any assign will do the trick. The GPC (GNU Pascal Compiler) supports this, but FPC does not. The actual storage is implementation-specific, but it’ll probably stay in RAM, I guess. On my Ubuntu system it used the systemd‑path temporary‑large directory though (i. e. /var/tmp).
  • With FPC, however, you may use sysUtils.getTempDir/sysUtils.getTempFileName: On a properly configured Unix-like system this’ll give you something like /tmp/TMP00000.tmp, which resides in a tmpfs.
« Last Edit: July 13, 2020, 10:00:37 am by Kays »
Yours Sincerely
Kai Burghardt

user5

  • Sr. Member
  • ****
  • Posts: 357
Re: Put text file into memory?
« Reply #7 on: July 13, 2020, 10:23:47 am »
    Wow. You guys have given me some cool things to really think about. cdbc, I never would have been able to come up with something as good as the example you provided.
    It also occurred to me that if necessary I could use arrays to store strings and have a simple converter procedure that would translate the custom read and write procedures below to access the strings in the arrays. Each array could simulate a text file. Each array would of course be 2-dimensional with the first array representing a line in the simulated text file and with the second array containing a string in that line. A counter would keep track of what line is current. If I do that then I will post the code.
    In any event, I think that I'm in business. If I have trouble then I may post again.
    Thank you all very much!

    readit(simfile1,tempstr,linecounter); //A line read procedure - simfile1 is an array and readit is a procedure that extracts a line from that array/simulated text file
    writeit(simfile2,tempstr,linecounter); //A line write procedure

user5

  • Sr. Member
  • ****
  • Posts: 357
Re: Put text file into memory?
« Reply #8 on: July 13, 2020, 10:36:26 am »
    Correction: I think that I will use a 1-dimensional array instead of a 2-dimensional array with each element of the array containing a string.

TRon

  • Hero Member
  • *****
  • Posts: 2435
Re: Put text file into memory?
« Reply #9 on: July 13, 2020, 11:00:16 am »
    Correction: I think that I will use a 1-dimensional array instead of a 2-dimensional array with each element of the array containing a string.
Good of you that you were able to correct yourself there....

.. however that brings a question: why (w/sh)ould you want to do that ? Is it for a better understanding on how things work ? Because in case all you really need is quick and easy access to some lines of text that are originally stored inside some text-file, then you could perhaps consider using a TStringList.

https://wiki.freepascal.org/TStringList-TStrings_Tutorial
« Last Edit: July 13, 2020, 11:02:15 am by TRon »

user5

  • Sr. Member
  • ****
  • Posts: 357
Re: Put text file into memory?
« Reply #10 on: July 13, 2020, 02:37:44 pm »
    Below is the code that I came up with to write and read text files to memory. The code is much simpler and less elegant than what was suggested to me but it gets the job done. I hope that I haven't left anything out of the uses clause.  Thanks again for taking the effort and time to help me. All praise to Lazarus.

Code: Pascal  [Select][+][-]
  1. unit Unit1;
  2.  
  3. {$mode objfpc}{$H+}
  4.  
  5. interface
  6.  
  7. uses
  8.   Classes, SysUtils, FileUtil, wincrt, Process, Forms, Controls, Graphics, Dialogs, StdCtrls;
  9.  
  10. type
  11.  grouper = array of string;
  12.  
  13. { TForm1 }
  14.  
  15. TForm1 = class(TForm)
  16.   Button13: TButton;
  17.  
  18.   procedure Button13Click(Sender: TObject); //Writes to array, reads from the array and writes to a text file.
  19.   procedure readline(filename:string; var textstr:string); //Read one line from one of four simulated text files (array).
  20.   procedure writeline(filename:string;textstr:string); //Write one line to one of four simulated text files (array).
  21.  
  22. var
  23.   arraycount1,arraycout2,arraycout3,arraycout4:integer; //Each counter goes with a particular array (simulated text file).
  24.   simtextfile1,simtextfile2,simtextfile3,simtextfile4:grouper; //Four simulated text files.
  25.  
  26. implementation
  27.  
  28. {$R *.lfm}
  29.  
  30. { TForm1 }
  31.  
  32. procedure TForm1.Button13Click(Sender: TObject);
  33. var i:integer;
  34.     filename1,tempstr:string;
  35.     writefile:TextFile;
  36. begin
  37.  setlength(simtextfile1,6);
  38.  arraycount1 := 0; //arraycount1 is associated with simtextfile1 etc.
  39.  filename1 := 'simtextfile1';
  40.  //Write seven lines of text into the array. Each element of the array represents a line of string text.
  41.  writeline(filename1,'Four'); //Write a line using the procedure "writeline".
  42.  writeline(filename1,'score');
  43.  writeline(filename1,'and');
  44.  writeline(filename1,'seven');
  45.  writeline(filename1,'years');
  46.  writeline(filename1,'ago');
  47.  
  48.  //Set the counter back to zero so as to read the first line in the array "simtextfile1"..
  49.  arraycount1 := 0;
  50.  assignfile(writefile,'testfile.txt');
  51.  rewrite(writefile);
  52.  
  53.  //Read six lines from an array (simulated text file) and write those same six lines to a test text file.
  54.  readline(filename1,tempstr);
  55.  writeln(writefile,tempstr);
  56.  readline(filename1,tempstr);
  57.  writeln(writefile,tempstr);
  58.  readline(filename1,tempstr);
  59.  writeln(writefile,tempstr);
  60.  readline(filename1,tempstr);
  61.  writeln(writefile,tempstr);
  62.  readline(filename1,tempstr);
  63.  writeln(writefile,tempstr);
  64.  readline(filename1,tempstr);
  65.  writeln(writefile,tempstr);
  66.  
  67.  closefile(writefile);
  68.  setlength(simtextfile1,0); //Shut down the array and return its memory to the system.
  69.  sound(350);
  70. end;
  71.  
  72.  
  73.  
  74. procedure TForm1.readline(filename:string; var textstr:string);
  75. var tempcount:integer;
  76. begin
  77.  
  78.  if filename = 'simtextfile1' then
  79.   begin
  80.    arraycount1 := arraycount1 + 1;
  81.    tempcount := arraycount1 - 1;
  82.    textstr := simtextfile1[tempcount];
  83.   end;
  84.  
  85.  if filename = 'simtextfile2' then
  86.   begin
  87.    arraycount2 := arraycount2 + 1;
  88.    tempcount := arraycount2 - 1;
  89.    textstr := simtextfile2[tempcount];
  90.   end;
  91.  
  92.  if filename = 'simtextfile3' then
  93.   begin
  94.    arraycount3 := arraycount3 + 1;
  95.    tempcount := arraycount3 - 1;
  96.    textstr := simtextfile3[tempcount];
  97.   end;
  98.  
  99.  if filename = 'simtextfile4' then
  100.   begin
  101.    arraycount4 := arraycount4 + 1;
  102.    tempcount := arraycount4 - 1;
  103.    textstr := simtextfile4[tempcount];
  104.   end;
  105.  
  106. end;
  107.  
  108. procedure TForm1.writeline(filename:string;textstr:string);
  109. var tempcount:integer;
  110. begin
  111.  
  112.  if filename = 'simtextfile1' then
  113.   begin
  114.    arraycount1 := arraycount1 + 1;
  115.    tempcount := arraycount1 - 1;
  116.    simtextfile1[tempcount] := textstr;
  117.   end;
  118.  
  119.  if filename = 'simtextfile2' then
  120.   begin
  121.    arraycount2 := arraycount2 + 1;
  122.    tempcount := arraycount2 - 1;
  123.    simtextfile2[tempcount] := textstr;
  124.   end;
  125.  
  126.  if filename = 'simtextfile3' then
  127.   begin
  128.    arraycount3 := arraycount3 + 1;
  129.    tempcount := arraycount3 - 1;
  130.    simtextfile3[tempcount] := textstr;
  131.   end;
  132.  
  133.  if filename = 'simtextfile4' then
  134.   begin
  135.    arraycount4 := arraycount4 + 1;
  136.    tempcount := arraycount4 - 1;
  137.    simtextfile4[tempcount] := textstr;
  138.   end;
  139.  
  140.  
  141. end;
  142.  
  143. end.

Warfley

  • Hero Member
  • *****
  • Posts: 1499
Re: Put text file into memory?
« Reply #11 on: July 13, 2020, 02:53:02 pm »
Take a look at the TextRec (the internal datastructure used for Text files):
Code: Pascal  [Select][+][-]
  1.   TextRec = {$ifdef VER2_6} packed {$endif} Record
  2.     Handle    : THandle;
  3. {$if defined(CPU8) or defined(CPU16)}
  4.     Mode      : Word;
  5. {$else}
  6.     Mode      : longint;
  7. {$endif}
  8.     bufsize   : SizeInt;
  9.     _private  : SizeInt;
  10.     bufpos,
  11.     bufend    : SizeInt;
  12.     bufptr    : ^textbuf;
  13.     openfunc,
  14.     inoutfunc,
  15.     flushfunc,
  16.     closefunc : codepointer;
  17.     UserData  : array[1..32] of byte;
  18.     name      : array[0..textrecnamelength-1] of TFileTextRecChar;
  19.     LineEnd   : TLineEndStr;
  20.     buffer    : textbuf;
  21. {$ifdef FPC_HAS_CPSTRING}
  22.     CodePage  : TSystemCodePage;
  23. {$endif}
  24.   End;    

Here you have the inoutfunc which is a function pointer you can overwrite, which will be called by readline and writeline.
You can "simply" override it with your own functions and you can do with the file what you want. See Crt Unit for an example (assigncrt function)

user5

  • Sr. Member
  • ****
  • Posts: 357
Re: Put text file into memory?
« Reply #12 on: July 13, 2020, 02:55:37 pm »
    To answer the last question posed to me about TStringList, I have reasons for wanting to use an array instead of TStringList. One of those reasons is the fact that I have some lengthy, intricate code that uses many readlns and writelns and I need to convert all that into faster statements. Reading and writing to disk is fairly fast but it's not fast enough for my purposes. The procedures readline and writeline in my new code are very similar to readln and writeln so basically all I have to do is substitue readln with readline etc. and this should make the work easier.

Thaddy

  • Hero Member
  • *****
  • Posts: 14205
  • Probably until I exterminate Putin.
Re: Put text file into memory?
« Reply #13 on: July 13, 2020, 02:58:32 pm »
    To answer the last question posed to me about TStringList, I have reasons for wanting to use an array instead of TStringList. One of those reasons is the fact that I have some lengthy, intricate code that uses many readlns and writelns and I need to convert all that into faster statements. Reading and writing to disk is fairly fast but it's not fast enough for my purposes. The procedures readline and writeline in my new code are very similar to readln and writeln so basically all I have to do is substitue readln with readline etc. and this should make the work easier.
In that case use memory mapped files. That will be faster and less risky.
Specialize a type, not a var.

TRon

  • Hero Member
  • *****
  • Posts: 2435
Re: Put text file into memory?
« Reply #14 on: July 14, 2020, 04:58:47 am »
Take a look at the TextRec (the internal datastructure used for Text files):
Sure. That is how you do it when you know what you are doing  :D

But, for mortals like me that find that a bit (over)complicated, there was the invention of streamIO as suggested by PascalDragon.

I mention it, in case you were not aware.

 

TinyPortal © 2005-2018