Recent

Author Topic: Text file problem  (Read 5054 times)

Thaddy

  • Hero Member
  • *****
  • Posts: 16580
  • Kallstadt seems a good place to evict Trump to.
Re: Text file problem
« Reply #15 on: November 25, 2020, 06:26:32 am »
There is also some Pascal code around. I seem to remember Bart? has some code.
There is also Pascal code for whoislocking.
I used it in the past, I guess it is win32 only.
But I am sure they don't want the Trumps back...

PascalDragon

  • Hero Member
  • *****
  • Posts: 5870
  • Compiler Developer
Re: Text file problem
« Reply #16 on: November 25, 2020, 07:25:02 am »
    Everything compiles fine though I had to change assign and close to assignfile and closefile respectively.
That should - almost - never be the case. You do not need xxxFile in most cases, just system and no dependency on sysutils.
But it does not hurt, except larger binaries.

It's necessary to rename calls to Assign to System.Assign or AssignFile inside descendants of TPersistent due to their own Assign method and Close to System.Close or CloseFile inside descendants of TCustomForm due to its own Close method.

Also the *File overloads are part of the ObjPas unit, not SysUtils and are only really forwarders, so that don't hurt that much.

loaded

  • Hero Member
  • *****
  • Posts: 857
Re: Text file problem
« Reply #17 on: November 25, 2020, 07:35:58 am »
A simple solution method I use for problems that may occur in systemic situations that improve after a certain period of time;

Code: Pascal  [Select][+][-]
  1. procedure TForm1.Button1Click(Sender: TObject);
  2. var
  3.   err:Boolean=false;
  4. begin
  5.   repeat
  6.     try
  7.     // Start of Creation, Deletion Operations
  8.     //....................
  9.     //....................
  10.     //....................
  11.     //....................
  12.     //....................
  13.     //....................
  14.     //....................
  15.     //....................
  16.     //End of Creation, Deletion Operations
  17.     err:=false;
  18.     except
  19.       on E : Exception do
  20.       begin
  21.       err:=true;
  22.       caption:=E.ClassName+' error raised, with message : '+E.Message;
  23.       end;
  24.     end;
  25.     Application.ProcessMessages;
  26.   until not err;
  27.   caption:=' Transaction Completed Successfully' ;
  28. end;  
  29.  
Check out  loaded on Strava
https://www.strava.com/athletes/109391137

Thaddy

  • Hero Member
  • *****
  • Posts: 16580
  • Kallstadt seems a good place to evict Trump to.
Re: Text file problem
« Reply #18 on: November 25, 2020, 09:06:09 am »
Since he is using old school IO routines I strongly disagree with using exceptions in this case.
Both code bloat and slow in a loop that can be predicted to fail sometimes.
Although Marco's code is also predictably slow because of his long timeout value, as he wrote himself.
« Last Edit: November 25, 2020, 09:09:30 am by Thaddy »
But I am sure they don't want the Trumps back...

marcov

  • Administrator
  • Hero Member
  • *
  • Posts: 12050
  • FPC developer.
Re: Text file problem
« Reply #19 on: November 25, 2020, 09:10:36 am »
Since he is using old school IO routines I strongly disagree with using exceptions in this case.
Both code bloat and slow in a loop that can be predicted to fail sometimes.
Although Marco's code is also predictably slow because of his long timeout value, as he wrote himself.

In my experience these locks range from 50ms to 500ms. Retrying immediately then only causes busywaiting. This is the reason for the timeouts.

user5

  • Sr. Member
  • ****
  • Posts: 365
Re: Text file problem
« Reply #20 on: November 25, 2020, 08:30:49 pm »
    Thanks winni, for looking for that C code. The code that I got from this thread was a valiant effort and I'm very appreciative but testing showed that they will not work except to tell you if a file is open or not. Like you say, aside from very hard-to-find solutions the only way to delete a text file is to first close it and to close you have to know what TextFile variable opened it.
    I changed my program so that the text files that I had trouble with are created by an external program that is launched by the main program. When the external program closes then the files are freed up. After the files are created then there is only one page in the main program that opens them and it opens them for reading only. The only other thing that is done to them is to delete them.
    I still don't know what caused the text files to have an open status but I think that the error is fixed. If I have any more problems then I'll use try statements to make sure that the files are closed before they are deleted, using the TextFile variables that I know opened them. If a file is open then the try statement will close it. If it's not open then the exception error is handled by the try statement and no harm is done.

dseligo

  • Hero Member
  • *****
  • Posts: 1462
Re: Text file problem
« Reply #21 on: November 27, 2020, 01:38:32 am »
Like you say, aside from very hard-to-find solutions the only way to delete a text file is to first close it and to close you have to know what TextFile variable opened it.
From this I assume you are opening multiple files or using multiple variables for files. This opens possibility that some file is not closed because of an error in your program. I suggest that you log in a file all you opening and closing files along with variable names (and/or maybe unique ID). When error occurs you take that file and look for pairs - an unmatched pair is location of you problem.

Something like this:
Code: Pascal  [Select][+][-]
  1. const LogName='mylog.txt';
  2.  
  3. procedure LogInit;
  4. var fLog:TextFile;
  5. begin
  6.   AssignFile(fLog,LogName);
  7.   Rewrite(fLog);
  8.   CloseFile(fLog);
  9. end;
  10.  
  11. procedure Log(s:String);
  12. var fLog:TextFile;
  13. begin
  14.   AssignFile(fLog,LogName);
  15.   Append(fLog);
  16.   WriteLn(fLog,FormatDateTime('yyyy-mm-dd hh:nn:ss.zzz ',Now),s);
  17.   CloseFile(fLog);
  18. end;
  19.  
  20. procedure MyCreateFile(var f:TextFile; sFileVarName, sFileName:String; ID:Integer);
  21. begin
  22.   AssignFile(f,sFileName);
  23.   Rewrite(f);
  24.   Log('create '+sFileVarName+' '+sFileName+' '+Inttostr(id));
  25. end;
  26.  
  27. procedure MyCloseFile(var f:TextFile; sFileVarName:String; ID:Integer);
  28. begin
  29.   CloseFile(f);
  30.   Log('close '+sFileVarName+' '+Inttostr(id));
  31. end;
  32.  
  33. ...
  34. LogInit; // at start of the program
  35. ....
  36.  
  37. //
  38. MyCreateFile(fTest,'fTest','testfile1.txt',1);
  39.  
  40. ...
  41.  
  42. MyCreateFile(fTest2,'fTest2','testfile2.txt',3);
  43.  
  44. ...
  45.  
  46. MyCloseFile(fTest,'fTest',2);
  47.  
  48. ...
  49.  
  50. MyCloseFile(fTest2,'fTest2',4);
  51.  

If you do this everywhere in your program where you use AssignFile, CloseFile, Rewrite, Reset or Append you should find if and where you left your file open.

user5

  • Sr. Member
  • ****
  • Posts: 365
"Text file problem" update
« Reply #22 on: December 01, 2020, 10:30:06 pm »
    On November 24 I posted a note ("Text file problem") about a problem I was having in which a text file that was legitimately closed by my program ended up being considered still open by Windows. marcov said "This is a known (Windows) issue". This problem was unpredictable and would only happen every once in a while but testing did show some interesting results. Sometimes when I run Windows Explorer I notice that its progress bar runs very, very slowly and never really ends, even going beyond the end of its containment box. Rebooting the computer usually fixes this.
    When the progress bar is behaving like this then the chances of the error occurring are vastly increased. My computer is a Windows 7 machine that was upgraded to Windows 10 and it seems to behave oddly at times. If I use YouTube a lot then the computer crashes, shutting itself off. I changed the power settings but it still happens if I play many YouTube videos.
    I did some research on the slow progress bar. I have been using Task Manager to check processing performance and I ended up changing Windows indexing options but I don't know if this is the solution to the text file problem.
    I did find an interesting Lazarus Forum topic titled "Delphi, Lazarus Pi Performance Benchmark" and I was wondering if it might be possible to programmatically do some kind of performance check before the program creates and uses those text files. My program is currently set up to provide work around options when the error happens but I haven't been able to solve the problem except by shutting the program down and thus freeing up the file/s involved.  I'm really just groping in the dark here.

dseligo

  • Hero Member
  • *****
  • Posts: 1462
Re: "Text file problem" update
« Reply #23 on: December 03, 2020, 01:27:48 am »
My computer is a Windows 7 machine that was upgraded to Windows 10 and it seems to behave oddly at times.
Are you experiencing this "Text file problem" only on this one computer?
If you have other issues with it, maybe it is time to reinstall Windows?

old_DOS_err

  • New member
  • *
  • Posts: 8
Re: Text file problem
« Reply #24 on: February 04, 2025, 07:30:50 pm »
Hi,

Really late entry to this post. But I was having a random problem opening files. Writing an explorer type program, with an added light security feature where part of a file can be replaced (designed for large files, where standard decryption would take too long when accessing the file). As part of the testing, I use a batch file called from within the code (lazy sod) to copy the original test files into the working folder, since once the files are converted, they no longer exist in their original form. I was getting this random problem on the open. I noted that some of these had that annoying block flag Windows sometimes adds to files. I used powershell to clear that (another forum entry, where was the internet when I was programming in the 80s). That wasn't the problem.

I am using stream, yes I know the original poster was using good old Pascal I/O, but this illustrates that it is a Windows issue, not the way the file was opened. I could not see any way to check if the file was open already, so just had to rely on open failing. The open is in a Boolean function in my file unit.

Anyway, I added a very simple For loop around the file open, added the Sleep (half a second, based on the timing given by marcov and seems a reasonable figure) and only if fails 3 times will it activate the error system.

Code: Pascal  [Select][+][-]
  1.  
  2. { my file unit code all has p_file__ at the front, so I know at a glance which unit any item is in, open_file and can_write had that removed to make easier to view
  3.   open_file is simply TFilestream.Create inside an Except plus check (FileExists)
  4.   can_write is just a boolean constant to tell the function to open in read & write mode, default is read only
  5.   full_path_file is gained from the list obtained by FindAllFiles, load_stream is the TStream
  6. }    
  7. For i := 1 To 3 Do Begin  // arbitrary, but 3 was similar to marcov and seems a sensible number
  8.     If open_file( load_stream, full_path_file, can_write ) Then Goto set_details;  // I like Goto, I'm old school, within small procedures they provide a neat flow solution
  9.  
  10.     Sleep( 500 );  // wait half a second for any ops to clear
  11. End;
  12. // error routine
  13.  

I wanted to add that very simple solution for anyone affected by the same issue, as it clearly is a Windows issue that has not gone away. I'm using Windows 10 Professional.

I'd guess what is happening is that maybe because the original files are on another drive, that when I copy them to the C drive, Windows (maybe the antivirus, Avira), wants to check the files out and because I call the batch file from within the code, that by the time the program reaches the files, the operating system is still dicking about. I hadn't got it to run more than once in a row without the error before the little add, just ran it 5 times in a row with no errors.

Thank you to the regular contributors, Jamie, Thaddy and marcov.

Phil (aka old DOS err)

Lazarus: 3.4 (date: 2024-05-25)   FPC: 3.2.2   Version: Win64
Operating system: Windows Pro 10, 22H2
Hardware: Intel Core i5-3570K CPU, 3.40GHz, RAM: 8GB, 64 bit

Thaddy

  • Hero Member
  • *****
  • Posts: 16580
  • Kallstadt seems a good place to evict Trump to.
Re: Text file problem
« Reply #25 on: February 05, 2025, 03:54:20 pm »
Phil,

I guess you fell into the random trap and why random should not be implementation detail:

Your old code relies probably on a different random system than freepascal uses today.

Well in the wiki I have written examples for:
- Delphi or TP compatibility
- Compatibility with older Freepascal versions.
https://wiki.freepascal.org/Delphi_compatible_LCG_Random
https://wiki.freepascal.org/A_simple_implementation_of_the_Mersenne_twister

One of those two should help you out.
That is not your fault, btw, it is a big flaw in the core team handlng PRNG's.
That is also why I had to write the above repair code after they damaged their random (again).

    first one is for TP/Delphi
    second is for FPC's old mersenne twister.

Why is that important?
Because certain types of encryption use the random seed as a key, so a prng with the same algorithm and seed always decrypts too.
(there are many on this forum who are still not educated enough to grasp this, so i politely request them to not comment. O:-))

« Last Edit: February 05, 2025, 04:00:13 pm by Thaddy »
But I am sure they don't want the Trumps back...

Thaddy

  • Hero Member
  • *****
  • Posts: 16580
  • Kallstadt seems a good place to evict Trump to.
Re: Text file problem
« Reply #26 on: February 05, 2025, 04:06:38 pm »
And the core team can look up The Hives again.
https://www.youtube.com/watch?v=Uz1Jwyxd4tE
But I am sure they don't want the Trumps back...

old_DOS_err

  • New member
  • *
  • Posts: 8
Re: Text file problem
« Reply #27 on: February 17, 2025, 07:47:57 pm »
Hi Thaddy,

Thanks for the fast reply as usual. Sorry for being late getting back, at my age, it needs a much smaller spanner to jam up the works. To quote Marvin "Life. Don't talk to me about life."

Apologies for confusion. Part of the reason I am reluctant to post, which includes social media, is I tend to word things poorly (despite editing and rereading numerous times). They get misunderstood, then I'm reluctant to post again, and so remain with poor communication skills in regards to posting. A chicken and egg problem I'm sure most young people have no idea about (posting more than 1 sentence seems to be more their issue).

When I mentioned "random" in my post, I was referring to problems opening the files, not the encryption method (I use random, those bytes are stored in the file and then combined with a password, so I don't need to generate the same random numbers, I never thought of anyone doing that). This was an operating system issue, either Windows itself or the antivirus.

I had a quick look at the links anyway, interesting stuff. Must admit, when I look at that sort of code (and a lot in this forum), it makes me realise how old school my programming is. Even back in the Turbo Pascal days, I tended to still write like an assembler programmer.

You caught me out with The Hives link, I thought I was going to see some Open University type 1970s video by a mathematician in a kipper tie, talking about random numbers. :D

Regards,

Phil (aka old DOS err)


Thaddy

  • Hero Member
  • *****
  • Posts: 16580
  • Kallstadt seems a good place to evict Trump to.
Re: Text file problem
« Reply #28 on: Today at 07:30:25 am »
This is just for completeness:

The Delphi compatible prng is also TP compatible, but needs a little tweak for 16 bit. I will write that and add it here. In essence it takes the high word instead of the high dword. The algorithm is otherwise the same as Delphi. Example with the correct values for TP 1 to 3:
Code: Pascal  [Select][+][-]
  1. { Turbo Pascal compatible 16 bit random. Implemented to read old TP data }
  2. {$mode tp}
  3. var
  4.   Seed: Word;
  5. (* the first one is for completeness.
  6. It is only for 16 bit DOS and has no meaning on 32/64 bit systems.
  7. You can use the standard randomize instead. See the second one.
  8.  
  9. procedure Randomize16;
  10. begin
  11.   // this is the real deal for 16 bit.
  12.   Seed := MemW[$0040:$006C]; { Use the system timer as the seed }
  13. end;
  14. // use this one on 32/64 bit systems
  15. procedure Randomize16
  16. begin
  17.   Randomize; // initialize randseed
  18.   Seed := RandSeed shr 16; // make 16 bit
  19. end;
  20. *)
  21. function Random16: Word;
  22. begin
  23.   Seed := (134775813 * Seed + 1) mod 65536;
  24.   Random16 := Seed;
  25. end;
For analysis of old data  and simulations you just need the random16 function with initial seed = 0. You don't need the randomize.

Tested in DosBox against TP1 and TP3 from the Embarcadero museum and believed to be 100% compatible.
« Last Edit: Today at 08:18:54 am by Thaddy »
But I am sure they don't want the Trumps back...

 

TinyPortal © 2005-2018