Cannot run a process procedure in a thread

[glubbish]

Hi,

I have a program that uses mkvmerge to add a subtitle to a video.
This is working fine (see attached subtitles.zip), the problem is the program is unresponsive while the process is running.

My thought was to do it in a thread, which turned out not to be easy.
My failing attempt is attached as (Subtitles Thread.zip).

I created another project that had this working (see attached 'thread subtitle try')
in this program if you comment out fullgrid and use oldproc it kind of works

procedure TForm1.FormActivate(Sender: TObject);
begin
  FillGrid('\\kodi\Movies');
  //FillGrid('\\kodi\TV');
  //oldproc;
  SkipRow;
  Form1.Display.Cells[0, CurrentRow] := 'Process Complete';
end;         

Can someone point me the right direction?

Thanks.

[Khrys]

Multithreading and GUIs don't play nice without some effort; whenever a GUI element is updated (i.e. by changing a caption, hiding something etc.) then that update has to be done from the main thread (usually accomplished via  Application.QueueAsyncCall).

However in your case I'd just keep things simple and stay within a single thread. The task is already running asynchronously as another process, so why parallelize further?
Specifically, I'd change this

Code: Pascal  [Select][+][-]

1. while AProcess.running = True do  // Show its working via counter
2. begin
3.   Inc(i);
4.   sleep(500);
5.   percent        := ((i * 17753936.16) / MySize) * 100;
6.   DisplayPercent := trunc(percent);
7.   if DisplayPercent > 100 then DisplayPercent := 100;
8.   ShowProgress(DisplayPercent);
9. end;

to this

Code: Pascal  [Select][+][-]

1. while AProcess.running = True do  // Show its working via counter
2. begin
3.   Inc(i);
4.   sleep(16);
5.   Application.ProcessMessages();
6.   percent        := ((i * 568125.96) / MySize) * 100;
7.   DisplayPercent := trunc(percent);
8.   if DisplayPercent > 100 then DisplayPercent := 100;
9.   ShowProgress(DisplayPercent);
10. end;

The key here is that  Application.ProcessMessages  is called at an interactive rate (around 60 times per second), allowing GUI events (paint requests, custom event handlers) to flow freely.
(I'd look for a more robust way to calculate the progress percentage, though...)

(And please use  [cоde][/code]  tags when pasting code)

[glubbish]

Thanks Khrys I will give that a try.

"The task is already running asynchronously as another process, so why parallelize further?"
- I was thinking that I could do several at the same time as well as having it responsive.

[Thaddy]

If you use TAsyncprocess correctly there is no need for application.processmessages.
You just connect the onreaddata and onterminate events.
That is, if mkvmerge outputs its progress to stdout. And it can ...afaik.
This will automatically keep your program responsive. I think the suggested solution is also too complex.

[glubbish]

Thanks Thaddy, I will do some reading on TAsyncprocess and give it a try.
Application.processmessages did not work, the program was still unresponsive. (It would minimize when the icon was clicked in the taskbar, but would not come back when clicked again).

[glubbish]

Not easy to find examples.
Does not seem to work for me.

Code: Pascal  [Select][+][-]

1. procedure RunProcess;
2. var
3.   AProcess:       TAsyncProcess;    

on compile gives:
main.pas(63,19) Error: Identifier not found "TAsyncProcess"

I am currently getting the output with the current TProcess procedure.

Code: Pascal  [Select][+][-]

1. procedure RunProcess;
2. var
3.   AProcess:       TProcess;
4.   AStringList:    TStringList;
5.   i:              integer = 0;
6.   percent:        real;
7.   DisplayPercent: integer;
8. begin
9.   AProcess            := TProcess.Create(nil);                      // Create Process. (nil) means it must be freed.
10.   AProcess.Executable := MkvMerge;                                  // Tell the new AProcess what the command to execute is.
11.   AProcess.Parameters.Add(MyParameter);                             // Pass MkvMerge parameters to the program
12.   AProcess.Options    := AProcess.Options + [poUsePipes];           // Use Pipes so we can check output
13.   AProcess.ShowWindow := swoShow;                                   // Hide the console window
14.   AProcess.Execute;                                                 // Now let AProcess run the program
15.  
16.   while AProcess.running = True do  // Show its working via counter
17.   begin
18.     Inc(i);
19.     sleep(16);
20.     Application.ProcessMessages();
21.     percent        := ((i * 568125.96) / MySize) * 100;
22.     DisplayPercent := trunc(percent);
23.     if DisplayPercent > 100 then DisplayPercent := 100;
24.     Form1.Display.Cells[3, CurrentRow]          := 'Processing (' + IntToStr(DisplayPercent) + '%)';
25.   end;
26.  
27.   AStringList := TStringList.Create;  // Keep the output
28.   AStringList.LoadFromStream(AProcess.Output);
29.  
30.   case AProcess.ExitCode of   // Check for errors
31.     1: begin                  // This is a warning. Display it and choose if we continue If timestamp out of order, then fix manually and run again.
32.       for i := 0 to AStringList.Count - 1 do if MessageDlg('Continue?', AStringList.Strings[i], mtConfirmation, [mbYes, mbNo], 0) = mrNo then
33.         begin
34.           AStringList.Free;
35.           AProcess.Free;
36.           halt(AProcess.ExitCode);
37.         end;
38.     end;
39.     2: begin  // This is an error. Display it, but no choice to continue. Also free
40.       for i := 0 to AStringList.Count - 1 do ShowMessage(AStringList.Strings[i]);
41.       AStringList.Free;
42.       AProcess.Free;
43.       halt(AProcess.ExitCode);
44.     end;
45.   end;
46.   AStringList.Free;    // This is not reached until MkvMerge stops running.
47.   AProcess.Free;
48. end;                                      

[Remy Lebeau]

Code: Pascal  [Select][+][-]

1. procedure RunProcess;
2. var
3.   AProcess:       TAsyncProcess;    

on compile gives:
main.pas(63,19) Error: Identifier not found "TAsyncProcess"

TAsyncProcess is declared in the AsyncProcess unit, did you add that unit to your uses clause?

[glubbish]

Application.Processmessages did not help
Using asyncprocess did not work either.

I cannot bring this form to the foreground when it is running, I have to minimize all other windows.

Threads were not working, but in the version I went from MyThread := TMyThread.Create(True); to
MyThread := TMyThread.Create(False);
MyThread.Start.

This gives an access violation on MyThread.Start.

This version attached.

[Thaddy]

All wrong. will post an answer later with working code..
Meanwhile read up on threads.
Start here:
https://www.seti.net/engineering/threads/threads.php
Everything there compiles with fpc 100%

[cdbc]

Hi
...and remember:

"The theoreticians have proven that this is unsolvable, but there's three of us, and we're smart ..."

Hehehe... I just love it 
Regards Benny

[glubbish]

Thanks for taking the time to help me with this Thaddy.
It has gone through several iterations, some more successful than others.
I really look forward to seeing how you incorporate threads with the process, which I could not find an example of.

[glubbish]

No help so far.
Is there anyone else who can help me get this going?

[440bx]

No help so far.
Is there anyone else who can help me get this going?

disclaimer: I do things very differently than you're trying to do them.  What follows are some suggestions, use them as you see fit.

First, you may not even need a separate thread (though, one could be well used... more on that later).  From what you've stated, you've created a process, you can simply poll for the process still running in a timer (executed every 5 ms or so)  That's the simplest way but, it's not the most efficient way.  Obviously, you wouldn't need Application.ProcessMessages since the polling is happening in a WM_TIMER message.

The most efficient way is to use a thread.  Create the thread telling it what process to start.  After the thread starts the process then it can simply wait for the process to end (WaitForSingleObject.)  Once the process has ended the thread can then post a message to the GUI thread informing it the process is done (the task is completed) and do whatever it's supposed to do once the process is done.  It would probably be a good idea to verify that the process ended successfully (GetExitCodeProcess can help.)  Also, the thread that creates/manages the process should _not_ have absolutely anything to do with GUI stuff.  It should have absolutely no need to process messages (that's the GUI's thread job, to pump and manage messages, not the  other thread's.)  Also, once the process has ended and the thread has notified the GUI that the process has finished then the thread should end (ExitThread/EndThread) since it has completed its job.

I have absolutely no idea how all this stuff is done using the OOP facilities provided by Lazarus.  It's trivial to do with the Windows API (which is all I know.)

The task you're trying to accomplish should _not_ need using Application.ProcessMessages, if you need it then you're not doing it quite right.

HTH.

[cdbc]

Hi
@glubbish: Have you read the /homework/ given to you by Thaddy?!?
If not... DO IT!
It's useless to try and help you in that mess, if you don't understand the tiniest  bit about threads! Cardinal rule in threading is: NO GUI-STUFF in thread.
Use messages, queues, synchronize or some other means of communicating.
Butchering your code, it /is/ possible to get it to run, but explaining it to you, will take a book's worth...
edit: hints

Code: Pascal  [Select][+][-]

1. const LM_PROGRESS = LM_USER + 7; //<- HERE
2.  
3. type
4.  
5.   { TForm1 }
6.  
7.   TForm1 = class(TForm)
8.     Display:   TStringGrid;
9.     ExitPanel: TPanel;
10.     procedure DisplayDrawCell(Sender: TObject; aCol, aRow: integer; aRect: TRect; aState: TGridDrawState);
11.     procedure FormActivate(Sender: TObject);
12.     procedure FormDestroy(Sender: TObject);
13.     procedure FormShow(Sender: TObject);
14.     procedure ExitPanelClick(Sender: TObject);
15.   private
16.     procedure LMProgress(var aMsg: TLMessage); message LM_PROGRESS; //<- HERE
17.   public
18.  
19.   end;
20. ...
21. var
22.   Form1:       TForm1;
23.   CurrentRow:  integer;
24.   Video:       string;
25.   Subtitle:    string;
26.   MkvMerge:    string;
27.   MyParameter: string;
28.   LineColor:   TColor;
29.   MysizeArray: array [1..100] of int64;
30.   Mysize:      int64;
31.   MyRow:       integer;
32.   MyThread:    TMyThread;
33.  
34.   frmHandle: ptrint; // <-- HERE
35. ...
36. procedure TMyThread.ShowProgress(Progress: integer); // <-- HERE
37. begin
38.   PostMessage(frmHandle,LM_PROGRESS,-1,Progress);
39. end;
40. ...
41. ////////////////////////////////////
42.  
43.     MyThread:= TMyThread.Create(True);  // <-- HERE
44.     MySize := MySizeArray[0];
45.     MyRow  := i;
46.     MyThread.Start;
47. ////////////////////////////////////
48. ...
49. procedure TForm1.FormActivate(Sender: TObject);
50. begin
51.   frmHandle:= Handle; // <-- HERE
52. ...
53. procedure TForm1.FormDestroy(Sender: TObject);
54. begin
55.   MyThread.Terminate; // <-- HERE
56.   MyThread.WaitFor; // <-- HERE
57.   Mythread.Free;
58. end;
59. ...
60.   while AProcess.running = True do
61.   begin
62.     Inc(i);
63.     sleep(5000);
64.     percent        := ((i * 17753936.16) / MySize) * 100;
65.     {$ifdef dbg}DisplayPercent:= 100; {$else} DisplayPercent := trunc(percent); {$endif}
66.     if DisplayPercent > 100 then DisplayPercent := 100;
67.     //Synchronize(@ShowProgress); // coded differently from the getgo would be an option
68.     showprogress(displaypercent);
69.     {$ifdef dbg}PostMessage(frmHandle,LM_PROGRESS,-1,DisplayPercent+12); {$endif}
70.   end;
71. ...
72. procedure TForm1.LMProgress(var aMsg: TLMessage);
73. begin
74.   {$ifdef dbg}writeln('Processing (',aMsg.LParam,'%)'); {$else}
75.   Display.Cells[3, MyRow] := 'Processing (' + IntToStr(aMsg.LParam) + '%)';
76.   Application.ProcessMessages; {$endif} // NOT sure it's necessary anymore
77. end;
78.  

Regards Benny

[TRon]

No help so far.
Is there anyone else who can help me get this going?

A few pointers.
- remove all the thread related code, seriously
- the way you add parameters to tprocess seems very wrong to me (I could not get that to work).
- why do you pass along 4 filenames to mkvmerge ? (should be one for result/output, one for original movie, one for subtitle)
- if you run mkvmerge in quiet mode then mkvmerge will be ... well quiet (no textual feedback unless an actual error occurs). Therefor your percentage update implementation will happen only one time
- even if you omit the quiet mode option there will not be much feedback
- you can get more feedback by adding the option --gui-mode which returns percentages but adding a srt to file will happen very quick and not many feedback is provided (only a couple of percentage feedback lines are returned depending on the lenght of your video)
- I am confused by the tproces running loop and the corresponding percentage calculations

that are the all to the tprocess related issues I was able to locate on quick notice.

Other than that a few questions/suggestions:
- Are you aware that it is possible to change the extention of a filename with ChangeFileExt ?
- Are you aware that you can use FindAllFiles to find all files that match a certain pattern (returned in a stringlist) ?

edit:: sorry forgot to add link : https://wiki.freepascal.org/Executing_External_Programs#Reading_large_output