Thread vs Application.ProcessMessages

[munair]

While the wiki has no problem promoting the use of Application.ProcessMessages if "you only want to make your application more responsive", a IMO good StackOverflow answer describes this style of programming as "widely considered to be extremely poor practice for a large number of very good reasons". See: https://stackoverflow.com/questions/25181713/i-do-not-understand-what-application-processmessages-in-delphi-is-doing.

Indeed, on my old computer running Debian 10, Application.ProcessMessages appears to go into an infinite loop. No matter what I tried, it didn't work. This could be GTK related, or just the greyness of the machine. 

So I decided to use a thread, and it seems to work like a charm. Granted, it is an old 32bit computer (Compaq Evo from 2001-2003), but it runs really well with Debian 10 Mate and Lazarus 2.0.4.

Going by my own experience, I have to agree that using Application.ProcessMessages seems poor practice and it may not work on every system. While testing I also found that under the hood LazSerial actually uses a combination of threading and Application.ProcessMessages. 

[Fungus]

So.. What's the question? Both Application.ProcessMessages and multithreading have each their forces and drawbacks. The trick is to select the right tool for the job at hand

[marcov]

A VCL/LCL application is per definition event driven. IOW some event happens, and you process that in relative short time, and return to the message/event loop. This keeps everything responsive.

Application.processmessages is a stopgap to do an occasional non-event driven programming, it allows you the run the event loop manually. For relatively simple applications this can be a solution, specially if you only use it in one spot, but it simply doesn't scale.

You can imagine if within a procedure/method that does Application.processmessages the event loops receives some message that calls another procedure/message that calls Application.processmessages. Then you get a kind of recursion (core event loop -> proc1 application.processmessages event loop->proc2.processmessages event loop -> etc). This also means that if you come back from the nested loop-with-nested processmessages to the outer one, that the outer must be prepared that a whole lot has happened during its application.processmessages and update its state accordingly

Many Delphi programmers simply don't understand these kinds of consequences, and randomly throw in some procesmessages "to keep things going". Which is fine, as long as it lasts. But they keep adding more cases, make the application more complex (and thus with heavier event-pressure), and at a certain point it blows up in your face.

If the above situation happens, the bad thing is that it doesn't just blow up, but it is hard to debug/fix/reproduce in such case. (you've painted/programmed yourself into a corner) There are many causes possible, and an shear infinite number of possibilities.

So a single case is fine, but if you rely to much, you really should look at threads. And whatever you do, try to keep an overview and understand what is happening. No technique takes ignorance particularly well.

But threading is the long term solution, but it requires some getting used to. Start with factoring the longest and/or most isolated calculations to threads.

[munair]

The problem that I had with Application.ProcessMessages was indeed a single case. No matter where I placed that statement, before, within, after the main loop, when debugging, the program just didn't go beyond the statement. When stepping into, execution continued to a GTK specific loop that processes up to 100 messages at a time. I would have to do more testing to see where exactly execution got stuck.

EDIT: This could be 32bit related as the same code runs fine on 64 bit, both GTK and Qt.

[jamie]

Then there is a bug in the ProcessMessages..

It is suppose to return when there aren't any messages in the que however, if the bug could of been in your own code where as a message being processed ending up in a code block that is yet posting another message on the block even if it's not on the same code block.
 
 Proper way to do this is when the Processmessages is called it should take a note on the current time stamp and process all messages that are current or older, any new messages coming in while this is taking place should not be processed this root while in the loop.

 Oh well, such as life..

[totya]

The problem that I had with Application.ProcessMessages was indeed a single case. No matter where I placed that statement, before, within, after the main loop, when debugging, the program just didn't go beyond the statement. When stepping into, execution continued to a GTK specific loop that processes up to 100 messages at a time. I would have to do more testing to see where exactly execution got stuck.

EDIT: This could be 32bit related as the same code runs fine on 64 bit, both GTK and Qt.

The best way, if you can use threads. Application.ProcessMessages slow down an app if execute very often (for ex. every for cycle). If I need Application.ProcessMessages, I use it with time measure (for ex. 50ms). But for example BeginUpdate can suspend your gui too... hard to say anything without visible source code/sample code...

[marcov]

The problem that I had with Application.ProcessMessages was indeed a single case. No matter where I placed that statement, before, within, after the main loop, when debugging, the program just didn't go beyond the statement. When stepping into, execution continued to a GTK specific loop that processes up to 100 messages at a time. I would have to do more testing to see where exactly execution got stuck.

If the event with the processmessages can be triggered multiple times, that would explain it. So first thing in that method, do something to prevent the event triggering it from happening. (and then restore it at the end)

[munair]

The problem really exists only on the old computer. I got the same code running (24/7) on two 64bit machines, no problem. I'll do some more testing starting with a simple project.

[marcov]

The problem really exists only on the old computer. I got the same code running (24/7) on two 64bit machines, no problem. I'll do some more testing starting with a simple project.

That is surely possible and doesn't say much. If your code is not 100% tight, small fluctuations in timing can hide or expose problems.

E.g. if you take my original example (with the case where application.processmessage starts nesting) IF some event is processed fast enough, it might not nest, because when it does application.processmessages, the call hasn't been made yet.

If you want to persist with application.processmessages, or at least try to, try to see if disabling the event(s) that trigger the method-with-application.procemessages are disabled. Check this if need be (with some global boolean, and an if check with  exit ).

So

Code: Pascal  [Select][+][-]

1. procedure xxx.somemethod(sender:boolean);
2.  
3. var inprocedure : boolean = false;  // not threadsafe, but LCL is not threaded. Nor are these events delivered cycles apart.
4.  
5. begin
6.    if inprocedure then
7.      exit; // redundant event
8.    inprocedure:=true;
9.    for i:=0 to very long do
10.       begin
11.          dosomething();
12.          if i mod 100 =0 then   // if dosomething is relatively cheap. Rule of thumb is application.processmessage every 0.1 - 0.25 s
13.            application.processmessages;
14.       end;
15.    inprocedure:=false;
16. end;  
17.  

This is more a test to see if you still get events.

When I got these kind of problems myself, I fixed the immediate problem, but in time the next incarnation popped up (but I had multiple processmessages and a lot of events that were highly interconnected).

In the end I rewrote the framework to be threading, and that has been a blessing.

P.s. I'm not much near computers this week, so if I don't respond, don't panic, it's only a week (  )

[munair]

AFAIK there is no nesting, but I'll do more tests as soon as the computer is available. Right now it is doing measurements 24/7 with an Arduino hooked to it and I need to keep it running for a while without interrupts.

[marcov]

Note that nearly all my experience in this is with Windows. I do use Linux, but most is batchprocessing rather than complex GUI or GUI+measurement&control.

[PascalDragon]

Code: Pascal  [Select][+][-]

1. procedure xxx.somemethod(sender:boolean);
2.  
3. var inprocedure : boolean = false;  // not threadsafe, but LCL is not threaded. Nor are these events delivered cycles apart.
4.  
5. begin
6.    if inprocedure then
7.      exit; // redundant event
8.    inprocedure:=true;
9.    for i:=0 to very long do
10.       begin
11.          dosomething();
12.          if i mod 100 =0 then   // if dosomething is relatively cheap. Rule of thumb is application.processmessage every 0.1 - 0.25 s
13.            application.processmessages;
14.       end;
15.    inprocedure:=false;
16. end;  
17.  

Shouldn't inprocedure be a const in that case (of course with WritableConsts enabled )? Otherwise that code would be rather useless...

[winni]

Hi

inprocedure must be a global var - otherwise it's useless.

I use this scheme every time I call a procedure from a timer.

Winni

[munair]

I think that marcov provided some pseudo-code to illustrate how Application.ProcessMessages can get trapped.

[Mr.Madguy]

Dunno. Old docs said, that you should have avoided threads, as they aren't "free" - they're essentially CPU states, that put some load on kernel. I don't know, how relevant this problem is today.

Application.ProcessMessages - is about software multitasking, while threads are about hardware one. In most cases hardware one isn't actually needed - it can be avoided via using software one. And you don't even need Application.ProcessMessages. You can split your task into small chunks and execute them one by one right from the start. Application.ProcessMessages is just a crutch, that makes things a little bit easier.

Threads are absolute necessary, only when they can be blocked. Even more. They should stay blocked most of the time. Waiting for hardware event for example. Another case - multi-core support. Modern CPUs have already reached their performance threshold. Only way to increase performance even further - is parallel computing.