Recent

Author Topic: [SOLVED]error when modifying main form from indy thread  (Read 14520 times)

rvk

  • Hero Member
  • *****
  • Posts: 3842
Re: [SOLVED]error when modifying main form from indy thread
« Reply #15 on: October 19, 2015, 12:51:41 pm »
F.Y.I. I tested your thread_sendmessage2 on
Linux debian 3.16.0-4-586 #1 Debian 3.16.7-ckt9-3~deb8u1 (2015-04-24) i686 GNU/Linux
with Lazarus 1.5 r48991M FPC 2.6.5 i386-linux-gtk 2
and Lazarus 1.0.10 through 1.2.4

All work fine (with all 3 methods) if you don't choose "update and refresh".
However, when choosing "update and refresh" the sendmessage fails in all versions.
(The other 2, synchronize and postmessage keep working, regardless the sleep)

For sendmessage/"update and refresh", just the ProcessMessages doesn't give any problem.
Update and Refresh both give problems.

I did find this post:
SendMessage works different on the gtk+ widgetset than win32.

on Win32. SendMessage sends the message to the queue, and blocks sending thread, until executing thread process the message. If sending thread is the same as process thread, the message is processed immediatly.
But on gtk+ widgetset Send Message is processed directly from the sending thread, instead of putting message to the queue and wait for processing thread to process it! And this cause bugs, especially with GUI applications.

is it a bug or feature?
If this is really true (i.e. the SendMessage is processed IN the sending thread) then using SendMessage is the same as directly calling handler_send_message from the thread (which is a big no no). So that could be the problem when using SendMessage on Linux.

So policy should be... always use PostMessage or Synchronize.

taazz

  • Hero Member
  • *****
  • Posts: 5363
Re: [SOLVED]error when modifying main form from indy thread
« Reply #16 on: October 19, 2015, 01:07:56 pm »
But on gtk+ widgetset Send Message is processed directly from the sending thread, instead of putting message to the queue and wait for processing thread to process it! And this cause bugs, especially with GUI applications.

is it a bug or feature?

If it is true then it is a bug(incomplete implementation), not a feature and must be corrected. 

If this is really true (i.e. the SendMessage is processed IN the sending thread) then using SendMessage is the same as directly calling handler_send_message from the thread (which is a big no no). So that could be the problem when using SendMessage on Linux.

So policy should be... always use PostMessage or Synchronize.
Basically even if it did work as advertised it has no difference from using synchronize, so I would say that use synchronize in the 1st place instead of sendmessage so you have no illusions of what really goes on.

Good judgement is the result of experience … Experience is the result of bad judgement.

OS : Windows 7 64 bit
Laz: Lazarus 1.4.4 FPC 2.6.4 i386-win32-win32/win64

Michael Collier

  • Sr. Member
  • ****
  • Posts: 253
Re: [SOLVED]error when modifying main form from indy thread
« Reply #17 on: October 19, 2015, 07:00:59 pm »
F.Y.I. I tested your thread_sendmessage2 on
Linux debian 3.16.0-4-586 #1 Debian 3.16.7-ckt9-3~deb8u1 (2015-04-24) i686 GNU/Linux
with Lazarus 1.5 r48991M FPC 2.6.5 i386-linux-gtk 2
and Lazarus 1.0.10 through 1.2.4

All work fine (with all 3 methods) if you don't choose "update and refresh".

Just tried again, I've updated to lazaraus 1.4 since my previous posts, but anyway.. sendmessage() fails even without "update and refresh", you have to select a higher number of message count =100 , and execute the task several times before the application crashes, say 5 time ish. ... just tried postmessage() again.. yes it crashes for me, just takes more attempts

rvk

  • Hero Member
  • *****
  • Posts: 3842
Re: [SOLVED]error when modifying main form from indy thread
« Reply #18 on: October 19, 2015, 07:43:05 pm »
Just tried again, I've updated to lazaraus 1.4 since my previous posts, but anyway.. sendmessage() fails even without "update and refresh", you have to select a higher number of message count =100 , and execute the task several times before the application crashes, say 5 time ish. ... just tried postmessage() again.. yes it crashes for me, just takes more attempts
I can understand why SendMessage fails. If the message is executed in the same thread as where you call it, it doesn't matter if you call update/refresh or not. It will eventually fail (because you're updating the main-thread-components in another thread). But the fact that PostMessage fails puzzles me. It should post a message to the message queue and continue (and the queue should be handled by the correct thread).

What parameters did you use to make the PostMessage crash on Laz1.4?

I'll try to do some tests tomorrow to see if I can identify if the SendMessage is executed in the same thread. (Does anybody have code-snippet suggestions for that :)) I tried GetCurrentThreadId() but that doesn't seem to work in a thread on Linux. (It keeps giving me -1233851584, GetThreadId too.)

Code: Pascal  [Select]
  1. // in main.pas
  2.     Memo1.Lines.Add( 'sendmessage :'
  3.     + IntToStr( Message.lParam )
  4.     + ' thr: ' + IntToStr( Message.wParam ) // received via sendmessage from thread
  5.     + ' cur: ' + IntToStr( GetCurrentThreadId )
  6. ...
  7.     Memo1.Lines.Add( 'postmessage :'
  8.     + IntToStr( Message.lParam )
  9.     + ' thr: ' + IntToStr( Message.wParam ) // received via postmessage from thread
  10.     + ' cur: ' + IntToStr( GetCurrentThreadId )
  11. ...
  12. // in thread_classes.pas
  13.       mm_sendmessage : begin
  14.         SendMessage( form_thread_message.Handle
  15.                    , LM_SEND_MESSAGE_TEST
  16.                    ,  GetCurrentThreadId()
  17.                    , FCount
  18.                    )                        ;
  19.       end;
  20.       mm_postmessage : begin
  21.         PostMessage( form_thread_message.Handle
  22.                    , LM_POST_MESSAGE_TEST
  23.                    , GetCurrentThreadId()
  24.                    , FCount
  25.                    )                        ;

Michael Collier

  • Sr. Member
  • ****
  • Posts: 253
Re: [SOLVED]error when modifying main form from indy thread
« Reply #19 on: October 19, 2015, 08:04:17 pm »
What parameters did you use to make the PostMessage crash on Laz1.4?

I set the message count to 100, sleep to zero, then just clicked away at it, to be honest I nearly gave up thinking I had mis-remembered the original test, but in the end it did crash. I also forced it to fail via sendmessage() first - because I wasn't sure if previous failures had any affect on subsequent attempts. This was probably just superstition on my part but that's what I did.


rvk

  • Hero Member
  • *****
  • Posts: 3842
Re: [SOLVED]error when modifying main form from indy thread
« Reply #20 on: October 19, 2015, 08:41:54 pm »
Even with setting 100 times and 0 sleep and update/refresh I couldn't get PostMessage to crash. I did however got Synchronize to hang with that  %) (see last paragraph because it could also be the problem with your PostMessage)

With PostMessage you can see the thread is terminated at about the 2nd or 3rd iteration. All the messages are in queue and the thread has done its job. If I click really fast after each other the threads are already terminated while the messages are still being processed. I don't know how many times you did this? Is there a maximum of messages for a queue? I tested with 100.000 PostMessages and the thread ended when about message nr. 1.000 was being executed.

The reason I got Synchronize to hang is probably due to your btn_free_server_threadClick procedure. It does a my_server_thread.Terminate but doesn't wait until it is really terminated before FreeAndNil. And I clicked free_server while the thread was still running !! And then it hangs. So if you did this same test with the PostMessage and you clicked btn_free_server_threadClick before the thread was terminated then it's logical that hangs too. You should change it to this:

Code: Pascal  [Select]
  1. procedure Tform_thread_message.btn_free_server_threadClick(Sender: TObject);
  2. begin
  3.   my_server_thread.Terminate      ;
  4.   my_server_thread.waitfor        ;
  5.   FreeAndNil ( my_server_thread ) ;
  6.   configure_buttons ( True)       ;
  7. end;

Another options would be to only activate the btn_free_server_thread when the thread has ended. Now that button is enabled when the thread is started.

What happens with your tests if you build in the tthread.waitfor?
« Last Edit: October 19, 2015, 08:43:28 pm by rvk »

Michael Collier

  • Sr. Member
  • ****
  • Posts: 253
Re: [SOLVED]error when modifying main form from indy thread
« Reply #21 on: October 19, 2015, 09:47:30 pm »
What happens with your tests if you build in the tthread.waitfor?

Thanks for pointing that out, I have now added the waitfor statement. I tried again and postmessage() still crashed. At first I thought you had cured it, because I was clicking away and it didn't crash. I upped the qty to 1000 and also tried dragging the main form during the execution cycle. It fails much more quickly after failing once. It stopped at count = 220.


rvk

  • Hero Member
  • *****
  • Posts: 3842
Re: [SOLVED]error when modifying main form from indy thread
« Reply #22 on: October 19, 2015, 10:07:29 pm »
Ok, now for me, on Linux, I got an SIGSEGV at message 11.427 (of the 100.000) after the thread was already finished a long time ago. I'm wondering if this is really a thread-problem or something else. If the thread is finished (terminated but not yet freed) it shouldn't have anything more to do with the message-queue, so why it gives a SIGSEGV is beyond me.

For me it crashed in XRenderCompositeText8.

I'll try to debug this some more, tomorrow.

I'm wondering how stable the message-queue is under Linux. If I do 100.000/1ms/PostMessage the process slows down at about 5000 and hangs at about 6000 if I choose it without Update/Refresh.

With 100.000/0ms/PostMessage/no update I even got a SIGABRT at #510. I don't even know what that is :)  (abort signal, heap corrupt/overflow??)
« Last Edit: October 19, 2015, 10:13:50 pm by rvk »

rvk

  • Hero Member
  • *****
  • Posts: 3842
Re: [SOLVED]error when modifying main form from indy thread
« Reply #23 on: October 20, 2015, 12:30:54 pm »
Mmmm, I did some tests. I've stripped the threads from the test-project and wanted to only test PostMessage and SendMessage. On Windows, when using PostMessage, it doesn't go beyond 10.000. Apparently the maximum Posts you can do is 10.000. But it doesn't crash or hang.

Then Linux. When doing the same test it does go beyond 10.000 but it crashes with a SIGSEGV at 11.624. Constantly at exactly the same message. Even if I do 7.000 and click again right away (posting 14.000 messages) it still runs only to 4.624 for the second run (so message 11.624). The fact it crashes seems a bug. If the number of messages is a problem it should stop at 10.000 like it does in Windows.

In this test-project I couldn't get the SendMessage to crash but that's logical. No threads and SendMessage directly executed the message.

@Michael, does this test-project crash for you sooner than 11.624? (100.000/0ms/PostMessage/no update & update)

Michael Collier

  • Sr. Member
  • ****
  • Posts: 253
Re: [SOLVED]error when modifying main form from indy thread
« Reply #24 on: October 20, 2015, 05:03:52 pm »
Linux crashes  @11,586 every time (100.000/0ms/PostMessage/no update & update)
I can reduce by 1 to 11, 585 and repeat quickly without crashing, so my machine behaves differently to yours, although the number at which it fails seems very close. My linux box has 2.0 GiB memory.

A bit more..

I modified your test so that the loop counter Cnt is global (to make it accessible from do_updates() ), then only did processmessages() if (cnt mod 10 = 0). Thus reducing the number of processmessages() without reducing the number of postmessages(). It goes much further now, - is still running without crashing, count of postmessage() >190,000. Not sure if this helps but thought I'd let you know.






rvk

  • Hero Member
  • *****
  • Posts: 3842
Re: [SOLVED]error when modifying main form from indy thread
« Reply #25 on: October 20, 2015, 05:44:54 pm »
I modified your test so that the loop counter Cnt is global (to make it accessible from do_updates() ), then only did processmessages() if (cnt mod 10 = 0). Thus reducing the number of processmessages() without reducing the number of postmessages(). It goes much further now, - is still running without crashing, count of postmessage() >190,000.
Not exactly sure how you did that but making cnt (from the btn_executeClick) a global variable doesn't work. Because the PostMessages are executed first, so cnt is always 100.000 (or max) when the messages are really executed.

I did it like this. In handler_post_message I called do_updates(Message.lParam) and that way you can do the (cnt mod 10) correctly. But my guess is that you did something similar otherwise it wouldn't run so far.

So it seems to be a combination of PostMessage and Application.ProcessMessages that's giving problems here. I did find that for Windows the limit is 10.000. So that's the reason in Windows it stops at 10.000.

Why the crash happens in Linux, I don't know. Unfortunately, I don't have that much knowledge about the messaging system in Linux.

I'm also not sure why the thread-example crashes. For me it sometimes hangs and sometimes crashes with a SIGSEGV (with PostMessage). But that could be the same cause as the PostMessage in the no-thread example. Sometimes I can see it slow down... and hang (at around 2000-3000) and sometimes it just hangs at 1500 (all with sleep(0)). With sleep(1) it goes to about 8000-9000. So my guess is there is really something "fishy" with the messaging-queue in Linux. I did find mention about the messaging-queue not being thread-safe in the past but that should have been solved.

So if you have a need for lots of messages in your thread then stay away from PostMessage and use Synchronize.
(Sendmessage was already a "no no" because it's executed in the same thread)

Any Linux/Lazarus gurus out there (with knowledge about the messaging-queue in Lazarus on Linux) who want to find out why the PostMessage crashes at about 11.000?

taazz

  • Hero Member
  • *****
  • Posts: 5363
Re: [SOLVED]error when modifying main form from indy thread
« Reply #26 on: October 20, 2015, 06:21:07 pm »
So if you have a need for lots of messages in your thread then stay away from PostMessage and use Synchronize.
(Sendmessage was already a "no no" because it's executed in the same thread)
Wrong conclusion. If the message exchange is low then a simple synchronize should not be a problem if its to high then the use of synchronise is out of the question it will show down the thread speed to a crawl, you have to use postmessage to keep things speedy.

There are a number of problems in the test case described here (I'll take a look on the code later tonight on windows) but the most important are 1) the number of messages is to high for normal operation 2) never ever call processmessages its the worst think you can do. although lcl makes it impossible you should realy try to avoid it at all costs.
Good judgement is the result of experience … Experience is the result of bad judgement.

OS : Windows 7 64 bit
Laz: Lazarus 1.4.4 FPC 2.6.4 i386-win32-win32/win64

rvk

  • Hero Member
  • *****
  • Posts: 3842
Re: [SOLVED]error when modifying main form from indy thread
« Reply #27 on: October 20, 2015, 08:13:38 pm »
1) the number of messages is to high for normal operation
I'll admit that the number of messages is indeed abnormal. But it does shows a flaw in the underlying messaging-system on Linux. And I'm not sure if that flaw show its head in other cases. For instance in the example with the threads where PostMessage method already hangs at about 1.500 messages (which might be a real-time number in normal operations). And that example even hangs without a ProcessMessages on Liinux.

2) never ever call processmessages its the worst think you can do. although lcl makes it impossible you should realy try to avoid it at all costs.
Yeah... lcl makes it impossible but I should avoid it at all costs :) Nice. And how would I avoid it in this example?

I added a more simpler test-project. It just does 20.000 PostMessages in a loop (without ProcessMessages) and after the loop is done the messages are processed with a ProcessMessages. There is no chance of recursion here and it's a simple example so using ProcessMessage shouldn't be a problem. On Windows the maximum messages is 10.000 which shows quite clearly. And it does the handling perfectly. On Linux it crashes for me at around 13.000 with a SIGSEGV. I also noticed that on Linux the 20.000 PostMessages are done in an instance while on Windows it takes a while. (Maybe because the messaging on Windows is handled by Windows itself?)

(note: On Windows this all runs perfectly so it really needs to be tested on Linux)
« Last Edit: October 20, 2015, 08:22:40 pm by rvk »

taazz

  • Hero Member
  • *****
  • Posts: 5363
Re: [SOLVED]error when modifying main form from indy thread
« Reply #28 on: October 21, 2015, 02:42:28 am »
1) the number of messages is to high for normal operation
I'll admit that the number of messages is indeed abnormal. But it does shows a flaw in the underlying messaging-system on Linux. And I'm not sure if that flaw show its head in other cases. For instance in the example with the threads where PostMessage method already hangs at about 1.500 messages (which might be a real-time number in normal operations). And that example even hangs without a ProcessMessages on Liinux.

2) never ever call processmessages its the worst think you can do. although lcl makes it impossible you should realy try to avoid it at all costs.
Yeah... lcl makes it impossible but I should avoid it at all costs :) Nice. And how would I avoid it in this example?
1) what is processmessages? what it does and when is used?
In this case you simple don't need to use it at all. the processing is in an other thread the messages will be processed in due time from the main thread stop disrupting the normal flow of things for no reason.
Good judgement is the result of experience … Experience is the result of bad judgement.

OS : Windows 7 64 bit
Laz: Lazarus 1.4.4 FPC 2.6.4 i386-win32-win32/win64

Cyrax

  • Hero Member
  • *****
  • Posts: 758
Re: [SOLVED]error when modifying main form from indy thread
« Reply #29 on: October 21, 2015, 06:22:42 am »
1) the number of messages is to high for normal operation
I'll admit that the number of messages is indeed abnormal. But it does shows a flaw in the underlying messaging-system on Linux. And I'm not sure if that flaw show its head in other cases. For instance in the example with the threads where PostMessage method already hangs at about 1.500 messages (which might be a real-time number in normal operations). And that example even hangs without a ProcessMessages on Liinux.

2) never ever call processmessages its the worst think you can do. although lcl makes it impossible you should realy try to avoid it at all costs.
Yeah... lcl makes it impossible but I should avoid it at all costs :) Nice. And how would I avoid it in this example?
1) what is processmessages? what it does and when is used?
In this case you simple don't need to use it at all. the processing is in an other thread the messages will be processed in due time from the main thread stop disrupting the normal flow of things for no reason.

Code: Pascal  [Select]
  1. procedure TApplication.ProcessMessages;
  2. begin
  3.   if Self=nil then begin
  4.     // when the programmer did a mistake, avoid getting strange errors
  5.     raise Exception.Create('Application=nil');
  6.   end;
  7.   WidgetSet.AppProcessMessages;
  8.   ProcessAsyncCallQueue;
  9. end;

It calls widgeset specific message handling procedure and procedure which handles asynchronous call queues periodically.  Widgeset specific message handling procedure (in case win32 widgeset) calls CheckSynchronize and CheckPipeEvents procedures.

CheckSynchronize procedure is important one and related to handling thread execution succesfully. If you are using threads, you need to call CheckSynchronize periodically in your main program loop.