Recent

Author Topic: Synchronize not working  (Read 21527 times)

loaded

  • Sr. Member
  • ****
  • Posts: 437
Re: Synchronize not working
« Reply #45 on: November 26, 2021, 06:07:22 pm »
Because sleep(t) means "pause this thread for t mSec", i.e. it doesn't need anything external to kick it back into life. It's the sort of thing you'd use if e.g. enforcing a protocol delay in background communications.
I thought the use of sleep or application.processmessage was considered negative from a programming perspective.
Is sleep used in professional programs?
If Ide=Lazarus 2.0.10 32 Bit and Os=Win 10 Home 64 Bit then Get up and do something useful! Because God is the helper of those who start again;

Warfley

  • Hero Member
  • *****
  • Posts: 654
Re: Synchronize not working
« Reply #46 on: November 26, 2021, 06:39:11 pm »
Yes of course, sleep is very usefull if you have to wait for a certain amount of time (of course not in interactive programs like GUIs, but for threads or non interactive programs it is really usefull)
« Last Edit: November 26, 2021, 06:41:13 pm by Warfley »

MarkMLl

  • Hero Member
  • *****
  • Posts: 3670
Re: Synchronize not working
« Reply #47 on: November 26, 2021, 06:41:21 pm »
I thought the use of sleep or application.processmessage was considered negative from a programming perspective.
Is sleep used in professional programs?

This breaks down into two issues.

Apropos APM: you can't use it in a background thread, if you've got a tight loop in the main thread you /have/ to use it otherwise the GUI will be unresponsive.

Apropos sleep(): use it where necessary in a background thread, you can't use it in the main thread since it will make the GUI unresponsive (i.e. the main loop not running so there is nothing to read and despatch mouse/keyboard/etc. events.

If you find yourself using either of those to avoid a race condition, /then/ you're doing it wrong.

MarkMLl
MT+86 & Turbo Pascal v1 on CCP/M-86, multitasking with LAN & graphics in 128Kb.
Pet hate: people who boast about the size and sophistication of their computer.
GitHub repositories: https://github.com/MarkMLl?tab=repositories

440bx

  • Hero Member
  • *****
  • Posts: 2623
Re: Synchronize not working
« Reply #48 on: November 26, 2021, 06:43:55 pm »
But can I please point out that the only thing that is remotely relevant to OP's question is use of Application.ProcessMessages. All of this business about Sleep() etc. is spurious.
Absolutely.  At this stage I'd like to point out that the whole thing about Sleep() took a life of its own because SymbolicFrank suggested that Sleep() could be used as a better alternative to replacement than Application.ProcessMessages. 

As far as CPU usage goes, a well written application avoids gratuitously consuming CPU which is what Sleep(0) ends up doing and, that is particularly bad on laptops because it drains the battery for nothing.



I thought the use of sleep or application.processmessage was considered negative from a programming perspective.
Is sleep used in professional programs?
Application.ProcessMessages is usually negative now because today's OSs support multi-threading and implement preemptive multitasking.  Application.ProcessMessages is something that was often necessary in the versions of Windows that required cooperative multitasking, their scheduler depended on the program voluntarily relinquishing the CPU to the scheduler and processing messages which was the gateway to those O/Ss schedulers regaining control.

Sleep (except Sleep(0)) is usually an indicator that the programmer doesn't want to gratuitously waste clock cycles which, obviously, is detrimental to the smooth operation of the system as a whole.  That said, even Sleep (even if not zero) can be misused.  For instance, calling Sleep(somenonzerovalue) in a GUI thread is likely always a bad idea (as @MarkMLI pointed out while I was typing this post.)

In addition to what @Warfley mentioned, there are times when the need to poll for a non-critical event is needed, that's a shoe-in for Sleep.
« Last Edit: November 26, 2021, 06:48:44 pm by 440bx »
FPC v3.0.4 and Lazarus 1.8.2 on Windows 7 64bit.

loaded

  • Sr. Member
  • ****
  • Posts: 437
Re: Synchronize not working
« Reply #49 on: November 26, 2021, 07:05:55 pm »
MarkMLl and Warfley and 440bx  thank you very much for the enlightening information.
A little off topic:
I have a lot of questions in my mind, one of them is;
Does the running speed of threads change according to the language they are written in?
So is it possible that a thread written in a different language needs sleep, while a thread written in another language does not need sleep?
If Ide=Lazarus 2.0.10 32 Bit and Os=Win 10 Home 64 Bit then Get up and do something useful! Because God is the helper of those who start again;

MarkMLl

  • Hero Member
  • *****
  • Posts: 3670
Re: Synchronize not working
« Reply #50 on: November 26, 2021, 07:57:15 pm »
MarkMLl and Warfley and 440bx  thank you very much for the enlightening information.
A little off topic:
I have a lot of questions in my mind, one of them is;
Does the running speed of threads change according to the language they are written in?
So is it possible that a thread written in a different language needs sleep, while a thread written in another language does not need sleep?

Hypothetically: if a program were written in a reincarnation of BASIC which guaranteed that it behaved the same as some particular model of 8-bit home computer, then it would be reasonable for this to be implemented by intercalating a sleep() after every few emulated opcodes.

Apart from that... I'd be surprised if the language per se made any difference. I'd be surprised if the OS or the CPU said "Aha, I can see this is a non-standard calling convention so I'm going to insert execution delays"... and then I'd go looking for an alternative platform, since that almost certainly indicated something dodgy going on. And in that extreme case, while the OS might insert sleep(t) I'd not expect the CPU to, since in the general case the CPU is thread-agnostic (special cases: Sun "Niagara", Intel "Hyperthreads", and similar now implemented by others... but I'm not sure whether the CPU can act as a scheduler there other than as dictated by the cache being filled).

Again hypothetically: there have been attempts at various dataflow languages and there was also Mystic Pascal, many if not all of which had some measure of multithreading implicit to their implementation and which /might/ have needed explicit delays to allow background activities to complete. However since that would not have been implemented by the OS (they generally predate the concept of OS-supported threads significantly) it would be strictly incorrect to say that sleep() was being used. Ditto for coroutines in Modula-2 etc.

Finally, as a counterexample, there have been cases where development environments recognised that they were being asked to compile a benchmark such as- notoriously- the Sieve of Eratosthenes, and messed about with the code generation to make the result look good. Also there are languages such as APL which can do some quite surprising things to improve apparent efficacy... note that I'm not saying efficiency here since it is a regrettable fact that people who promoted them were oblivious to real-World constraints like finite memory.

Hope that isn't too vague. I suggest that further discussion might be better continued in the "Other" topic.

Later, rereading:

"So is it possible that a thread written in a different language needs sleep, while a thread written in another language does not need sleep?"

Strictly speaking, there is no reason why code written in one language should be faster or slower than equivalent code written in a different language. However different /implementations/ may behave very differently, e.g. if one is highly-optimised native code while another is purely interpretive.

And if the question really is "is an *explicit* sleep() ever needed to get threads to synchronise", the unavoidable answer is "if that is the case you're doing something badly wrong".

However, "doing it right" might involve various synchronisation or IPC (Inter-Process Communication) facilities provided by the runtime library or the OS, and they will almost certainly have *implicit* invocations of sleep() internally.

Alternatively, if the question is about e.g. implementing a communications protocol where a fragment in Pascal might look like this:

Code: Pascal  [Select][+][-]
  1. const
  2.   turnaround= 50;
  3.  
  4. begin
  5.   SerSetRTS(serHandle, true);
  6.   Sleep(turnaround);                    // Artificial pacing
  7.   try
  8. ...
  9.     while not (ignoreCts or SerGetCts(serHandle)) do
  10.       Sleep(10);
  11. ...
  12.   finally
  13.     SerDrain(serHandle);
  14.     Sleep(turnaround);                  // Artificial pacing
  15.     SerSetRTS(serHandle, false);
  16.     Sleep(turnaround)                   // Artificial pacing
  17.   end
  18. end { TSerialThread.sendMessage } ;
  19.  

I think it would be incorrect to re-implement it in e.g. interpreted BASIC without doing /something/ to ensure that the turnaround time mandated by the protocol was observed... even if the current BASIC implementation and platform weren't fast enough to make it an issue (remember the number of programs with timing that went wildly wrong when the IBM AT came out?).

MarkMLl
« Last Edit: November 26, 2021, 11:40:54 pm by MarkMLl »
MT+86 & Turbo Pascal v1 on CCP/M-86, multitasking with LAN & graphics in 128Kb.
Pet hate: people who boast about the size and sophistication of their computer.
GitHub repositories: https://github.com/MarkMLl?tab=repositories

loaded

  • Sr. Member
  • ****
  • Posts: 437
Re: Synchronize not working
« Reply #51 on: November 26, 2021, 08:20:51 pm »
MarkMLl, thank you very much for taking your precious time to reply. I will try to understand what you wrote by giving up a little sleep.
For now at least I learned about the Sieve of Eratosthenes.
If Ide=Lazarus 2.0.10 32 Bit and Os=Win 10 Home 64 Bit then Get up and do something useful! Because God is the helper of those who start again;

Warfley

  • Hero Member
  • *****
  • Posts: 654
Re: Synchronize not working
« Reply #52 on: November 27, 2021, 02:07:31 pm »
A few more practical examples for sleep:
If you write a Game that, due to very low graphic requirements runs crazy fast on modern hardware (like 500fps or so) you can use sleep to slow down the process to limit the framerate to 60fps.

Or, if you want to write a TCP connect with a timeout, you need to do it non blocking. Normal, blocking sockets will just take your thread off the CPU until the operation was successful. So you would require a second thread to kill the connect on a timeout, which is a waste of resource. Rather you can use non-blocking sockets, which will return immediatly each call with a message that the call didn't finish. You can then just wait until that call finished and continue:
Code: Pascal  [Select][+][-]
  1. function TimeoutConnect(Socket: TSocket; Addr: String; Port: Integer; Timeout: Int64): Boolean;
  2. var
  3.   StartTime: Int64;
  4. begin
  5.   StartTime := GetTickCount64;
  6.   Result := Connect(Socket, addr, port);
  7.   if not Result and SocketWasBlocking then
  8.   begin
  9.     while (GetTickCount - Start) < TimeOut do
  10.       if ConnectionEstablished(Socket) then
  11.          Exit(True)
  12.       else
  13.         Sleep(10);
  14.     Result := False;
  15.   end;
  16. end.
This way you don't waste CPU time (which just costs energy)

So there are a few use-cases for sleep

Mr.Madguy

  • Hero Member
  • *****
  • Posts: 748
Re: Synchronize not working
« Reply #53 on: November 27, 2021, 02:38:30 pm »
My opinion: yeah, it can be harder to implement, but in such cases, when one needs async processes and API doesn't provide some built-in event mechanism - it's better to use threads. It's very bad idea to use any blocking API like Sleep in modern "passive" applications. It will most likely cause waste of CPU resources. Some cycles would be "lost" due to "sleeping" overhead and some cycles would be wasted due to constantly "waking up" to check external state.  It's always better to use something like WaitForSingleObject or WaitForMultipleObjects inside thread. In this case there would be three methods to unblock thread: 1) You can unblock it by your own event - for example to terminate thread. 2) You can unblock it on timeout, if you need to do some tasks from time to time. 3) End you can unblock it by actual event, you need to wait for. This way thread will only consume CPU time, when it's really needed. And this way it could be woken at any moment without any overhead.

At the end it's not that hard:
Code: Pascal  [Select][+][-]
  1.   TPausableThread = class(TThread)
  2.     protected
  3.       PauseEvent:TEvent;
  4.       procedure Pause(ATimeout:Cardinal);
  5.     public
  6.       constructor Create;
  7.       destructor Destroy;override;
  8.       procedure StopAndWaitFor;
  9.   end;
  10.  
  11.   TPingThread = class(TPausableThread)
  12.     protected
  13.       Gateway:String;
  14.       PingSend:TPingSend;
  15.       FOnPingFailed:TNotifyEvent;
  16.       procedure Execute;override;
  17.       procedure DoPingFailed;
  18.     public
  19.       constructor Create(AGateway:String);
  20.       destructor Destroy;override;
  21.       property OnPingFailed:TNotifyEvent read FOnPingFailed write FOnPingFailed;
  22.   end;
  23.  
  24. {TPausableThread}
  25.  
  26. constructor TPausableThread.Create;
  27. begin
  28.   inherited Create(True);
  29.   PauseEvent := TSimpleEvent.Create;
  30. end;
  31.  
  32. destructor TPausableThread.Destroy;
  33. begin
  34.   PauseEvent.Free;
  35.   inherited Destroy;
  36. end;
  37.  
  38. procedure TPausableThread.StopAndWaitFor;
  39. begin
  40.   Terminate;
  41.   PauseEvent.SetEvent;
  42.   WaitFor;
  43. end;
  44.  
  45. procedure TPausableThread.Pause(ATimeout:Cardinal);
  46. begin
  47.   PauseEvent.WaitFor(ATimeout);
  48. end;
  49.  
  50. {TPingThread}
  51.  
  52. constructor TPingThread.Create(AGateway:String);
  53. begin
  54.   inherited Create;
  55.   Gateway := AGateway;
  56.   PingSend := TPingSend.Create;
  57. end;
  58.  
  59. destructor TPingThread.Destroy;
  60. begin
  61.   PingSend.Free;
  62.   inherited Destroy;
  63. end;
  64.  
  65. procedure TPingThread.Execute;
  66. begin
  67.   while not Terminated do begin
  68.     if not PingSend.Ping(Gateway) then begin
  69.       Queue(@DoPingFailed);
  70.     end;
  71.     Pause(60000);
  72.   end;
  73. end;
  74.  
  75. procedure TPingThread.DoPingFailed;
  76. begin
  77.   if Assigned(OnPingFailed) then OnPingFailed(Self);
  78. end;
  79.  
« Last Edit: November 27, 2021, 02:52:09 pm by Mr.Madguy »
29.12.2021 - migration to DynamicData 4.1 is completed - complete overhaul of data access driver.
My project still requires full Delphi 2009 support to be ported to Lazarus.
It's time to finally do it, because Delphi 2009 is 12 years old.

MarkMLl

  • Hero Member
  • *****
  • Posts: 3670
Re: Synchronize not working
« Reply #54 on: November 27, 2021, 04:03:48 pm »
But all you've done there is replace Sleep() in a thread with Pause(), and wrapped it with event handling.

Even if you were waiting for a library routine to complete something, there would still be something comparable with Sleep() involved. It might not be Sleep() as such since it might have separate code in the OS to detect the termination condition, but there would still be something which had the effect "suspend this thread but check occasionally to see if some condition is satisfied".

Interestingly, back in the earliest days of multitasking OSes, one ALGOL-60 implementation of which I'm aware had a procedure which allowed an (unprivileged) user's program to suspend itself until an absolute memory address was changed. I can't remember what this was supposed to be used for, but I'm sure that it really was a nightmare since a program could be moved to disc and then back to core while running at which point there was no guarantee that the absolute address being watched was still in its own storage area.

MarkMLl
MT+86 & Turbo Pascal v1 on CCP/M-86, multitasking with LAN & graphics in 128Kb.
Pet hate: people who boast about the size and sophistication of their computer.
GitHub repositories: https://github.com/MarkMLl?tab=repositories

Mr.Madguy

  • Hero Member
  • *****
  • Posts: 748
Re: Synchronize not working
« Reply #55 on: November 27, 2021, 04:37:23 pm »
But all you've done there is replace Sleep() in a thread with Pause(), and wrapped it with event handling.

Even if you were waiting for a library routine to complete something, there would still be something comparable with Sleep() involved. It might not be Sleep() as such since it might have separate code in the OS to detect the termination condition, but there would still be something which had the effect "suspend this thread but check occasionally to see if some condition is satisfied".

Interestingly, back in the earliest days of multitasking OSes, one ALGOL-60 implementation of which I'm aware had a procedure which allowed an (unprivileged) user's program to suspend itself until an absolute memory address was changed. I can't remember what this was supposed to be used for, but I'm sure that it really was a nightmare since a program could be moved to disc and then back to core while running at which point there was no guarantee that the absolute address being watched was still in its own storage area.

MarkMLl
This example shows all three conditions, i.e. timeout condition is one of them. But it's not necessary. The biggest difference here - you can interrupt pause manually, that is done in StopAndWaitFor not to wait for whole minute for thread to terminate. So block can actually be infinite. No reason to unblock from time to time to check thread termination condition.
29.12.2021 - migration to DynamicData 4.1 is completed - complete overhaul of data access driver.
My project still requires full Delphi 2009 support to be ported to Lazarus.
It's time to finally do it, because Delphi 2009 is 12 years old.

Warfley

  • Hero Member
  • *****
  • Posts: 654
Re: Synchronize not working
« Reply #56 on: November 27, 2021, 07:35:01 pm »
My opinion: yeah, it can be harder to implement, but in such cases, when one needs async processes and API doesn't provide some built-in event mechanism - it's better to use threads. It's very bad idea to use any blocking API like Sleep in modern "passive" applications. It will most likely cause waste of CPU resources. Some cycles would be "lost" due to "sleeping" overhead and some cycles would be wasted due to constantly "waking up" to check external state.  It's always better to use something like WaitForSingleObject or WaitForMultipleObjects inside thread. In this case there would be three methods to unblock thread: 1) You can unblock it by your own event - for example to terminate thread. 2) You can unblock it on timeout, if you need to do some tasks from time to time. 3) End you can unblock it by actual event, you need to wait for. This way thread will only consume CPU time, when it's really needed. And this way it could be woken at any moment without any overhead.

No, using threads only makes sense for long living processes. If you only need a timeout for a single operation (like the connect in my example), the creation of the thread structure in the operating system, the allocation of the stack memory and the management in the scheduler will add far more overhead to the system then having a polling mechanism.

Also, your code does polling like mine, it is just hidden. TThread.Queue just adds an object to a thread safe queue, which requires the main thread to poll in regular intervals. So you basically do the exact same thing as I do, just with much more overhead and 4 times the amount of code.

But let's stick to my example with the timeout on connect. Because this is a very simple example you might come accross quite easiely and it has no obvious solution.
I see exactly two possibilities, either do it like I did, or create a new watchdog thread, which will after the timout occured invalidate the socket which results in the abortion of the blocking connect call.
Adding a watchdog thread, introduces much more code, runtime overhead (unless you do thread caching when doing this multiple times, which in turn requires even more code) and is pretty hard to understand, not to mention that invalidating the socket to stop the blocking call is really ugly. I don't see any advantages with this
« Last Edit: November 27, 2021, 07:52:07 pm by Warfley »

MarkMLl

  • Hero Member
  • *****
  • Posts: 3670
Re: Synchronize not working
« Reply #57 on: November 27, 2021, 08:04:31 pm »
No, using threads only makes sense for long living processes. If you only need a timeout for a single operation (like the connect in my example), the creation of the thread structure in the operating system, the allocation of the stack memory and the management in the scheduler will add far more overhead to the system then having a polling mechanism.

It does have to be said though that unix doctrine seems to fork processes with gay abandon... and that would be expected to be substantially more complex that thread creation since it involves creating a new addressspace and either copying the existing process or setting up enough MMU state that it can be COWed on demand.

Noting of course that the OS/2 and Win32 APIs were heavily oriented towards asynchronous syscalls with callbacks (hence, reputedly, Horace Cutler's rant about byte-by-byte I/O in UNIX), I suppose that one possibility would be to emulate that by having a single background thread to which every blocking activity could be enqueued. That would obviously need to be used carefully, and there would obviously still be a sleep() or equivalent call somewhere.

MarkMLl



MT+86 & Turbo Pascal v1 on CCP/M-86, multitasking with LAN & graphics in 128Kb.
Pet hate: people who boast about the size and sophistication of their computer.
GitHub repositories: https://github.com/MarkMLl?tab=repositories

Warfley

  • Hero Member
  • *****
  • Posts: 654
Re: Synchronize not working
« Reply #58 on: November 27, 2021, 08:21:29 pm »
It does have to be said though that unix doctrine seems to fork processes with gay abandon... and that would be expected to be substantially more complex that thread creation since it involves creating a new addressspace and either copying the existing process or setting up enough MMU state that it can be COWed on demand.
Yes forking is really slow, around 1MS. Creating a fork is usually an order of magnitude faster at like 0.1 to 0.5 ms but this is still a massive overhead.
Also there are alternatives to forking by using clone, which can be configured to behave differently, e.g. work in the same address space or not copy the file descriptors and so on. But it is usally adviced not to use it (because it is very complex) but rather use functions like posix_spawnp which use clone internally. https://www.man7.org/linux/man-pages/man2/clone.2.html
On Linux a thread is btw nothing more then a clone call with a certain configuration.

Especially fork gets slower the more often you call it from the same process. I once ran into the problem that a single fork took more than 1 minute. Solved it by using vfork instead.

Noting of course that the OS/2 and Win32 APIs were heavily oriented towards asynchronous syscalls with callbacks (hence, reputedly, Horace Cutler's rant about byte-by-byte I/O in UNIX), I suppose that one possibility would be to emulate that by having a single background thread to which every blocking activity could be enqueued. That would obviously need to be used carefully, and there would obviously still be a sleep() or equivalent call somewhere.
Yes the WinAPI also provides an asynchronous connect for sockets, but if you want to be cross compatible and stick to the Berkley Sockets API you won't get around either a watchdog thread or a polling sleep loop.

When handling large parallel systems managing millions of connections it is even advised to not use threading and events managed by the system (and scheduled by the scheduler), because it is to slow. There it is best to have a few threads, each managing a few thousand connections, polling them at a high frequency and when having nothing to do do busy waiting. But then you also don't use sleep

MarkMLl

  • Hero Member
  • *****
  • Posts: 3670
Re: Synchronize not working
« Reply #59 on: November 27, 2021, 11:11:51 pm »
On Linux a thread is btw nothing more then a clone call with a certain configuration.

I always find it useful to consider things like this, in particular what (things that look like) syscalls actually transfer to system mode or have the overhead of switching from the application thread to the scheduler. But I've a microkernel to my name :-)

Quote
Especially fork gets slower the more often you call it from the same process. I once ran into the problem that a single fork took more than 1 minute. Solved it by using vfork instead.

Ouch :-(

Quote
When handling large parallel systems managing millions of connections it is even advised to not use threading and events managed by the system (and scheduled by the scheduler), because it is to slow. There it is best to have a few threads, each managing a few thousand connections, polling them at a high frequency and when having nothing to do do busy waiting. But then you also don't use sleep

The ISP we've used for 20 years designs and sells their own routers. Even their frontline techies are good for a technology discussion :-) https://hackaday.com/2017/12/14/adsl-robustness-verified-by-running-over-wet-string/ https://www.firebrick.co.uk/

MarkMLl
MT+86 & Turbo Pascal v1 on CCP/M-86, multitasking with LAN & graphics in 128Kb.
Pet hate: people who boast about the size and sophistication of their computer.
GitHub repositories: https://github.com/MarkMLl?tab=repositories

 

TinyPortal © 2005-2018