Unleashed Pascal (async/await, parallel for, match, string interpolation & more)

[hedgehog]

I tried the future

Code: Pascal  [Select][+][-]

1. procedure DownloadBigFile(
2.   OnStartEvent, OnProcessEvent, OnDoneEvent:  TNotifyEvent);
3. begin
4.   if Assigned(OnStartEvent) then OnStartEvent(nil);
5.   Sleep(1000);
6.   if Assigned(OnProcessEvent) then OnProcessEvent(nil);
7.   Sleep(1000);
8.   if Assigned(OnDoneEvent) then OnDoneEvent(nil);
9. end;
10.  
11. procedure tform1.OnStart(sender: tobject);
12. begin
13.   memo1.Append('Start.');
14. end;
15.  
16. procedure tform1.OnDone(sender: tobject);
17. begin
18.   memo1.Append('Done!');
19. end;
20.  
21. procedure tform1.OnProcess(sender: tobject);
22. begin
23.   memo1.Append('Downloading...');
24. end;
25.  
26. procedure tform1.btndownloadclick(sender: tobject);
27. begin
28.   async DownloadBigFile(@OnStart, @OnProcess, @OnDone);
29.   memo1.Append('Return to main loop');
30. end;

I expected to get:

Return to main loop
Start.
Downloading...
Done!

Actually:

Return to main loop
Done!
Downloading...
Start.

Did something go wrong, or is there something I don't understand?

[creaothceann]

It seems like packed records in an union doesn't work correctly?

Code: Pascal  [Select][+][-]

1. type
2.         TMyStruct_1 = packed record  align 8
3.                 private  // Wrong syntax highlighting? Writing "align 8;" fixes it.  ( see https://i.imgur.com/PtKUfYn.png )
4.                 procedure _Test;
5.  
6.                 public
7.                 union  // Wrong syntax highlighting?
8.                         Value : QWord;
9.                         Bytes : array[0..7] of Byte;
10.                 end;
11.         end;
12.  
13.  
14.         TMyStruct_2 = packed record  align 8
15.                 private
16.                 procedure _Test;
17.  
18.                 public
19.                 union
20.                         Value : QWord;
21.                         packed record
22.                                 b0, b1, b2, b3, b4, b5, b6, b7 : Byte;
23.                         end;
24.  
25.                 end;
26.         end;
27.  
28.  
29.         TMyStruct_3 = packed record  align 8
30.                 private
31.                 type
32.                         _Bytes = packed record
33.                                 b0, b1, b2, b3, b4, b5, b6, b7 : Byte;
34.                         end;
35.  
36.                 procedure _Test;
37.  
38.                 public
39.                 union
40.                         Value : QWord;
41.                         embed _Bytes;
42.                 end;
43.         end;
44.  
45.  
46. procedure TMyStruct_1._Test;  begin  Bytes[0] := 0;  end;
47. procedure TMyStruct_2._Test;  begin  b0       := 0;  end;  // 'Error: Identifier not found "b0"'
48. procedure TMyStruct_3._Test;  begin  b0       := 0;  end;  // 'Error: Identifier not found "b0"'

[Fibonacci]

Took a short nap and came back to a flood of bug reports

Well... I guess that's what happens when a project starts gaining momentum - more people use it, more edge cases get exercised, and the bugs start coming out of hiding.

I'd still prefer if bug reports were opened as GitHub issues whenever possible. It's much easier for me to keep track of them, prioritize them, and make sure nothing gets lost. But it's all good - keep them coming

Btw. I tried the installer on a PC with a 4k display too, with the Windows font scaling set to 150%. It makes the text a bit too big, but it still works.

Will look into it later.

I tried the future

Code: Pascal  [Select][+][-]

1. procedure DownloadBigFile(
2.   OnStartEvent, OnProcessEvent, OnDoneEvent:  TNotifyEvent);
3. begin
4.   if Assigned(OnStartEvent) then OnStartEvent(nil);
5.   Sleep(1000);
6.   if Assigned(OnProcessEvent) then OnProcessEvent(nil);
7.   Sleep(1000);
8.   if Assigned(OnDoneEvent) then OnDoneEvent(nil);
9. end;
10.  
11. procedure tform1.OnStart(sender: tobject);
12. begin
13.   memo1.Append('Start.');
14. end;
15.  
16. procedure tform1.OnDone(sender: tobject);
17. begin
18.   memo1.Append('Done!');
19. end;
20.  
21. procedure tform1.OnProcess(sender: tobject);
22. begin
23.   memo1.Append('Downloading...');
24. end;
25.  
26. procedure tform1.btndownloadclick(sender: tobject);
27. begin
28.   async DownloadBigFile(@OnStart, @OnProcess, @OnDone);
29.   memo1.Append('Return to main loop');
30. end;

I expected to get:

Return to main loop
Start.
Downloading...
Done!

Actually:

Return to main loop
Done!
Downloading...
Start.

Did something go wrong, or is there something I don't understand?

Real bug, thanks for the report. Fixed. Two separate things going on in your code though.

1) Compiler bug. async F(a, b, c) handed the arguments to the worker in reverse, so DownloadBigFile(@OnStart, @OnProcess, @OnDone) actually ran as DownloadBigFile(@OnDone, @OnProcess, @OnStart). Any call form with two or more arguments hit it. Arguments of different types got caught by the type checker, but three TNotifyEvent parameters swap silently, which is why it looked like the callbacks fired backwards.

2) Not a bug, just threading - and it would still bite you after the fix. The routine runs on a worker thread, and so does everything it calls, including your three callbacks. memo1.Append is a cross-thread LCL call, which is never safe no matter how it looks.

Good timing though: I just added a sync keyword for exactly this. It's the mirror of async - it hands a statement (or a begin..end block) to the main thread and waits for it to run there. So the fix is one word per callback:

Code: Pascal  [Select][+][-]

1. procedure tform1.OnStart(sender: tobject);
2. begin
3.   sync memo1.Append('Start.');
4. end;

Same for OnProcess and OnDone. A single statement needs no begin..end. Your btndownloadclick doesn't await the future, so it returns straight to the message loop, which pumps the queue and runs the sync bodies - nothing else to do on your side. Output comes out in order:

Return to main loop
Start.
Downloading...
Done!

sync lowers to TThread.Synchronize under the hood, so it needs the Classes unit (an LCL form already has it). One rule to remember: don't await a future on the main thread while its worker is sitting in sync - each would wait for the other. Fire-and-forget like yours is fine.

It seems like packed records in an union doesn't work correctly?

Fixed both: syntax highlighting and the "Identifier not found" error (the workaround was to use Self, BTW).

[hedgehog]

Hi, Fibonacci
First of all, I want to thank you. I really enjoy exploring these new features.
If you don't mind, I'd like to give my amateur opinion.
The most important thing in all this is the future/async/await.  (Why isn't async(...).then (@callback) ?)
In second place is string interpolation! (For some reason, I rarely use the Format() function.)

2) Not a bug, just threading - and it would still bite you after the fix. The routine runs on a worker thread, and so does everything it calls, including your three callbacks. memo1.Append is a cross-thread LCL call, which is never safe no matter how it looks

Of course, you're right. It was a quick and dirty test.

procedure tform1.OnStart(sender: tobject);
begin
  sync memo1.Append('Start.');
end;

But what if this is the case? Will it work?

Code: Pascal  [Select][+][-]

1. procedure tform1.OnStart(sender: tobject);
2. begin
3.   memo1.Append('Start.');
4. end
5.  
6. procedure DownloadBigFile(
7.   OnStartEvent, OnProcessEvent, OnDoneEvent:  TNotifyEvent);
8. begin
9.   if Assigned(OnStartEvent) then sync OnStartEvent(nil);
10.   ...
11. end;

[Fibonacci]

Hi, Fibonacci
First of all, I want to thank you. I really enjoy exploring these new features.

Thanks, I appreciate the kind words. I'm glad you're enjoying the new features.

The most important thing in all this is the future/async/await.  (Why isn't async(...).then (@callback) ?)

This is C++ std::async, not JS promises: one async = one thread, one await = one join, and deliberately no event loop or scheduler. .then only really makes sense when something is responsible for scheduling continuations. And the moment you add .then, people reasonably expect the rest of the machinery: chaining, .all/.race, error propagation down the chain. Right now it's small and easy to reason about, and I'd rather keep it that way. Even standard C++ std::future never got .then, for essentially the same reason.

If you want a continuation, you already can - just spawn one that waits on the future:

Code: Pascal  [Select][+][-]

1. var f := async DownloadBigFile(@OnStart, @OnProcess, @OnDone);
2. async begin
3.   while not f.Done do sleep(5);
4.   // your "then" code here
5. end;

That block runs on a worker thread, so if the "then" code touches the GUI, wrap that part in sync too.

But what if this is the case? Will it work?

Code: Pascal  [Select][+][-]

1. procedure tform1.OnStart(sender: tobject);
2. begin
3.   memo1.Append('Start.');
4. end
5.  
6. procedure DownloadBigFile(
7.   OnStartEvent, OnProcessEvent, OnDoneEvent:  TNotifyEvent);
8. begin
9.   if Assigned(OnStartEvent) then sync OnStartEvent(nil);
10.   ...
11. end;

Yes, and honestly that's the cleaner place for it. Put sync once at each call site in the worker, and your callbacks stay plain; they don't need to know they're being marshalled. One sync per call beats sprinkling it inside every handler.

Just remember who pumps the queue:

- LCL app: the main message loop calls CheckSynchronize for you - nothing to do, it just works.
- Console app: no loop, so the main thread has to pump it. Either call CheckSynchronize periodically in your own main loop, or hold the future and wait on it without awaiting:

Code: Pascal  [Select][+][-]

1. var f := async DownloadBigFile(@OnStart, @OnProcess, @OnDone);
2. while not f.Done do CheckSynchronize(10); // runs the queued sync bodies on this thread

Don't await f on the main thread here - the worker is parked in sync waiting for the main thread, and await would park the main thread waiting for the worker. Mutual deadlock. Pump, don't await.

[creaothceann]

There's still something wrong with packed records and unions. I've opened an issue here.

It's fixed.

[Fibonacci]

Btw. I tried the installer on a PC with a 4k display too, with the Windows font scaling set to 150%. It makes the text a bit too big, but it still works.

Should be fixed in the nightly: https://github.com/fpc-unleashed/installer/releases/tag/nightly

Let me know if it looks right now, whenever you get a chance.

It's time to get familiar with the Anchor Editor.

Nope, it was just one label missing AutoSize=True, plus Scaled=True on the form

[Fibonacci]

New feature: inline out-variables (var x / _)

On by default in {$mode unleashed}. Outside: {$modeswitch outvar}.

Docs: docs/out-var.md

Declare a variable inline in an out argument, or discard the value entirely, instead of pre-declaring a throwaway local for every output. Two forms: var name declares a fresh variable whose type is inferred from the matched out parameter and remains in scope in the enclosing block; a bare _ discards the value (the call still executes; the compiler passes a hidden local that cannot be named).

Code: Pascal  [Select][+][-]

1. program outvar_demo;
2.  
3. {$mode unleashed}
4.  
5. function TryHalve(x: integer; out half: integer): boolean;
6. begin
7.   if Odd(x) then exit(false);
8.   half := x div 2;
9.   result := true;
10. end;
11.  
12. procedure SplitByte(w: word; out lo, hi: byte);
13. begin
14.   lo := w and $FF;
15.   hi := w shr 8;
16. end;
17.  
18. begin
19.   // inline out-var: `half` declared here, type inferred, stays in scope
20.   if TryHalve(10, var half) then
21.     writeln('half = ', half);              // half = 5
22.  
23.   // declare multiple variables at once
24.   SplitByte($ABCD, var lo, var hi);
25.   writeln($'lo={lo} hi={hi}');             // lo=205 hi=171
26.  
27.   // discard the one you don't want
28.   SplitByte($1337, _, var top);
29.   writeln('top = ', HexStr(top, 2));       // top = 13
30.  
31.   // discard all - run for side effect only
32.   SplitByte($FFFF, _, _);
33.  
34.   writeln('done');
35.   readln;
36. end.

Notes

• Allowed only for out parameters. For var, const, or value parameters var x and _ are rejected - a var parameter is read on entry, so capturing it fresh would hide a bug; that restriction is deliberate.
• var name is a real declaration in the enclosing block (routine body, nested begin..end, or main block) - it remains in scope until the end of that block like any inline var, and a name already in scope is a duplicate-identifier error.
• No explicit type is allowed - it is inferred from the parameter after overload resolution. If two overloads differ only in the out type, the call is ambiguous; disambiguate it with another typed argument.
• Managed types (string, dynamic array, interface, Variant) are ordinary locals: initialized on entry, finalized at scope end - including a _ discard inside a loop, so no leaks.
• Backward compatible: _ is a discard only when no identifier _ is in scope. Declare var _: integer and _ means that variable everywhere, whether the modeswitch is enabled or not. Intrinsics (Write/Read/Str) have no out params, so _ is never treated as a discard there.

[creaothceann]

Btw. I tried the installer on a PC with a 4k display too, with the Windows font scaling set to 150%. It makes the text a bit too big, but it still works.

Should be fixed in the nightly: https://github.com/fpc-unleashed/installer/releases/tag/nightly

Let me know if it looks right now, whenever you get a chance.

Looks fine: https://i.imgur.com/wFuKTR5.png


Btw. here's a fresh bug

Code: Pascal  [Select][+][-]

1. type
2.         TMyRecord = record
3. //              union
4. //                      YX : Word;
5.                         packed record
6.                                 _X, _Y : Byte;
7.                         end;
8. //              end;
9.                 property X : Byte read _X;  // Error: Unknown class field or method identifier "_X"
10.         end;

[Fibonacci]

Btw. here's a fresh bug

Fixed. Nice catch. You're a good tester 

[srvaldez]

good day Fibonacci 
I just tried FPC unleashed with a pet project and the performance increased 75% 
Q: when installing a new version, do I uninstall the previous version ?

[flowCRANE]

I just tried FPC unleashed with a pet project and the performance increased 75% 

I haven't read all the posts in this thread, but what's the actual situation regarding machine code optimization in Unleashed FPC? Assuming that I’m using -O3 optimizations (i.e., aggressive but safe) in the FPC stock, is Unleashed FPC capable of generating faster machine code if I use the aggressive and safe optimizations it supports—both standard and newly introduced ones (as far as I know, Unleashed supports these)? Has anyone tested this using any more comprehensive benchmarks?

[Thaddy]

As far as I know there are no optimizations. It is the same code generator. There are just extensive syntax additions. Code generation is already highly optimized, certainly on x86_64 and i386. (Better than Delphi32 and rivals other compilers)

[Fibonacci]

good day Fibonacci 
I just tried FPC unleashed with a pet project and the performance increased 75% 
Q: when installing a new version, do I uninstall the previous version ?

Good day to you too

But... how? Unleashed is mostly about new language features rather than performance. My guess is that you were using the latest stable release (FPC 3.2.2), which is over 5 years old, and switching to the current trunk-based compiler got you all the optimizations that have accumulated since then. Or did you make use of any of the threading features (parallel for, futures)?

As for installing: there's no need to uninstall the previous version. Just run the installer again and point it to the existing installation directory. The "Install" button will automatically change to "Update", preserving your configuration.

[Thaddy]

And you should try it!