Unleashed Pascal (async/await, parallel for, match, string interpolation & more)

[Thaddy]

That is a pity because it will loose its ancestor.
It is also a pity because this fork goes tooooo quick as opposed to fpc being tooooo slow in release cycles.

There is a risk this becomes niche before it even started.
(And apart from inline vars! it has proper commits that should have been in fpc years ago)

Basically both sides of the argument are doing a good job, but (both sides) be careful what you wish for. FPC: way too slow, Unleashed: way too fast.

Well, I don't care, I simply use both, I don't have to care:pensionado. (pension payed for - to a large extend - by Pascal, the rest being management)

[Fibonacci]

@Thaddy

To be clear about one thing: Unleashed Pascal stays a fork. It is not detaching from anything, and I am not going my own way with the codebase. Upstream syncs continue exactly as before - I pull FPC trunk and Lazarus main, and I intend to keep doing that.

The rename is only a rename. I was "asked" not to use the FPC name, so I stopped using it. The name on the box changed; nothing under the lid did.

So the ancestor isn't lost - it is still literally the same tree with my commits on top, and every upstream fix still lands here. It just doesn't say "FPC" on it anymore.


On "too fast": fair concern, and I hear it. But everything is opt-in behind {$mode unleashed}, and the sync with upstream is what keeps the fast part from drifting into its own universe.

[Fibonacci]

Unleashed Pascal is now installable via fpcupdeluxe - no manual config

Until now, installing Unleashed Pascal through fpcupdeluxe meant hand-editing an .ini file first. That's over: as of fpcupdeluxe v2.4.0jpu the unleashed compiler and IDE are available straight from the UI - pick the unleashed entries, click, done.

Big thanks to @DonAlfredo for adding it.

Download: https://github.com/LongDirtyAnimAlf/fpcupdeluxe/releases/tag/v2.4.0jpu

[Mike.Cornflake]

G'day,

Just wanted to offer my congratulations on this project. And I'm really enjoying the conversations in this part of the forum.  I'm learning more about Pascal by watching you push the capabilities.

Keep it up :-)

Mike

[creaothceann]

Sorry, wall of text ahead...

Until recently, my emulator's main loop looked like this:

Code: Pascal  [Select][+][-]

1. procedure CPU.Run(var stop : boolean);
2. label
3.         Opcode[u8];  // 256 opcodes
4. begin
5.         goto Opcode[IR];  // IR contains the current bytecode value
6.  
7.         Opcode[$00]:
8.                 {cycle 1}
9.                 {cycle 2}
10.                 {.......}
11.                 {cycle 8}  Fetch_IR;  if stop then exit;  goto Opcode[IR];
12.  
13.         Opcode[$01]:
14.                 {cycle 1}
15.                 {cycle 2}
16.                 {.......}
17.                 {cycle 6}  Fetch_IR;  goto Opcode[IR];
18.         {...}
19. end;

This worked reasonably well: each instruction loads the next instruction's opcode and jumps to its handler, turning the code into an (almost) endless loop.

Recently however it became necessary to emulate each instruction one cycle at a time:

Code: Pascal  [Select][+][-]

1. procedure CPU.Step;
2. label
3.         // labels for 256 opcodes
4.         Opcode[u8];
5.         // labels for each cycle of an opcode
6.         BRK     [0..7],  // 8 cycles for opcode $00: "BRK"
7.         ORA_d_x [0..5],  // 6 cycles for opcode $01: "ORA (d,x)"
8.         {...}
9.         SBC_al_x[0..4];  // 5 cycles for opcode $FF: "SBC al,x"
10. var
11.         Cycle : uint;
12. begin
13.         Cycle := T;  T := Cycle + 1;  goto Opcode[IR];  // first dispatch
14.  
15.         // opcode handlers: second dispatch
16.         Opcode[$00]:  goto BRK     [Cycle];
17.         Opcode[$01]:  goto ORA_d_x [Cycle];
18.         {...}
19.         Opcode[$FF]:  goto SBC_al_x[Cycle];
20.  
21.         // opcode cycle handlers
22.         BRK[0]:  {...}                   exit;
23.         BRK[1]:  {...}                   exit;
24.         {...}
25.         BRK[7]:  T := 0;  Fetch_Opcode;  exit;
26.  
27.         //----------------
28.         ORA_d_x[0]:  {...}                   exit;
29.         ORA_d_x[1]:  {...}                   exit;
30.         {...}
31.         ORA_d_x[5]:  T := 0;  Fetch_Opcode;  exit;
32.  
33.         //----------------
34.         // {...}
35.  
36.         //----------------
37.         SBC_al_x[0]:  {...}                   exit;
38.         SBC_al_x[1]:  {...}                   exit;
39.         {...}
40.         SBC_al_x[4]:  T := 0;  Fetch_Opcode;  exit;
41. end;
42.  

This has a problem though: since a goto to indexed labels is basically a case-of behind the scenes, and the number of cycle labels is almost always relatively small, the compiler usually turns the "second dispatch" jump table (which would be only 1 jump) into a sequence of many subtractions and jumps (less desirable). Assuming @Fibonacci has no easy way to fix that "feature" of the compiler, it can still be manually worked around by increasing the number of labels per instruction (e.g. label BRK[0..15];) and leaving them somewhere, e.g. at the end of the subroutine.

I thought about creating my own "manual" case-of or computed goto, by storing the difference between a label's address and a base address (e.g. the first label in the group) in a static array of 16-bit words, and then reconstructing the full address later when needed via simple addition. Unfortunately this doesn't work: trying to take the address of an indexed label (in a for loop) results in Error: Illegal qualifier.

(In the interest of reducing the number of jumps I also briefly tried turning every cycle into its own function, in the hope that all these functions would simply return with a RET (which is basically free), but at least on Windows it seems that each and every function starts/ends with SEH (structured exception handling) and stack frame initialization/finalization. So I reverted it, and all these cycles still add another jump to the end of their function. Maybe SetJmp/LongJmp can help later as an optimization.)

- - -

Regarding indexed labels:

Code: Pascal  [Select][+][-]

1. label Op[u8];      // works fine
2. label Op[0..255];  // works fine
3. label Op[256];     // Error: Label used but not defined "Op[256]"

A bug, or on purpose?

[Fibonacci]

Regarding indexed labels:

Code: Pascal  [Select][+][-]

1. label Op[u8];      // works fine
2. label Op[0..255];  // works fine
3. label Op[256];     // Error: Label used but not defined "Op[256]"

A bug, or on purpose?

On purpose, though the docs could say it more clearly.

[256] is not a count - it is a value list that happens to have one value in it, so it declares a single label with index 256, nothing else. The supported forms are a range ([0..255]), an ordinal type ([u8]), a value list ([1, 2, 3]) or a mix ([1..3, 7]). There is no "N labels" form - which is why [u8] and [0..255] work and [256] does not do what it looks like it does.

While checking this I did find a real bug though. If the declaration only covers one index and the rest come in through lazy labels, the runtime dispatch silently collapses - every index jumps to the first label:

Code: Text  [Select][+][-]

1. label Op[4]          label Op[0..3]
2. i=0 -> label 0       i=0 -> label 0
3. i=1 -> label 0       i=1 -> label 1
4. i=2 -> label 0       i=2 -> label 2
5. i=3 -> label 0       i=3 -> label 3

No error, no warning, just wrong jumps. That one gets fixed.

I will get to the rest of your post later, a bit busy right now.

[creaothceann]

On purpose [...].

Ah, OK.

I did get some internal compiler errors when playing around with labels, but right now I can't seem to reproduce it again.

EDIT: writing "label Op[u8];" and only defining "Op[$00]:" in the code results in warnings Label not defined "Op[1]" to Label not defined "Op[255]", followed by Error: Internal error 2007050701.

- - -

Sorry for adding to the list when you're already busy...

Regarding goto with pointers (I didn't find that in the docs, only in the thread), I noticed that it can do pointer arithmetic:

Code: Pascal  [Select][+][-]

1. procedure CPU.Step;
2. label
3.         Init_Op, Init_Done;
4. static
5.         Delta_Op    : array[u8] of u16;
6.         initialized : bool = False;
7. var
8.         Cycle : u8;
9. begin
10.         if (not initialized) then goto Init_Op;
11.         Init_Done:
12.         {...}
13.         goto @Init_Op + Delta_Op[IR];

Nice, not sure if that's already a feature of Free Pascal itself. Usually I have to cast the pointer to PtrUInt first.

Btw. the Delta_Op above in line 13 is marked with the warning "does not seem to be initialized", even though it should be all zeroes at program start - which is what I do want. It seems there are 3 ways to suppress that warning, all rather lacking:

- Delta_Op[0] := 0;  // treats the entire array as initialized
- surrounding the entire subroutine with {$warn 5037 off}/{$warn 5037 on}
- specifying 256 zeroes in brackets behind the type

[Akira1364]

One thing that both trunk FPC and this still lack (but that nearly all other compilers for other languages can do nowadays) is the ability to inline function pointer calls. By which I mean `Foo(@Bar)` will never ever inline `Bar` under any circumstances. However the direct equivalent of that in e.g. C++ or even regular C certainly would with fairly standard compiler options. Is this something you've considered looking into?

[Fibonacci]

@creaothceann:

All of it is fixed on the devel branch - update to it (the installer lets you pin the branch) and check unleashed/docs/indexed-labels.md there, it was rewritten and now covers what previously only existed in this thread.

What changed, of the things you hit:

- The second dispatch gets a jump table. A goto over an indexed family is exempt from the case-size heuristic that turned small families into a compare chain, so BRK[0..7] is one table jump - no padding with unused labels.
- @Op[index] works, constant and runtime index both; a runtime index matching no member yields nil. That was your "Illegal qualifier", and the delta-table dispatch you described now builds without casting anything.
- goto on a pointer expression, arithmetic included, is documented, with your delta-table pattern as the example.
- Your internal error is gone. It reduced to a family with a declared-but-undefined member and a variable-index goto: codegen dereferenced a goto that had no label and crashed at -O2. An undefined member now falls through to the statement after the goto, as documented.
- label Op[u8] with only Op[$00] defined no longer warns 255 times. Undefined members of a family are legal by design, so they don't warn at all now. A constant-index goto naming an undefined member still errors.
- label Op[256] is a proper error now. The index spec is a set of values, never a count, so [256] declared the single label Op[256]. Use [256..256], a value list, or a type.
- Adding an index outside the range of an already generated variable-index goto is an error. Before, a lazily created label could land outside the dispatch and out-of-range indices silently fell through.

Btw. the Delta_Op above in line 13 is marked with the warning "does not seem to be initialized", even though it should be all zeroes at program start - which is what I do want. It seems there are 3 ways to suppress that warning, all rather lacking:

https://github.com/unleashedpascal/compiler/blob/main/unleashed/docs/zeroinit.md


@Akira1364:

Confirmed, and it's worse than "not inlined" - the target is already known and still called indirectly. At -O2:

Code: Pascal  [Select][+][-]

1. g := Doubler(21);         // movl $42,g
2. g := Apply(@Doubler, 21); // leaq Doubler(%rip),%rax
3.                           // call *%rax

Apply gets inlined; the inner f(x) stays an indirect call through a register loaded with the address of Doubler one instruction earlier. Missing step: after inlining, spot that the target is now constant, turn the indirect call into a direct one, inline again. FPC inlines once and stops - the decision needs a resolved procdef, and a procvar call never has one.

The narrow case (argument is literally @Proc, or a local assigned @Proc) looks doable. The general one needs whole-function constant propagation FPC doesn't have. Not on my list before your post - it is now.

[Fibonacci]

@Akira1364:

How about that?

Before:

Code: ASM  [Select][+][-]

1. devirt.lpr:43 g := Apply(@Doubler, 21);
2. 0000000100001695 | 488D05C4FFFFFF                                  | LEA RAX,[RIP-0x3C]               | RIP (0x100001660) Doubler devirt.lpr:14
3. 000000010000169C | B915000000                                      | MOV ECX,0x15                     |
4. 00000001000016A1 | FFD0                                            | CALL RAX                         |
5. 00000001000016A3 | 890567490100                                    | MOV [RIP+0x14967],EAX            | RIP (0x100016010) U_$P$DEVIRT_$$_G
6. devirt.lpr:44 end;
7. 00000001000016A9 | 90                                              | NOP                              |
8. 00000001000016AA | 488D642428                                      | LEA RSP,[RSP+0x28]               |
9. 00000001000016AF | C3                                              | RET                              |

After:

Code: ASM  [Select][+][-]

1. devirt.lpr:51 g := Apply(@Doubler, 21);
2. 0000000100001695 | 488D05C4FFFFFF                                  | LEA RAX,[RIP-0x3C]               | RIP (0x100001660) Doubler devirt.lpr:14
3. 000000010000169C | C7056A4901002A000000                            | MOV [RIP+0x1496A],0x2A           | RIP (0x100016010) U_$P$DEVIRT_$$_G
4. devirt.lpr:52 end;
5. 00000001000016A6 | 90                                              | NOP                              |
6. 00000001000016A7 | 488D642428                                      | LEA RSP,[RSP+0x28]               |
7. 00000001000016AC | C3                                              | RET                              |

Code: Pascal  [Select][+][-]

1. var
2.   g: longint;
3.  
4. // ...
5.  
6. function Doubler(x: longint): longint; inline;
7. begin
8.   Result := x * 2;
9. end;
10.  
11. function Apply(f: TFn; x: longint): longint; inline;
12. begin
13.   Result := f(x);
14. end;
15.  
16. // ...
17.  
18. g := Apply(@Doubler, 21);
19. writeln(g);

[creaothceann]

fixed on the devel branch - update to it (the installer lets you pin the branch) and check unleashed/docs/indexed-labels.md there, it was rewritten and now covers what previously only existed in this thread.

Looks fine to me.

Unfortunately I can't test the rest of the changes - the project hits an "Error: Internal error 2014010312", so the relevant assembly file is not created. I've attached a reduced version of the project, if you want to take a look.

(Curiously, Lazarus takes about 2 seconds between showing the error in the messages window and stopping the compilation, so perhaps some internal resources are being taxed.)

Btw. the Delta_Op above in line 13 is marked with the warning "does not seem to be initialized", even though it should be all zeroes at program start - which is what I do want. It seems there are 3 ways to suppress that warning, all rather lacking:

https://github.com/unleashedpascal/compiler/blob/main/unleashed/docs/zeroinit.md

"zeroinit" would clear the static array every time the subroutine is entered, right? That'd be a problem for me because it's one of the most often called subroutines in the project (between 2.7 to 3.6 million times per second).

Not a problem though, I'll just move the array outside before the subroutine instead.

[Fibonacci]

Thanks for the code - I couldn't reproduce it. I had found and fixed a different "Compilation raised exception internally" and assumed that was the one, but you meant a literal internal error.

Fixed (branch devel).

"zeroinit" would clear the static array every time the subroutine is entered, right? That'd be a problem for me because it's one of the most often called subroutines in the project (between 2.7 to 3.6 million times per second).

No, zeroinit only touches ordinary locals and the function result. A static variable keeps its storage across calls by definition, so zeroing it on entry would defeat the point - it is left alone.

Your array lands in .bss as a plain .zero 512, so it is all zeroes at program start exactly as you want, and nothing at all runs on entry no matter how often you call the routine.

The one case that is not literally free is an inline static with a runtime initializer, where a hidden boolean guard is tested before first use. That is a test and a branch, and the initializer itself runs once.

And if the warnings are all you are after, you can always put this near the top of the file, below the {$mode ...}:

Code: Text  [Select][+][-]

1. {$WARN 5036 off : Local variable "$1" does not seem to be initialized}

[Akira1364]

@Akira1364:

How about that?

Before:

Code: ASM  [Select][+][-]

1. devirt.lpr:43 g := Apply(@Doubler, 21);
2. 0000000100001695 | 488D05C4FFFFFF                                  | LEA RAX,[RIP-0x3C]               | RIP (0x100001660) Doubler devirt.lpr:14
3. 000000010000169C | B915000000                                      | MOV ECX,0x15                     |
4. 00000001000016A1 | FFD0                                            | CALL RAX                         |
5. 00000001000016A3 | 890567490100                                    | MOV [RIP+0x14967],EAX            | RIP (0x100016010) U_$P$DEVIRT_$$_G
6. devirt.lpr:44 end;
7. 00000001000016A9 | 90                                              | NOP                              |
8. 00000001000016AA | 488D642428                                      | LEA RSP,[RSP+0x28]               |
9. 00000001000016AF | C3                                              | RET                              |

After:

Code: ASM  [Select][+][-]

1. devirt.lpr:51 g := Apply(@Doubler, 21);
2. 0000000100001695 | 488D05C4FFFFFF                                  | LEA RAX,[RIP-0x3C]               | RIP (0x100001660) Doubler devirt.lpr:14
3. 000000010000169C | C7056A4901002A000000                            | MOV [RIP+0x1496A],0x2A           | RIP (0x100016010) U_$P$DEVIRT_$$_G
4. devirt.lpr:52 end;
5. 00000001000016A6 | 90                                              | NOP                              |
6. 00000001000016A7 | 488D642428                                      | LEA RSP,[RSP+0x28]               |
7. 00000001000016AC | C3                                              | RET                              |

Code: Pascal  [Select][+][-]

1. var
2.   g: longint;
3.  
4. // ...
5.  
6. function Doubler(x: longint): longint; inline;
7. begin
8.   Result := x * 2;
9. end;
10.  
11. function Apply(f: TFn; x: longint): longint; inline;
12. begin
13.   Result := f(x);
14. end;
15.  
16. // ...
17.  
18. g := Apply(@Doubler, 21);
19. writeln(g);

Looks good if it works stablely without any edge-case regressions! You'd want to test it quite robustly before merging it, I think.

One other thing: your latest one-click installer with the default settings fails to build the IDE, on windows at least. Something related to LazarusPackageIIntf I think was the file in question.

[Fibonacci]

One other thing: your latest one-click installer with the default settings fails to build the IDE, on windows at least. Something related to LazarusPackageIIntf I think was the file in question.

Can't reproduce. Both main and devel compiler, and IDE, install fine - I ran it literally just now, two installs side by side, defaults, no problems at all.

Which build did you use, 0.1.4 or the Pixie nightly? And could you post installer.log? Without it there is nothing to go on.

[creaothceann]

"zeroinit" would clear the static array every time the subroutine is entered, right? That'd be a problem for me because it's one of the most often called subroutines in the project (between 2.7 to 3.6 million times per second).

No, zeroinit only touches ordinary locals and the function result. A static variable keeps its storage across calls by definition, so zeroing it on entry would defeat the point - it is left alone.

Ah, I was going from the documentation that said "Records and static arrays are zero-filled recursively, including inline anonymous compound types".

(I do want to disable the warning, but not for the entire subroutine.)


The internal error is now gone, but it seems that in the ASM file the compiler references some labels that aren't defined anywhere, see the attached project... It has 2 versions of the array initialization, a "regular" one and an optimized one that doesn't waste any cycles, but would perhaps induce some rotation in Wirth's grave.
EDIT: this is just an experiment, I can of course also go back to a simple "goto Op[IR];".

I also get a "Warning: unreachable code" for a simple for-loop, not sure what that one is about.