Feature announcement: Function References and Anonymous Functions

[Peter H]

This works in Free Pascal:

The point is, (unlike Delphi) it doesnt work, if the anonymous procedures are coded inside the initial

Code: [Select]

begin
end.
It gives access violation or random results, depending on the code.

It must be coded inside a procedure.
Then it works as intended.

Code: Pascal  [Select][+][-]

1. program project1;
2.  
3. {$mode objfpc}{$H+}
4. {$modeswitch functionreferences}
5. {$modeswitch anonymousfunctions}
6. // {$warn 5036 off}// "Warning: (5036) Local variable "$Capturer" does not seem to be initialized"
7.  
8. uses
9.   {$IFDEF UNIX}
10.   cthreads,
11.   {$ENDIF}
12.   Classes,Sysutils { you can add units after this };
13.  
14. type  Tpr = reference to procedure;
15. var   p_inner: Tpr;
16.  
17. procedure caller;
18. begin
19.   p_inner();
20. end;
21.  
22. procedure main;
23.  
24. var str: String;
25. begin
26.  
27.     str := 'Hello World!';
28.  
29.     p_inner := function(s:String):Tpr //This captures local and persistent copy of "str"
30.     begin
31.       Result := procedure
32.       begin
33.          Writeln(s);
34.       end;
35.       Writeln('Outer function was called');
36.     end(str);  //This does NOT print the string "s"!
37.                //it just instantiates the outer function and p_inner and captures their local context.
38.     SetLength(str,0); //Erase the string content
39.  
40.     Writeln('now calling p_inner');
41.     caller();  //This line prints the string s="Hello World!", which was captured by the outer function.
42.                //p_inner will be called from an external context, just for test and demonstration
43. end;
44.  
45. begin
46.   main;
47.   p_inner(); //calling p_inner again, when the context of main() is destroyed
48.   readln;
49. end.
50.  

This is just a nontrivial  example for learning and testing purposes.
The power of lambdas and closures becomes obvious, it you consider, "p_inner" could be called from
other units and even from other threads via Synchronize() or via Queue().

[Thaddy]

the type can be outside the procedure..

[Peter H]

the type can be outside the procedure..

Yes, compatible types are globally predefined in LCL.
But for this test I preferred to define everything myself, just for clarity, so it is clear to see what I did,
without looking up the LCL definitions.

Edit:
I accept your proposal. However this is just test code, therefore I did not consider this.
I changed the type definition, so "p_inner" can be called from other procedures.

[PascalDragon]

So the outer anonymous function is essentially useless.

I do not think so. This would be useless:

Code: Pascal  [Select][+][-]

1. procedure(var i,j,k:integer)
2. begin
3.     Queue(procedure begin Sleep(500); Writeln(i,' ',hexstr(@i),' ',j,' ',k);end);
4. end(i,j,k);
5.  
6. inc(i);
7. .......
8.  

This captures the variables by reference.

Right, I hadn't noticed that this essentially captured by value...

Edit: I tested in FPC:
Lazarus 2.3.0 (rev 0c96c0e3a5) FPC 3.3.1 x86_64-win64-win32/win64
Unfortunately it does not run, it raises an access violation.
The problem is line 12:  Writeln(s);, if I comment it out, it runs.
When I use integer instead of String, it doesnt crash but prints an invalid number.
So there are still some fundamental problems with anonymous procedures.

Please report a bug with a reproducible example.

Just a remark: The word anonymous is somewhat misleading. Anonymous functions do not need a name, because there is - and always must be - a named or indexed reference pointing to them.

In both Delphi and FPC the anonymous refers to the fact that the function body has no name, no matter if you assign it to a variable or pass it as a parameter to another function.

This is my Freepascal example:
It MUST print "1", but it prints a random number.

Please report a bug for this as well.

[Peter H]

This is my Freepascal example:
It MUST print "1", but it prints a random number.

Please report a bug for this as well.

The two bugs are the same.
When I use integers it gives random results, when I use strings it gives access violations, because strings under the hood use pointers.

I assume the developers already know this, because this feature, anonymous functions, and the trunk compiler are work in progress and heavily tested.

[Peter H]

I have tried to write a bug report, but do not understand this markup code I must use.
(I could write it in TeX, but do not understand this markup code ;-)

If I type a line feed in the editor, then when I activate the preview, all linefeeds are gone.
When I insert source code I get errormessages.
It is much easier to write here, so I ask someone else to report it.

This code compiles and works ok:
It outputs 4 lines of text.

Code: Pascal  [Select][+][-]

1. program project1;
2.  
3. {$mode objfpc}{$H+}
4. {$modeswitch functionreferences}
5. {$modeswitch anonymousfunctions}
6. // {$warn 5036 off}// "Warning: (5036) Local variable "$Capturer" does not seem to be initialized"
7.  
8. uses
9.   {$IFDEF UNIX}
10.   cthreads,
11.   {$ENDIF}
12.   Classes,Sysutils { you can add units after this };
13.  
14. type  Tpr = reference to procedure;
15. var   p_inner: Tpr;
16.  
17. procedure caller;
18. begin
19.   p_inner();
20. end;
21.  
22. procedure main;
23.  
24. var str: String;
25. begin
26.  
27.     str := 'Hello World!';
28.  
29.     p_inner := function(s:String):Tpr //This captures local and persistent copy of "str"
30.     begin
31.       Result := procedure
32.       begin
33.          Writeln(s);
34.       end;
35.       Writeln('Outer function was called');
36.     end(str);  //This does NOT print the string "s"!
37.                //it just instantiates the outer function and p_inner and captures their local context.
38.     SetLength(str,0); //Erase the string content
39.  
40.     Writeln('now calling p_inner');
41.     caller();  //This line prints the string s="Hello World!", which was captured by the outer function.
42.                //p_inner will be called from an external context, just for test and demonstration
43. end;
44.  
45. begin
46.   main;
47.   p_inner(); //calling p_inner again, when the context of main() is destroyed
48.   readln;
49. end.
50.  

If I modify the code of main() so that it is between begin and end. (no main procedure) then it compiles, but gives access violation when it runs:

Code: Pascal  [Select][+][-]

1. program project1;
2.  
3. {$mode objfpc}{$H+}
4. {$modeswitch functionreferences}
5. {$modeswitch anonymousfunctions}
6. // {$warn 5036 off}// "Warning: (5036) Local variable "$Capturer" does not seem to be initialized"
7.  
8. uses
9.   {$IFDEF UNIX}
10.   cthreads,
11.   {$ENDIF}
12.   Classes,Sysutils { you can add units after this };
13.  
14. type  Tpr = reference to procedure;
15. var   p_inner: Tpr;
16.  
17. procedure caller;
18. begin
19.   p_inner();
20. end;
21.  
22. //procedure main;
23.  
24. var str: String;
25. begin
26.  
27.     str := 'Hello World!';
28.  
29.     p_inner := function(s:String):Tpr //This captures local and persistent copy of "str"
30.     begin
31.       Result := procedure
32.       begin
33.          Writeln(s);
34.       end;
35.       Writeln('Outer function was called');
36.     end(str);  //This does NOT print the string "s"!
37.                //it just instantiates the outer function and p_inner and captures their local context.
38.     SetLength(str,0); //Erase the string content
39.  
40.     Writeln('now calling p_inner');
41.     caller();  //This line prints the string s="Hello World!", which was captured by the outer function.
42.                //p_inner will be called from an external context, just for test and demonstration
43. end.
44.  
45.  

[Marc]

To write a block of code in gitlab (the bugtracker), start your block of code with three ``` and end your block with three ``` like:

Code: [Select]

```
procedure MyProc;
begin
end;
```




[Thaddy]

Yese, three back ticks, note you get pascal formatted code if you do it like this:
```pascal
/// your code
```

[Peter H]

@Marc, Thaddy, thank you!
How can I know this without reading documentation for hours?
To insert source code, I clicked the symbol </> in the editor and this creates only two ticks.
And when I then view it in the preview I see a bunch of error messages.

Anyway, I believe it is too early to report errors about this feature because I would not find an end.
The problems are obvious if you compile or run nontrivial examples, which run without problems in Delphi.

I modified the source, so that it compiles unchanged without problems in Delphi.
Compiler version is: Lazarus 2.3.0 (rev 0c96c0e3a5) FPC 3.3.1 x86_64-win64-win32/win64
I do however believe, this is a general problem not specific to windows. So more people should test this on other OS before reporting bugs. (I have no Linux currently) And I am almost 70 years old and retired so I will stay with windows, because I need it for my other stuff.

The following source code gives access violation at runtime in FPC.
If I modify the source and declare "f_outer" local, then it gives internal compiler error in FPC.
Both versions compile and run ok in Delphi.

Code: Pascal  [Select][+][-]

1. program project1;
2.  
3. {$mode objfpc}{$H+}
4. {$modeswitch functionreferences}
5. {$modeswitch anonymousfunctions}
6. // {$warn 5036 off}// "Warning: (5036) Local variable "$Capturer" does not seem to be initialized"
7.  
8. uses
9.   {$IFDEF UNIX}
10.   cthreads,
11.   {$ENDIF}
12.   Classes,Sysutils { you can add units after this };
13.  
14. type  Tproc = reference to procedure;
15.       Tfunc = reference to function(s:String):Tproc;
16. var   p_inner: Tproc;
17.       f_outer: Tfunc;
18.  
19. procedure caller;
20. begin
21.   p_inner();
22. end;
23.  
24. procedure main;
25.  
26. var str: String;
27. //    f_outer: Tfunc;    // <-----  doesnt compile when this is uncommented
28.  
29. begin
30.  
31.     str := 'Hello World!';
32.  
33.     f_outer := function(s:String):Tproc //This captures local and persistent copy of "str"
34.     begin
35.       Result := procedure  //  <----Access violation at runtime here in FPC, but not in Delphi. Program runs ok in Delphi.
36.       begin
37.          Writeln(s);
38.       end;
39.       Writeln('Outer function was called');
40.     end;
41.     p_inner := f_outer(str);
42.  
43.  
44.                //This does NOT print the string "s"!
45.                //it just instantiates the outer function and p_inner and captures their local context.
46.     SetLength(str,0); //Erase the string content
47.  
48.     Writeln('now calling p_inner');
49.     caller();  //This line prints the string s="Hello World!", which was captured by the outer function.
50.                //p_inner will be called from an external context, just for test and demonstration
51. end;
52.  
53. begin
54.   main;
55.   p_inner(); //calling p_inner again, when the context of main() is destroyed
56.   readln;
57. end.
58.  

[Thaddy]

It can be just a name clash since in freepascal Tproc and Tfunc are already defined. Try it with different names.

[PascalDragon]

The two bugs are the same.
When I use integers it gives random results, when I use strings it gives access violations, because strings under the hood use pointers.

Ah, right, somehow I had read the first one as triggering an internal error instead of an access violation

I assume the developers already know this, because this feature, anonymous functions, and the trunk compiler are work in progress and heavily tested.

I am one of those developers.

Anyway, I believe it is too early to report errors about this feature because I would not find an end.
The problems are obvious if you compile or run nontrivial examples, which run without problems in Delphi.

No, it's not too early, because I explicitly requested users to test this in the initial post, because the tests that I do have do work correctly and we need to find those cases that don't:

As these two features are rather complicated there might still be a huge bundle of bugs lurking around so I ask you to test them to year heart's content and report found bugs to the issues on GitLab so that we can fix as many of them as possible before the next major version (which is not yet planned, so don't worry ).

The following source code gives access violation at runtime in FPC.
If I modify the source and declare "f_outer" local, then it gives internal compiler error in FPC.
Both versions compile and run ok in Delphi.

Please report once for the access violation / random data and once for the internal error.

[Peter H]

Ok thank you.

I do however think, as long as there are compiler problems it doesnt make sense to report all runtime problems.
I changed my project a little bit, I converted the procedure into a function, which returns a string.
Now I get compiler errors only.
The program compiles and runs in Delphi.

Also this cumbersome bunch of confusing type declarations is only necessary due to the lack of type inference in FPC.
And I do not see how type inference could be possible without inline variables.

Look for line 37.

Then, if I uncomment line 27, I get different errors.
So I could report problems without end, but I do not want to do this, it makes no sense.

So I believe, runtime errors are irrelevant as long as the compiler has obvious problems.
It would be too confusing to report every and each runtime error, because these are secondary to compiler errors. They will hopefully automagically vanish if the compiler problems are fixed.

(I do not want to warm up annoying discussions about inline variables. If there are other solutions, I am open for it)

Code: Pascal  [Select][+][-]

1. program project1;
2. {$ifdef FPC}
3. {$mode objfpc}{$H+}
4. {$modeswitch functionreferences}
5. {$modeswitch anonymousfunctions}
6. // {$warn 5036 off}// "Warning: (5036) Local variable "$Capturer" does not seem to be initialized"
7. {$endif}
8. uses
9.   {$IFDEF UNIX}
10.   cthreads,
11.   {$ENDIF}
12.   Classes,Sysutils { you can add units after this };
13.  
14. type  TfuncS = reference to function:String;
15.       TfuncF = reference to function(s:String):TfuncS;
16. var   f_inner: TfuncS;
17.       f_outer: TfuncF;
18.  
19. procedure caller;
20. begin
21.   f_inner();
22. end;
23.  
24. procedure main;
25.  
26. var str: String;
27.    // f_outer: TfuncF;    // <----doesnt compile in FPC when this is uncommented, but compiles and runs ok in Delphi
28.  
29. begin
30.  
31.     str := 'Hello World!';
32.  
33.     f_outer := function(s:String):TfuncS //This captures local and persistent copy of "str"
34.     begin
35.       Result := function:String
36.       begin
37.         Result:=s;  // <---- project1.lpr(37,9) Error: Internal error 2011010304
38.         Writeln(s);
39.       end;
40.       Writeln('Outer function was called');
41.     end;
42.     f_inner := f_outer(str);   //This instantiates the outer function and f_inner and captures their local context.
43.  
44.     SetLength(str,0); //Erase the string content
45.  
46.     Writeln('now calling f_inner');
47.     caller();  //This line prints the string s="Hello World!", which was captured by the outer function.
48.                //f_inner will be called from an external context, this is just for test and demonstration
49. end;
50.  
51. begin
52.   main;
53.   Writeln('Now the context of "main()" is lost. Can we still print the string "str"?');
54.   f_inner();
55.   readln;
56. end.

[Peter H]

I have now created a bug report.
https://gitlab.com/freepascal.org/fpc/source/-/issues/40315

I must apologize, I cannot master this markup language.

In the first (code) section linebreaks are ok.

In the two following sections not, I do not know why.
If I write a linebreak, I mean a linebreak and it should not remove it. How to achieve this?

The "</>" symbol in the editor does not work here, maybe it is my browser? (Edge)
It outputs only two backticks, but the documentation of the markup language says I must use three before the code section and three after. :-(

The content of this bug report /should/ be like this:

Code: Pascal  [Select][+][-]

1. program project1;
2. //This program compiles and runs in Delphi and in FPC. (at least should run in FPC)
3. //It is intentionally designed this way.
4. {$ifdef FPC}
5.   {$mode objfpc}{$H+}
6.   {$modeswitch functionreferences}
7.   {$modeswitch anonymousfunctions}
8.   // {$warn 5036 off}// "Warning: (5036) Local variable "$Capturer" does not seem to be initialized"
9. {$endif}
10. uses
11.   {$IFDEF UNIX}
12.   cthreads,
13.   {$ENDIF}
14.   Classes,Sysutils { you can add units after this };
15.  
16. type  TfuncS = reference to function:String;
17.       TfuncF = reference to function(s:String):TfuncS;
18. var   f_inner: TfuncS;
19.       f_outer: TfuncF;
20.  
21. procedure caller;
22. begin
23.   f_inner();
24. end;
25.  
26. procedure main;
27.  
28. var str: String;
29.    // f_outer: TfuncF;  // <---- doesnt compile in FPC when this is uncommented, but compiles and runs ok in Delphi
30.  
31. begin
32.  
33.     str := 'Hello World!';
34.  
35.     f_outer := function(s:String):TfuncS //This captures local and persistent copy of "str"
36.     begin
37.       Result := function:String
38.       begin
39.         Result := s;  // <---- project1.lpr(37,9) Error: Internal error 2011010304
40.                       // if the line is commented out it compiles, but gives access violation at runtime
41.  
42.         Writeln(s);
43.       end;
44.       Writeln('Outer function was called');
45.     end;
46.     f_inner := f_outer(str);   //This instantiates the outer function and f_inner and captures their local context.
47.  
48.     SetLength(str,0); //Erase the string content
49.  
50.     Writeln('now calling f_inner');
51.     caller();  //This line prints the string s="Hello World!", which was captured by the outer function.
52.                //f_inner will be called from an external context, this is just for test and demonstration
53. end;
54.  
55. begin
56.   main;
57.   Writeln('Now the context of "main()" is lost. Can we still print the string "str"?');
58.   if f_inner()='Hello World!' then writeln('Yes! :-)') else writeln ('No! :-(');
59.  
60.   readln;
61. end.
62.  

However, beginning from here, all intentional linebreaks are lost in GitLab, and I do not know, why.

Code: [Select]


Compile Project, Target: C:\pasPRJ\AnonymousProcs\project1.exe: Exit code 1, Errors: 1, Hints: 2
Hint: Start of reading config file C:\fpc\fpc\bin\x86_64-win64\fpc.cfg
Hint: End of reading config file C:\fpc\fpc\bin\x86_64-win64\fpc.cfg
Verbose: Free Pascal Compiler version 3.3.1-12746-g9bfb45dc05 [2023/06/10] for x86_64
Verbose: Copyright (c) 1993-2023 by Florian Klaempfl and others
Verbose: Target OS: Win64 for x64
Verbose: Compiling project1.lpr
Progress: 2 233/768 Kb Used
project1.lpr(39,9) Error: Internal error 2011010304
Verbose: C:\fpc\fpc\bin\x86_64-win64\ppcx64.exe returned an error exitcode


Code: [Select]


The output of the program, when run in Delphi is:

Outer function was called
now calling f_inner
Hello World!
Now the context of "main()" is lost. Can we still print the string "str"?
Hello World!
Yes! :-)


[Bad Sector]

Please report a bug with a small, self contained, reproducible example.

What is internal error 200204175 about? I tried to put the code in a single self-contained program but that compiles without the error. Perhaps it can give me some idea what exactly causes it so i can make a reproducible example.

[Peter H]

So, if you did not get the error, I am happy and want to know which OS and compiler you use.

And which exact source.

I am then happy to compile again and compare.
Please try my later examples, which are on GitLab.
These are intentionally made to compile out of the box.
Or, if they do not compile, they show errors out of the box.
The sourcecode is selfcontaining and has no other  external depencies than "WriteLn".
All compile on Delphi and deliver the expected result.

Please ignore the older postings, I improved the informations and error reports inbetween.
When I initially posted this, it was not totally clear to myself, but now at least the problem is clear to me.

Just paste the source code into an  empty project, compile, and if possible, run.
It should be a matter of seconds.

https://gitlab.com/freepascal.org/fpc/source/-/issues/40315
https://gitlab.com/freepascal.org/fpc/source/-/issues/40316