Anonymous Methods

[Mr.Madguy]

Old thread. It was suggested to me to create new thread, as old one has been inactive for too long.

As I see, anonymous methods aren't implemented in FPC yet and there is no plans to add them. I personally need them for my projects to be ported from Delphi to FPC, cuz I want them to be Linux compatible. Why do I need anonymous methods? They're very useful every time, when I need to:

1) Wrap some arbitrary data into interface in order to pass it to another object, when implementation should be as universal, as possible (in case of generics for example). Export data from generic to some other arbitrary object for example.
2) There is data, that need to be passed to many functions, so best way to do it - create object, turn this data into it's properties and this functions - into it's methods. Best example - test cases in unit test.

In both cases declaration of new interface/class is needed every time, I need to do it. With anonymous methods this could be done without wasting time on meaningless declarations, such as interface/class declarations.

So, I would like to contribute. Bad thing - I don't know structure of FPC compilator. Could somebody help me with it?

What is needed to do:

This code:

Code: Pascal  [Select][+][-]

1. type
2.     TRef = reference to procedure(W:Type);
3.  
4.     TClass=class
5.         public
6.             Z:TType1;
7.             function Proc(X:TType2):TRef;
8.     end;
9.  
10. function TClass.Proc(X:TType2):TRef;
11.     var Y:TType3;
12. begin
13.     Y := 10;
14.     Result := procedure(W:TType4)
15.     begin
16.         Z := X + Y + W;
17.     end;
18. end;
19.  

Should be converted into this code:

Code: Pascal  [Select][+][-]

1. type
2.     TAnonymousProc = procedure(W:TType4);
3.  
4.     IFrameInterface=interface
5.         procedure AnonymousProc(W:TType4);
6.     end;
7.  
8.     TAnonymousRef = record
9.         {Same, as for procedure/function of object, but for interface, instead of object}
10.         {Refrence to method itself is required, as one frame object can have several anonymous methods}
11.         Intf:IFrameInterface;
12.         Proc:TAnonymousProc;
13.     end;
14.  
15.     TClass=class
16.         public
17.             Z:TType1;
18.             function Proc(X:TType2):TAnonymousRef;
19.     end;
20.  
21.     TFrameObject = class(TInterfacedObject, IFrameInterface)
22.         public
23.             Object:TClass:
24.             X:TType2;
25.             Y:TType3;
26.             procedure AnonymousProc(W:TType4);
27.     end;
28.  
29. procedure TFrameObject.AnonymousProc(W:TType4);
30. begin
31.     Object.Z := X + Y + W;
32. end;
33.  
34. function TClass.Proc(X:TType2):TAnonymousRef;
35.     var FrameObject:TFrameObject;
36. begin
37.     FrameObject:= TFrameObject.Create;
38.     FrameObject.Object := Self;
39.     FrameObject.X := X;
40.     FrameObject.Y := 10;
41.     Result.Intf := FrameObject;
42.     Result.Proc := @TFrameObject.AnonymousProc;
43. end;
44.  

Also when you perform <Reference to procedure> = <Procedure of object> operation, following code should be generated:

Code: Pascal  [Select][+][-]

1. TProcOfObject = procedure(W:TType) of object;
2. TRefToProc = reference to procedure(W:TType);
3.  
4. X:TProcOfObject;
5. Y:TRefToProc;
6.  
7. Y := X;
8.  

Should be converted into:

Code: Pascal  [Select][+][-]

1. TProcOfObject = procedure(W:TType) of object;
2. TRefToProc = reference to procedure(W:TType);
3.  
4. X:TProcOfObject;
5. Y:TRefToProc;
6.  
7. Y := procedure(W:TType2)
8. begin
9.     X(W);
10. end;
11.  

[Thaddy]

As I see, anonymous methods aren't implemented in FPC yet and there is no plans to add them. I personally need them for my projects to be ported from Delphi to FPC, cuz I want them to be Linux compatible.

Well, that's not entirely true. Work is being done. For OSX there is already the blocks feature, which is close to anonymous methods.
In the mean time I solve these kind of issues by simply using procedural variables. That is also close to an  "official" work-around.
That is not the eye-candy that anonymous methods really are, but works just about the same.

[Mr.Madguy]

Well, that's not entirely true. Work is being done. For OSX there is already the blocks feature, which is close to anonymous methods.
In the mean time I solve these kind of issues by simply using procedural variables. That is also close to an  "official" work-around.
That is not the eye-candy that anonymous methods really are, but works just about the same.

Can this code be achieved via procedural variables when Container is generic and for arbitrary AnotherContainer?

Code: Pascal  [Select][+][-]

1. Container.Query(Predicate, procedure(AItem:TItem)
2.     begin
3.         AnotherContainer.Add(AItem);
4.     end;
5. );
6.  

Yeah, Container can have enumerator and this can be changed into:

Code: Pascal  [Select][+][-]

1. for Item in Container do begin
2.     if Predicate(Item) then begin
3.          AnotherContainer.Add(Item);
4.     end;
5. end;
6.  

But the way, data is being extracted from Container via Query method, is determined by Container itself - is can be linked, indexed, sorted, hashed, have tree structure for example. So, "Add" procedure should be callback. But this callback should have reference to AnotherContainer. And procedural variables can't do it - only objects. And object requires bunch of completely unnecessary declarations.

[Thaddy]

You mean something like:

Code: Pascal  [Select][+][-]

1. program untitled;
2. {$ifdef fpc}{$mode delphi}{$H+}{$endif}
3. uses generics.collections;
4.  
5. type
6.   TMyProc = procedure;
7.  
8. procedure MyProc;
9. begin
10.   writeln('Test me');
11. end;
12.  
13. var
14.   List:TList<TMyProc>;
15.   Proc:TMyProc;  
16. begin
17.   List := TList<TMyProc>.Create;
18.   try
19.     List.Add(MyProc);
20.     Proc := List[0];
21.     Proc;
22.   finally
23.     List.free;
24.   end;
25. end.

[Mr.Madguy]

You mean something like:

Can this code be achieved via procedural variables?

Code: Pascal  [Select][+][-]

1. function TList<T>.Query(APredicate:TPredicate<T>):TList<T>;
2.     var List:TList<T>;
3. begin
4.     List := TList<T>.Create;
5.     DoQuery(APredicate,
6.         procedure(AItem:T)
7.         begin
8.             List.Add(AItem);
9.         end;
10.     )
11.     {Needed, cuz Result can't be captured}
12.     Result := List;
13. end;
14.  

[guest58172]

They would be useful also for functional programming. For high order functions such as map, fold, filter, etc you usually just need predicates declared in the current scope and that are not useful elsewhere.

[Mr.Madguy]

They would be useful also for functional programming. For high order functions such as map, fold, filter, etc you usually just need predicates declared in the current scope and that are not useful elsewhere.

Yeah. Closures are very powerful in case of script languages, but it case of compilers they shouldn't be overused, i.e. used in cases, when other solutions are possible, cuz they're expensive.

Python example:

Code: Python  [Select][+][-]

1. def f(x):
2.     def f(y):
3.         if y<=x:
4.             print y
5.             f(y*2)
6.             f(y*2+1)
7.     f(1)
8.  

Can be changed into:

Code: Python  [Select][+][-]

1. def g(x,y):
2.     if y<=x:
3.         print y
4.         g(x,y*2)
5.         g(x,y*2+1)
6.  
7. def f(x):
8.     g(x,1)
9.  

But in some cases they're really needed - as in case of callback, that should carry some references to other data. I.e. in case, where declaration of object is longer, than payload code, it contains, and declaring 10, 20 or 30 of such objects - is just a waste of time.

[Thaddy]

You mean something like:

Can this code be achieved via procedural variables?

Yes. Another example:

Code: Pascal  [Select][+][-]

1. program untitled;
2. {$ifdef fpc}{$mode delphi}{$H+}{$endif}
3. uses
4.   generics.collections;
5.  
6. Type
7.    Tmyfunc = function(const value:boolean):boolean;
8.    
9.    function function1(const value:boolean):boolean;
10.    begin
11.      result := value;
12.    end;
13.  
14. var
15.   Conditions:TList<TMyfunc>;
16. begin
17.     Conditions := TList<TMyFunc>.Create;
18.     try
19.       Conditions.Add(function1);
20.       writeln(Conditions[0](true));
21.     finally
22.       Conditions.Free;
23.     end;
24. end.

[guest58172]

They would be useful also for functional programming. For high order functions such as map, fold, filter, etc you usually just need predicates declared in the current scope and that are not useful elsewhere.

Yeah. Closures are very powerful in case of script languages, but it case of compilers they shouldn't be overused, i.e. used in cases, when other solutions are possible, cuz they're expensive.

No they're also available in strongly typed and compiled languages (e.g F#, haskell, D, probably others too). In D they're semantically equivalent to a templatized function with parameter type inference...So not more expansive that what's everybody already uses all the time. For the dynamic and interpreted languages I don't say...

[Mr.Madguy]

Yes. Another example:

Yes? Then, show me, how to fix my code to work via procedural variables. But please note, that List - is local variable, not global. It has different value every time, Query is called. And that DoQuery shouldn't know anything about List. Cuz it could be TSet<T>, TStack<T> or may be TQueue<T>. All, DoQuery should do - call some DoAdd(AItem) callback, when APredicate.Query(AItem) = True.

[Mr.Madguy]

No they're also available in strongly typed and compiled languages (e.g F#, haskell, D, probably others too). In D they're semantically equivalent to a templatized function with parameter type inference...So not more expansive that what's everybody already uses all the time. For the dynamic and interpreted languages I don't say...

Please note, that closures require object creation and object creation requires memory allocation. Yeah, modern generation of programmers think, that GetMem and FreeMem - are instant operations. But they aren't. The reason, why strings and objects were static in Turbo Pascal - memory allocation was way too slow on old computers. In script languages it doesn't matter, cuz everything there - is objects anyway. But scripts are optimized for this: memory manager works like stack there - when it allocates memory, it always allocates new blocks to avoid searching for free ones and it doesn't actually free blocks at all. Garbage collector performs both memory free and heap defragmentation tasks. But it isn't the case for Delphi/FPC. So, if you can avoid memory alloc/free operations in your algorithm - you should do it. In my example second algorithm uses stack only. In Python second function object is being created anyway, so there is no difference. But in Delphi/FPC functions are static - no alloc/free operations will be performed.

I.e. if you use closure - you should do it, only if algorithm would require object creation anyway.

[Leledumbo]

So, I would like to contribute.

The issue is still open: http://bugs.freepascal.org/view.php?id=24481

Bad thing - I don't know structure of FPC compilator. Could somebody help me with it?

Existing contributors just dig in the source until they get the structure in their head and are able to contribute. Though I haven't contributed any such low level code, I've dug the code a bit while debugging for code generator issues. I say it's not that hard to follow, it just employs classic recursive descent parser that generates syntax tree, which is further optimized as requested before given to platform specific code generator.

[guest60499]

No they're also available in strongly typed and compiled languages (e.g F#, haskell, D, probably others too). In D they're semantically equivalent to a templatized function with parameter type inference...So not more expansive that what's everybody already uses all the time. For the dynamic and interpreted languages I don't say...

Please note, that closures require object creation and object creation requires memory allocation. Yeah, modern generation of programmers think, that GetMem and FreeMem - are instant operations. But they aren't. The reason, why strings and objects were static in Turbo Pascal - memory allocation was way too slow on old computers. In script languages it doesn't matter, cuz everything there - is objects anyway. But scripts are optimized for this: memory manager works like stack there - when it allocates memory, it always allocates new blocks to avoid searching for free ones and it doesn't actually free blocks at all. Garbage collector performs both memory free and heap defragmentation tasks. But it isn't the case for Delphi/FPC. So, if you can avoid memory alloc/free operations in your algorithm - you should do it. In my example second algorithm uses stack only. In Python second function object is being created anyway, so there is no difference. But in Delphi/FPC functions are static - no alloc/free operations will be performed.

I.e. if you use closure - you should do it, only if algorithm would require object creation anyway.

Unfortunately I will have a hard time providing a source you might find adequate, but anonymous functions and classes must create a closure. The memory usage is still there though the code reference is in the program's .text section. I personally support the addition of anonymous procedures and functions, but, at the same time, I don't have the time to contribute the code to do so.

As an aside I'm slightly aghast that FPC now has generics that use the old angle bracket notation. I am saddened no one could find a more Pascal-like idiom for them.

[guest58172]

No they're also available in strongly typed and compiled languages (e.g F#, haskell, D, probably others too). In D they're semantically equivalent to a templatized function with parameter type inference...So not more expansive that what's everybody already uses all the time. For the dynamic and interpreted languages I don't say...

Please note, that closures require object creation and object creation requires memory allocation. Yeah, modern generation of programmers think, that GetMem and FreeMem - are instant operations. But they aren't. The reason, why strings and objects were static in Turbo Pascal - memory allocation was way too slow on old computers. In script languages it doesn't matter, cuz everything there - is objects anyway. But scripts are optimized for this: memory manager works like stack there - when it allocates memory, it always allocates new blocks to avoid searching for free ones and it doesn't actually free blocks at all. Garbage collector performs both memory free and heap defragmentation tasks. But it isn't the case for Delphi/FPC. So, if you can avoid memory alloc/free operations in your algorithm - you should do it. In my example second algorithm uses stack only. In Python second function object is being created anyway, so there is no difference. But in Delphi/FPC functions are static - no alloc/free operations will be performed.

I.e. if you use closure - you should do it, only if algorithm would require object creation anyway.

I don't see how a closure would involve a heap allocation ! I think that you don't consider them with the right point of view. In the languages mentioned previously they're template "value" parameters ! This means known at compile time, optimized and inlined by the compiler...e.g you won't see any CALL in the disassembly.

[marcov]

I don't see how a closure would involve a heap allocation ! I think that you don't consider them with the right point of view. In the languages mentioned previously they're template "value" parameters ! This means known at compile time, optimized and inlined by the compiler...e.g you won't see any CALL in the disassembly.

An anonymous method like TThread.Queue finishes before the closure is called (since it is scheduled in a different thread), so the stack would already have been cleaned up.

FreePascal's nested procvars are something like that. (the stack frame of the nested procedure and its code when passed is a kind of closure) But anonymous methods are not.

P.s. I use anonymous methods in my delphi code but quite sparingly actually. They are useful for quick and easy stuff, but IMHO a bit overrated. For the more important constructs I use a pool of messenger objects that are not adhoc objects/interfaces and can be easier rerouted, logged etc.