[CLOSED (almost)] Mixed "procedure" and "procedure of object" callback

[440bx]

@440bx

I wouldn't want to go off topic, so I'll try to be concise. I have an open attitude towards programming paradigms, in the sense that I try to use the paradigm that in the language I'm using at that time solves, in my opinion, a certain problem at its best.

I understand - I won't contribute to the off topic subject but, I do want to state that, I believe, maybe incorrectly, that I have an open attitude too. I'm probably more demanding.

[Otto]

 A few days ago I opened a thread (New wiki topics) to talk about the topics you have appointed. Comparing the opinions of those who have used different programming languages is, in my opinion, useful to improve themselves as software developers.

[marcov]

To return to the thread's original question, I believe that it can be solved through anonymous functions (also called lambda functions), in Delphi or in C#, they are used to solve similar problems.

The problem is still that you then you can only pass anonymous functions (reference to procedure) to them, or at least have to do some boxing from normal methods to anonymous methods.

But anonymous methods are a slower concept, so that would be quite a big step, and while it might be fine for some libraries, it would hurt others, which makes it less universal.

It has been suggested in the past to automatically generate multiple instantiations of the calling code (so either accept one or the other callback), but that complicates making them virtual methods.

[Martin_fr]

And (really sadly) almost always when people talk about "anonymous functions" they assume that they are equal to "closures". And vice versa.

Most languages may combine the 2 ideas into one. I really dislike that.

- An anonymous function, can be a normal (probably not nested) function, just declared in code.
- A closure does not have to be anonymous.

And you could even have anonymous methods....

[marcov]

Moreover, Delphi anonymous methods are a bit funky as to what they encapsulate. Just call  tthread,queue() in a loop with the loopvariable as capture  value, and e.g. add the result to a memo.

Because the address of the loopvalue is captured, (by reference) when it is shown in the memo it is not the same value as it was when you queued() that proc

[Martin_fr]

Moreover, Delphi anonymous methods are a bit funky as to what they encapsulate. Just call  tthread,queue() in a loop with the loopvariable as capture  value, and e.g. add the result to a memo.

Because the address of the loopvalue is captured, (by reference) when it is shown in the memo it is not the same value as it was when you queued() that proc

<sarcasm>
That is why there is a need for inline declared variables
</sarcasm>
In JS IIRC you need to declare a variable (that you want to capture) inside the loop. Variables declared outside the loop, have the same issue.

IMHO a closure should (whether defined inline or up front) have some syntax that allows it to specify what to capture. (If needs must, also current value vs value at the end of the outer proc)

--
And to be the devils advocate: Why does a closure have to be refcounted? It could be freed by the user.
In either case the user needs a way to manually release it, as refcounts can be circular (the closure could capture the variable holding itself)

[440bx]

Comparing the opinions of those who have used different programming languages is, in my opinion, useful to improve themselves as software developers.

I concur.  When it comes to OOP, I try hard not to say anything since I am well aware that whatever I say is utterly futile.   Obviously, there are times when I can't help myself and say something about it. 

This thread https://forum.lazarus.freepascal.org/index.php/topic,46097.15.html has a comparison of a function implemented normally and the same function implemented using OOP.  IMO, the non-OOP implementation is obviously _vastly_ superior to the OOP one but, it seems I am the only one who thinks so. 

I can say one good thing for OOP.  It is more likely to provide instant gratification to a programmer than structured programming.  Instant gratification seems very important to a lot of people in general (instant coffee, 5 minute microwaved dinners and, drag'n'drop programs.)

It takes a lot longer to see some results with structured programming than with OOP but, the former is like a dinner at a La Tour D'argent while the later is a microwaved TV dinner.  The former is a pleasure, the latter as a stop-gap measure.

[Otto]

 Hello Martin.
 

And (really sadly) almost always when people talk about "anonymous functions" they assume that they are equal to "closures". And vice versa.

Most languages may combine the 2 ideas into one. I really dislike that.

- An anonymous function, can be a normal (probably not nested) function, just declared in code.
- A closure does not have to be anonymous.

And you could even have anonymous methods....

  To reinforce your speech, I report:  “The Evolution of Delegates in C#”. The terminology of OOP is often used differently depending on the programming language used, which is why it would always be good to specify the language to which it refers.

Code: C  [Select][+][-]

1. class Test
2. {
3.     delegate void TestDelegate(string s);
4.     static void M(string s)
5.     {
6.         Console.WriteLine(s);
7.     }
8.  
9.     static void Main(string[] args)
10.     {
11.         // Original delegate syntax required
12.         // initialization with a named method.
13.         TestDelegate testDelA = new TestDelegate(M);
14.  
15.         // C# 2.0: A delegate can be initialized with
16.         // inline code, called an "anonymous method." This
17.         // method takes a string as an input parameter.
18.         TestDelegate testDelB = delegate(string s) { Console.WriteLine(s); };
19.  
20.         // C# 3.0. A delegate can be initialized with
21.         // a lambda expression. The lambda also takes a string
22.         // as an input parameter (x). The type of x is inferred by the compiler.
23.         TestDelegate testDelC = (x) => { Console.WriteLine(x); };
24.  
25.         // Invoke the delegates.
26.         testDelA("Hello. My name is M and I write lines.");
27.         testDelB("That's nothing. I'm anonymous and ");
28.         testDelC("I'm a famous author.");
29.  
30.         // Keep console window open in debug mode.
31.         Console.WriteLine("Press any key to exit.");
32.         Console.ReadKey();
33.     }
34. }
35. /* Output:
36.     Hello. My name is M and I write lines.
37.     That's nothing. I'm anonymous and
38.     I'm a famous author.
39.     Press any key to exit.
40.  */

[PascalDragon]

And to be the devils advocate: Why does a closure have to be refcounted? It could be freed by the user.

Because it can be copied around. It can be passed to function A and function B, maybe function A stores it in variable X and function B stores it in class instance Y. Some time later class instance Y is freed, but X is still active. So there must be some form of reference counting. In Delphi they simply used what was already there and supports methods: reference counted interfaces.

In either case the user needs a way to manually release it, as refcounts can be circular (the closure could capture the variable holding itself)

That is indeed an interesting one...

[marcov]

Moreover, Delphi anonymous methods are a bit funky as to what they encapsulate. Just call  tthread,queue() in a loop with the loopvariable as capture  value, and e.g. add the result to a memo.

Because the address of the loopvalue is captured, (by reference) when it is shown in the memo it is not the same value as it was when you queued() that proc

<sarcasm>
That is why there is a need for inline declared variables
</sarcasm>

<blueeyes>
Why would you need that if you have a nested complete anonymous procedure declaration with parameters and local variables?
</blueeyes>

In JS IIRC you need to declare a variable (that you want to capture) inside the loop. Variables declared outside the loop, have the same issue.

Delphi has parameters to the nested anonymous procedures for that effect, and even assigning the loopvar to a local var in such function could be used as a sign to capture by value. But the latter afaik doesn't work, and parameters don't match the signature for the closure in the anonymous version of tthread.queue()

IMHO a closure should (whether defined inline or up front) have some syntax that allows it to specify what to capture. (If needs must, also current value vs value at the end of the outer proc)

Agree.

And to be the devils advocate: Why does a closure have to be refcounted? It could be freed by the user.
In either case the user needs a way to manually release it, as refcounts can be circular (the closure could capture the variable holding itself)

That aspect annoys me less. But probably to mimic C# behaviour which is what most language extensions of the same era come from, and at the same time have the same "easy" syntax. Extra lines for allocation/deallocation will be villified in the fora.

[Martin_fr]

And to be the devils advocate: Why does a closure have to be refcounted? It could be freed by the user.

Because it can be copied around. It can be passed to function A and function B, maybe function A stores it in variable X and function B stores it in class instance Y. Some time later class instance Y is freed, but X is still active. So there must be some form of reference counting. In Delphi they simply used what was already there and supports methods: reference counted interfaces.

I am not against the ref counted part. [1]
However, it does not implicitly follow from the concept of closure.

Method pointers can be passed around too. They include a pointer to an instance, which may be dangling if the instance was freed.
Of course method pointers do not need to be freed, because they are a record type, and do not by default allocate heap.

But instances of classes do alloc heap, and we free them ourself all the time. So that would work for closures, if desired (I am fine with it not being desired)

[1] => I understand that if smart pointers ever come, then ref-counting becomes available for more than just closures.
And in that case, especially (but not limited to) if closures would not be anonymous, but declared upfront, the declaration could include if recounting should apply.
(yes probably over engineered, but just as a point)

In the end certain concepts will end up tightly bound together as if they were one. There is no issue with that.
At the moment I can not help that many of those talking about it, are doing so with ignorance to those concepts.

It was just mentioned that an anonymous function (because it will be a closure) can not be used as a substitute for a normal callback (such as for TStringList.FOnCompareText)

Now FOnCompareText is a method pointer. Lets say we needed a function pointer: type TOnFoo = function: boolean;

Code: Pascal  [Select][+][-]

1. OnFoo := function: boolean begin .... end;

Currently (the way it is currently thought of afaik) that would create a closure, and would not work.

What if there were

Code: Pascal  [Select][+][-]

1. OnFooFunc    := function: boolean begin .... end;
2. OnFooMeth    := function of object: boolean begin .... end;
3. OnFooClosure := nested function: boolean begin .... end;

Creating:
- A plain unnamed function. This does *not* have access to the outer functions variables. It is not a nested function.
- A plain unnamed method. Only work, if the outer routine is a method. This does *not* have access to the outer functions variables, except it gets a copy of "self". It is not nested.
- A nested function, which becomes a closure.

Mind that a closure could also/optional be declared with a name, like a normal nested function. It would then be instantiated (i.e. outer vars would be captured) by the "@" operator, or with a dedicated operator, keyword, or similar.
I have not quite followed, if reference to normal nested function work and how, and if they differ from closures? If so, then the last item would need splitting.

With those distinction for anonymous functions, it would be possible to write code like this

Code: Pascal  [Select][+][-]

1. ATStringList.OnCompareText := function of object(const s1, s2: string): PtrInt begin ... end;

And that is without changing the current signature of TStringList.FOnCompareText

Personally, I think declaring a closure up-front (with a name, like a nested function), is way more pascal-ish. But I do see that there are cases, where having them inline can be a blessing. Still, at least, have the syntax make it very clear that indeed they are a closure.

And, yes, I know, there is the fact that we also need a version of what Delphi does....

[PascalDragon]

But instances of classes do alloc heap, and we free them ourself all the time. So that would work for closures, if desired (I am fine with it not being desired)

The thing is that users don't care about this. They see something that behaves like a method variable, so they'll happily throw it around and then complain about memory leaks (I'm ignoring the cycle reference in this discussion, because that is a really rare case).

Currently (the way it is currently thought of afaik) that would create a closure, and would not work.

What if there were

Code: Pascal  [Select][+][-]

1. OnFooFunc    := function: boolean begin .... end;
2. OnFooMeth    := function of object: boolean begin .... end;
3. OnFooClosure := nested function: boolean begin .... end;

Creating:
- A plain unnamed function. This does *not* have access to the outer functions variables. It is not a nested function.
- A plain unnamed method. Only work, if the outer routine is a method. This does *not* have access to the outer functions variables, except it gets a copy of "self". It is not nested.
- A nested function, which becomes a closure.

I don't think this form of micro management is necessary. The compiler could handle this a bit more intelligent: if an anonymous function does not access the outer scope then it can be passed as a procedure variable, a method variable or a function reference. If it accesses Self then it can be passed as a method variable or a function reference and if it accesses the outer function's scope then it can only be passed as a function reference. The compiler can then complain about incompatible types and maybe even point out that it is referencing Self or something from the outer scope.

Mind that a closure could also/optional be declared with a name, like a normal nested function.

An idea of mine was to allow the assignment of nested functions to function references as well. The compiler would do the necessary behind the scenes work to convert the nested function to a function reference.

It would then be instantiated (i.e. outer vars would be captured) by the "@" operator, or with a dedicated operator, keyword, or similar.
I have not quite followed, if reference to normal nested function work and how, and if they differ from closures? If so, then the last item would need splitting.

This is one of the points where marcov agreed with a syntax extension of mine:

Code: Pascal  [Select][+][-]

1. var
2.   x, y, z: LongInt;
3.   func: reference to procedure;
4. begin
5.   func := procedure with var x, const y, z begin ... end;
6. end;

In this case x would be passed by-reference (just as they are done in Delphi; it would also be the default at least in mode Delphi). y would be copied and inside the anonymous function be treated as const and z would be passed by-value. This allows control how variables are imported into the "closure".

[Martin_fr]

The thing is that users don't care about this.

<just a case in point - not serious or important>
User can learn to care. New users make other common mistakes too, until they learn that Pascal is not Javascript or whatever they knew before...
Anyway, I do not mind the implied ref counting. Especially since that is really common for closures across other languages.
</just a case in point - not serious or important>

I've just seen occasional references to the need for closures, when what was really desired was smart pointers. (ref counted pointer types).
But ref-counted closures do not prevent smart pointers. So that is ok.

I don't think this form of micro management is necessary. The compiler could handle this a bit more intelligent: if an anonymous function does not access the outer scope then it can be passed as a procedure variable, a method variable or a function reference. If it accesses Self then it can be passed as a method variable or a function reference and if it accesses the outer function's scope then it can only be passed as a function reference.

You mean the compiler looks at the type of the LHS expression?
- If the LHS target is declared a method, then it takes any anonymous function, except those with references to outer scope vars.
  A self will be passed, even if not needed (i.e. the code is compiled as method, so it is compatible with the type of the variable in which it is stored)
- If the LHS target is declared a plain function, then it takes only anonymous function, that do not have references to self or outer scope vars.

That be nice.

But

Code: Pascal  [Select][+][-]

1. begin
2.   procedure(A: Integer) begin foo(a+self.b); self:=nil; end(1);  // calling an anonymous func in place
3. end;

The compiler could not know what it should be.
-Is the above a method, that is called with the outer "self"
-Or is it a closure, that captures the outer "self"
It makes a difference, because it contains "self:=nil;", because a closure, with a reference to self, would modify the outer self too.

Mind that a closure could also/optional be declared with a name, like a normal nested function.

An idea of mine was to allow the assignment of nested functions to function references as well. The compiler would do the necessary behind the scenes work to convert the nested function to a function reference.

Close enough. Though I do not see, how to fit the below syntax extension.

I assume, capturing of the valuing will be done, when the ref is taken? Where the "@" operator is?

This is one of the points where marcov agreed with a syntax extension of mine:

Code: Pascal  [Select][+][-]

1. var
2.   x, y, z: LongInt;
3.   func: reference to procedure;
4. begin
5.   func := procedure with var x, const y, z begin ... end;
6. end;

In this case x would be passed by-reference (just as they are done in Delphi; it would also be the default at least in mode Delphi). y would be copied and inside the anonymous function be treated as const and z would be passed by-value. This allows control how variables are imported into the "closure".

Maybe in brackets?

Code: Pascal  [Select][+][-]

1.   func := procedure with (var x, const y, z) begin ... end;

It is kind of like a param list.
Also it makes it more clear that the "var" and "const" act as they do in a param list, and not as opening a var block, defining local variables.
(and yes, I noted that it differs anyway, as no types are needed).
Within the () it would even be possible to use ";" and have the "," for its usual list purpose.

Code: Pascal  [Select][+][-]

1.   func := procedure with (var x; const y, y2; z) begin ... end;

[Otto]

 Where can I find documentation of the FPC development plans related to the topics covered in the latest posts (anonymous functions, anonymous methods,closure, et cetera)?

 
@440bx
 
I read all the contents of the thread you referred to in your last post. In my opinion, the fundamental problem lies in the fact that some programmers only consider the code they write without considering everything that will be "added" and reworked by the compiler; moreover, a small minority believe that the OOP paradigm is implemented in the same way in every programming language; whereas, in my opinion, it is not a uniquely defined concept and should be used very carefully.

[Mr.Madguy]

I don't want to participate in OOP vs non-OOP holywar, but I just think, that all syntax sugar exists only to make programmers' life easier and make code shorter and more clear via automating routine operations. The same way, mathematicians use ☐ψ = 0 instead of δ²ψ/δx² + δ²ψ/δy² + δ²ψ/δz² - 1/c² * δ²ψ/δt² = 0. All you need - to understand, what happens behind this syntax sugar and use it properly. I.e. use it to solve proper problem. Not just for the sake of using it.

Simple thing. Whole high level programming is just syntax sugar around assembler. Why don't you use assembler then?

And closures are very powerful. They are reversed OOP. OOP allows procedures/functions to be bound to data. And closures allow you to bind some data to procedure/function without wasting your time on declaring object/interface for this purpose, that you would need otherwise.

Simple callback will never allow you to do this:

Code: Pascal  [Select][+][-]

1. procedure SomeProcedure;
2.   var MyList:TStrings;
3. begin
4.   MyList := TStringList.Create;
5.   SomeProcedureThatUsesCallbackAndCallsItSeveralTimes(
6.     procedure(X:String)
7.     begin
8.       MyList.Add(X);
9.     end
10.   );
11. end;
12.  

Only object can allow you to create local copy of MyList. I.e. not global variable, that isn't recursion/thread/etc safe. But declaring object - is just a waste of time and effort, if you need such operation only once in your life, so it will never be reused. Static callback can't do that. It requires data to be passed to it via parameters. And such solution isn't always available.