Recent

Author Topic: Feature announcement: Implicit generic function specializations  (Read 9042 times)

PascalDragon

  • Hero Member
  • *****
  • Posts: 5672
  • Compiler Developer
Dear FPC community,

The FPC developers are pleased to announce the implementation of a new feature: implicit generic function specializations. This feature was implemented by Ryan Joseph, so thank you very much, Ryan.

This feature allows you to use generic routines (functions, procedures, methods) without explicitely specializing them (“<…>” in Delphi modes and “specialize …<…>” in non-Delphi modes) as long as the compiler can determine the correct parameter types for the generic.

This feature is enabled with the modeswitch ImplicitFunctionSpecialization and is for now not enabled by default as this has the potential to break existing code.

Assume you have the following function:

Code: Pascal  [Select][+][-]
  1. generic function Add<T>(aArg1, aArg2: T): T;
  2. begin
  3.   Result := aArg1 + aArg2;
  4. end;

Up to now you could only use this function as follows:

Code: Pascal  [Select][+][-]
  1. SomeStr := specialize Add<String>('Hello', 'World');
  2. SomeInt := specialize Add<LongInt>(2, 5);

However with implicit function specializations enabled you can also use it as follows:

Code: Pascal  [Select][+][-]
  1. SomeStr := Add('Hello', 'World');
  2. SomeInt := Add(2, 5);

The compiler will automatically determine the type of the generic parameters based on the parameters you pass in (this is always done left to right). Depending on the passed in parameters (especially if you're using constant values like in the example instead of variables) the compiler might however pick a different type than you expected. You can enforce a specific type by either explicitely specializing the method as before or by inserting a type cast. In the example above the compile will specialize the call with the parameters “2, 5” using an 8-bit signed type (Pascal prefers signed types) instead of a LongInt as in the explicit specialization. If you use “LongInt(2), 5” as parameters then the compiler will pick that instead, however with “2, LongInt(5)” it will still pick an 8-bit type, because the parameter types are determined left to right.

If there exists a non-generic overload for which the parameters types match exactly, the compiler will pick that instead of specializing something anew. So assume you also have the following function in scope:

Code: Pascal  [Select][+][-]
  1. function Add(aArg1, aArg2: LongInt): LongInt;
  2. begin
  3.   Result := aArg1 + aArg2;
  4. end;

In the case of “Add(2, 5)” the compiler will not pick the non-generic function, because it determines that an 8-bit type is enough, however if you use “Add(LongInt(2), 5)” the compiler will pick the non-generic function.

Aside from simple parameters the compiler also supports arrays and function/method variables:

Code: Pascal  [Select][+][-]
  1. generic function ArrayFunc<T>(aArg: specialize TArray<T>): T;
  2. var
  3.   e: T;
  4. begin
  5.   Result := Default(T);
  6.   for e in aArg do
  7.     Result := Result + e;
  8. end;
  9.  
  10. type
  11.   generic TTest<T> = function(aArg: T): T;
  12.  
  13. generic function Apply<T>(aFunc: specialize TTest<T>; aArg: T): T;
  14. begin
  15.   Result := aFunc(aArg);
  16. end;
  17.  
  18. function StrFunc(aArg: String): String;
  19. begin
  20.   Result := UpCase(aArg);
  21. end;
  22.  
  23. function NegFunc(aArg: LongInt): LongInt;
  24. begin
  25.   Result := - aArg;
  26. end;
  27.  
  28. begin
  29.   Writeln(ArrayFunc([1, 2, 3])); // will write 6
  30.   Writeln(ArrayFunc(['Hello', 'FPC', 'World'])); // will write HelloFPCWorld
  31.  
  32.   Writeln(Apply(@StrFunc, 'Foobar')); // will write FOOBAR
  33.   Writeln(Apply(@NegFunc, 42)); // will write -42
  34. end.

There are of course a few restrictions for this feature:
  • all generic parameters must be used in the declaration of the routine (implementation only type parameters are not allowed)
  • all parameters that have a generic type must not be default parameters, they need to be used in the call or their type must have been fixed by a parameter further left (as currently default values for parameters of a generic type are not supported this is not much of a restriction, but should that change (e.g. Default(T)) then this restriction will apply)
  • the generic routine must not have constant generic parameters (this might be extended in the future with e.g. static arrays or file types, but for now this restriction stands)
  • the result type is not taken into account, so if only the result type of a routine is generic then an implicit specialization does not work either
  • function/method pointers to implicit specializations are not yet supported (pointers to explicit specializations are not yet supported either; once this changes the former will change as well)
  • the compiler will silently discard generic functions that it can't specialize the declaration of; however if the declaration can be specialized correctly, but for whatever reason the implementation can not then this will trigger a compilation error

This feature is by and in itself Delphi-compatible however there might be differences in what FPC can implicitely specialize and what Delphi can. Especially if Delphi can specialize something that FPC can not, this should be reported.

mas steindorff

  • Hero Member
  • *****
  • Posts: 545
Re: Feature announcement: Implicit generic function specializations
« Reply #1 on: April 20, 2022, 08:16:46 pm »
this sounds like it's just the ticket for what I'm working with. 
What version /package / ?? do I need to download to access these features?
windows 10 &11, Ubuntu 21+ IDE 3.4 general releases

Zaher

  • Hero Member
  • *****
  • Posts: 683
    • parmaja.org
Re: Feature announcement: Implicit generic function specializations
« Reply #2 on: April 20, 2022, 11:38:00 pm »
I love generics

Okoba

  • Hero Member
  • *****
  • Posts: 533
Re: Feature announcement: Implicit generic function specializations
« Reply #3 on: April 21, 2022, 09:23:14 am »
Thank you Sven and specially Ryan. This is a fantastic upgrade.

Okoba

  • Hero Member
  • *****
  • Posts: 533
Re: Feature announcement: Implicit generic function specializations
« Reply #4 on: April 21, 2022, 09:30:24 am »
If I have two overloaded functions in a unit, FPC detects the the correct one but if I split them in two units, it will raise an error:
Quote
project1.lpr(22,8) Error: Wrong number of parameters specified for call to "Test"
Before there was not any error, but now, even without the ImplicitFunctionSpecialization mode switch, there is this error.
Version I am testing: Lazarus 2.3.0 (rev main-2_3-1668-g34b3b9a49a) FPC 3.3.1 x86_64-win64-win32/win64

This code works:
Code: Pascal  [Select][+][-]
  1. program Project1;
  2.  
  3. {$mode Delphi}
  4.  
  5.   function Test(const AString: Rawbytestring): Integer; overload;
  6.   begin
  7.  
  8.   end;
  9.  
  10.   function Test<T>(const AList: TArray<T>): Integer; overload;
  11.   begin
  12.  
  13.   end;
  14.  
  15. var
  16.   S: Rawbytestring;
  17.   I: Integer;
  18. begin
  19.   I := Test(S);
  20. end.                    

This raise the error:

Code: Pascal  [Select][+][-]
  1. unit Unit1;
  2.  
  3. {$mode Delphi}
  4.  
  5. interface
  6.  
  7. function Test(const AString: Rawbytestring): Integer; overload;
  8.  
  9. implementation
  10.  
  11. function Test(const AString: Rawbytestring): Integer;
  12. begin
  13.  
  14. end;
  15.  
  16. end.
  17.  

Code: Pascal  [Select][+][-]
  1. unit Unit2;
  2.  
  3. {$mode Delphi}
  4.  
  5. interface
  6.  
  7. function Test<T>(const AList: TArray<T>): Integer; overload;
  8.  
  9. implementation
  10.  
  11. function Test<T>(const AList: TArray<T>): Integer;
  12. begin
  13.  
  14. end;
  15.  
  16. end.
  17.  

Code: Pascal  [Select][+][-]
  1. program Project1;
  2.  
  3. {$mode Delphi}
  4.  
  5. uses unit1, unit2;
  6.  
  7. var
  8.   S: Rawbytestring;
  9.   I: Integer;
  10. begin
  11.   I := Test(S);
  12. end.              
  13.  


marcov

  • Administrator
  • Hero Member
  • *
  • Posts: 11763
  • FPC developer.
Re: Feature announcement: Implicit generic function specializations
« Reply #5 on: April 21, 2022, 10:37:30 am »
If I have two overloaded functions in a unit, FPC detects the the correct one but if I split them in two units, it will raise an error:

(possibly related to  https://gitlab.com/freepascal.org/fpc/source/-/issues/39673 that I found last week)



Warfley

  • Hero Member
  • *****
  • Posts: 1580
Re: Feature announcement: Implicit generic function specializations
« Reply #6 on: April 21, 2022, 10:39:01 am »
The implicit specialization does not seem to handle inheritance very well:
Code: Pascal  [Select][+][-]
  1. program Project1;
  2.  
  3. {$mode objfpc}{$H+}
  4. {$ModeSwitch implicitfunctionspecialization}
  5.  
  6. uses
  7.   Generics.Collections;
  8.  
  9. generic procedure Foo<T>(lst: specialize TEnumerable<T>);
  10. begin
  11. end;
  12.  
  13. var
  14.   lst: specialize TList<Integer>; // Inherits from TEnumerable
  15. begin              
  16.   Foo(lst); // Error
  17.   specialize Foo<Integer>(lst); // works
  18. end.

PascalDragon

  • Hero Member
  • *****
  • Posts: 5672
  • Compiler Developer
Re: Feature announcement: Implicit generic function specializations
« Reply #7 on: April 21, 2022, 02:22:26 pm »
this sounds like it's just the ticket for what I'm working with. 
What version /package / ?? do I need to download to access these features?

New features are first available only in the development version (aka main) until the next major release (which will be 3.4.0 in some undetermined time in the future). So you'll have to clone the Git repository and compile FPC yourself. Or as is the hype nowadays: use FpcUpDeluxe.

If I have two overloaded functions in a unit, FPC detects the the correct one but if I split them in two units, it will raise an error:

(possibly related to  https://gitlab.com/freepascal.org/fpc/source/-/issues/39673 that I found last week)

Maybe, yes...

@Okoba: please report it nevertheless with a nice, simple example.

The implicit specialization does not seem to handle inheritance very well:
Code: Pascal  [Select][+][-]
  1. program Project1;
  2.  
  3. {$mode objfpc}{$H+}
  4. {$ModeSwitch implicitfunctionspecialization}
  5.  
  6. uses
  7.   Generics.Collections;
  8.  
  9. generic procedure Foo<T>(lst: specialize TEnumerable<T>);
  10. begin
  11. end;
  12.  
  13. var
  14.   lst: specialize TList<Integer>; // Inherits from TEnumerable
  15. begin              
  16.   Foo(lst); // Error
  17.   specialize Foo<Integer>(lst); // works
  18. end.

Please report.


Okoba

  • Hero Member
  • *****
  • Posts: 533
Re: Feature announcement: Implicit generic function specializations
« Reply #8 on: April 21, 2022, 03:35:35 pm »
Reported all at #39677, #39678 and #39680.

PascalDragon

  • Hero Member
  • *****
  • Posts: 5672
  • Compiler Developer
Re: Feature announcement: Implicit generic function specializations
« Reply #9 on: April 21, 2022, 06:23:04 pm »
Reported all at #39677, #39678 and #39680.

Thank you.

Warfley

  • Hero Member
  • *****
  • Posts: 1580
Re: Feature announcement: Implicit generic function specializations
« Reply #10 on: April 22, 2022, 09:54:32 pm »
Some thing I've found, which I did not report as a bug yet, because I don't know if it is desired (as this might very well introduce a load of unwanted side effects), is that implicit specialization is not recognizing possible implicit casts:
Code: Pascal  [Select][+][-]
  1. program Project1;
  2.  
  3. {$mode objfpc}{$H+}
  4. {$ModeSwitch advancedrecords}
  5. {$ModeSwitch implicitfunctionspecialization}
  6.  
  7. type
  8.   generic TTest<T> = record
  9.     class operator :=(const rhs: T): specialize TTest<T>;
  10.   end;
  11.  
  12. generic procedure Foo<T>(arg: specialize TTest<T>);
  13. begin
  14. end;
  15.  
  16. begin
  17.   specialize foo<Integer>(42); // Works
  18.   foo(42); // Error
  19. end.
But it could be useful with for example the TNullable type

PascalDragon

  • Hero Member
  • *****
  • Posts: 5672
  • Compiler Developer
Re: Feature announcement: Implicit generic function specializations
« Reply #11 on: April 23, 2022, 12:31:46 pm »
Some thing I've found, which I did not report as a bug yet, because I don't know if it is desired (as this might very well introduce a load of unwanted side effects), is that implicit specialization is not recognizing possible implicit casts:

Currently this is indeed by design as at least Delphi does not allow it either. And I personally prefer that we don't open that pandora's box... (especially es it might become problematic to determine what is a suitable specialization if there are multiple valid implicit assignments to do this in more complex scenarios)

corpsman

  • New Member
  • *
  • Posts: 11
    • My Delphi / Lazarus Programming site ( www.Corpsman.de )
Re: Feature announcement: Implicit generic function specializations
« Reply #12 on: October 04, 2024, 11:30:02 am »
Hello all,
the implicit generic specialization is exactly what i was searching for, but. In the usermanual of FPC 3.2.2 ( https://sourceforge.net/projects/freepascal/files/Documentation/3.2.2/ ) it is not listed anymore.

Also my FPC compiler
Quote
corpsman@corpsman:~$ fpc -h
Free Pascal Compiler version 3.2.2+dfsg-9ubuntu1 [2022/04/11] for x86_64
Throws the message
Quote
project1.lpr(4,2) Warning: Illegal compiler switch "IMPLICITFUNCTIONSPECIALIZATION+"
when i try to compile this:
Code: Pascal  [Select][+][-]
  1. program Project1;
  2. {$mode objfpc}{$H+}
  3.  
  4. {$ModeSwitch implicitfunctionspecialization}
  5. begin
  6.  
  7. end.
  8.  
  9.  
Did i miss something ?
--
Just try it.

Warfley

  • Hero Member
  • *****
  • Posts: 1580
Re: Feature announcement: Implicit generic function specializations
« Reply #13 on: October 04, 2024, 12:28:59 pm »
Did i miss something ?

FPC 3.2.2 is from May 2021, this feature was introduced in 2022 to Trunk/Main so it will be only in the next major version. Also don't expect too much it has quite some limitations, so better be ready to still write "specialize ...<types>" for some time to come

dbannon

  • Hero Member
  • *****
  • Posts: 3069
    • tomboy-ng, a rewrite of the classic Tomboy
Re: Feature announcement: Implicit generic function specializations
« Reply #14 on: October 04, 2024, 01:24:40 pm »
Did i miss something ?

Yes, several posts further up, PascalDragon said -

New features are first available only in the development version (aka main) until the next major release (which will be 3.4.0 in some undetermined time in the future). So you'll have to clone the Git repository and compile FPC yourself. Or as is the hype nowadays: use FpcUpDeluxe.


3.4.0 release date is still undetermined.

Davo
Lazarus 3, Linux (and reluctantly Win10/11, OSX Monterey)
My Project - https://github.com/tomboy-notes/tomboy-ng and my github - https://github.com/davidbannon

 

TinyPortal © 2005-2018