Lazarus

Free Pascal => General => Topic started by: 440bx on June 11, 2019, 05:55:00 pm

Title: Considerations in the decision of adding a feature to the FPC compiler?
Post by: 440bx on June 11, 2019, 05:55:00 pm
Hello,

Obviously, if a feature is present in Delphi it will likely, at least, be considered for inclusion in FPC.

OTH, there are features in FPC that are not in Delphi and probably will never be in Delphi - such as +=, *=, etc - and those are the kinds of features I'd like to know how they get to make it into FPC.

Is there some somewhat "formal"/"structured" process that leads to the decision of including/excluding such features ?

Is developing the Pascal language _beyond_ any current implementation something that is of interest to the FPC developers ?

thank you.
Title: Re: Considerations in the decision of adding a feature to the FPC compiler?
Post by: Thaddy on June 11, 2019, 06:47:29 pm
Any feature is considered as you know from the developer list.
But some "features" are likely to be not implemented ever.
E.g: inline variables.... :o :D ;) O:-)

And Delphi is not the holy grail: They dropped some features from their last beta's that work in FPC like a charm: e.g (again) they dropped management operators for their releases, which are perfectly OK in FPC...... O:-)
OTOH I miss one single feature that Delphi has and FPC doesn't: Anonymous methods. That's the only one.(mind you, this is is costly regarding performance in Delphi!)
But in Delphi I miss even more important features like free standing operator overloads...... 8-)
Title: Re: Considerations in the decision of adding a feature to the FPC compiler?
Post by: marcov on June 11, 2019, 07:08:51 pm
Obviously, if a feature is present in Delphi it will likely, at least, be considered for inclusion in FPC.

But not necessarily now, and doing certain features *well* often relies on deeper work, and a quick hack solution will often not be accepted.

Quote
OTH, there are features in FPC that are not in Delphi and probably will never be in Delphi - such as +=, *=, etc - and those are the kinds of features I'd like to know how they get to make it into FPC.

Those are 2 decades old, so that is hard to answer, back then people like Peter or Florian simply committed them.

Quote
Is there some somewhat "formal"/"structured" process that leads to the decision of including/excluding such features ?

No. If a committer wants to commit them, there is sometimes discussion on maillists. Unfortunately IMHO it gets accepted way too often with the argument "you don't have to use it if you don't like it".

Quote
Is developing the Pascal language _beyond_ any current implementation something that is of interest to the FPC developers ?

This is a very coloured question, and as such unanswerable.

But IMHO FPC already got way to many redundant micro-syntax extensions (all of which were described by the authors as the axis that the whole world revolves around and as such a matter of life and death). But such features and the interactions with other features and platforms (new or existing) have to be maintained indefinitely, a fact that is often grossly underestimated by people peddling new syntax (which are often "done" if it works for x86 or x64)

FPC has a mission to build a production compiler, and that means all facets of the projects, not just language. For experimenting with language extensions, non compatible faster moving projects or forks are better suited I think.

I already think the rate of new (non compatible) syntax adoption in recent years has been to high.
Title: Re: Considerations in the decision of adding a feature to the FPC compiler?
Post by: Thaddy on June 11, 2019, 07:14:34 pm
[I already think the rate of new (non compatible) syntax adoption in recent years has been to high.
I agree.
The features are nice like a kid in the candy box, but they are currently overwhelming.
Only old hands can see through it.
Title: Re: Considerations in the decision of adding a feature to the FPC compiler?
Post by: 440bx on June 11, 2019, 07:59:41 pm
@Marco

I share your feeling that FPC includes features that are of "dubious" value.

From what you stated, I get the impression that, there really isn't a structured approach to deciding which features are good candidates to be added to the language.

Is developing the Pascal language _beyond_ any current implementation something that is of interest to the FPC developers ?

This is a very coloured question, and as such unanswerable.
I don't think so (but I don't want to argue about it.) 

IMO, there should be an individual or group of individuals who have a vision of where FPC is going as a flavor of Pascal and not just as a mostly "Delphi compatible" implementation of the language.

It's fairly obvious that, at one time, Florian (and possibly others) had a clear vision of what FPC was supposed to be and where it was going, which was, providing a Turbo Pascal/Delphi compatible implementation of the language.  As Borland/Inprise/Embarcadero fiddled with the language definition in ways that were "questionable", that goal became, justifiably, less desirable.

At this time, I am under the impression that Delphi compatibility, while still important, is no longer the compass that directs FPC's development.  Because of that, I think the developers should consider directions that are no longer associated with Delphi at all.  IOW, design that is Pascal in spirit but no longer Delphi.

The point I'm carefully leading to is, there are a significant number of very useful and very desirable features in other languages that can be implemented in a Pascal way that would be wonderful to have in FPC and would set FPC apart, not just as a Delphi compatible language but, as its own Pascal implementation.

Doing something like that requires a well thought out, well structured mechanism to decide which features will be included and how they will be implemented. No willy-nilly stuff, otherwise it is the seed of a mess.

That's what I'm really trying to find out, is there some thinking along those lines taking place among the FPC developers or will the design of the language remain a somewhat casual process ?.



Title: Re: Considerations in the decision of adding a feature to the FPC compiler?
Post by: marcov on June 11, 2019, 08:19:25 pm
IMO, there should be an individual or group of individuals who have a vision of where FPC is going as a flavor of Pascal and not just as a mostly "Delphi compatible" implementation of the language.

That is exact the mentality that got us all of the junk expansions.

Quote
It's fairly obvious that, at one time, Florian (and possibly others) had a clear vision of what FPC was supposed to be and where it was going, which was, providing a Turbo Pascal/Delphi compatible implementation of the language.  As Borland/Inprise/Embarcadero fiddled with the language definition in ways that were "questionable", that goal became, justifiably, less desirable.

No, the old core people simply got old, and bowed to pressure of newcomers that directly want to implement new language features.

In the old days, people would have to have at least done years of allround core work to qualify for that. That encouraged a certain sense of realism wrt language features.

That said, back then it was easier to be allround since the project was smaller and more overviewable.


Title: Re: Considerations in the decision of adding a feature to the FPC compiler?
Post by: 440bx on June 11, 2019, 08:34:18 pm
IMO, there should be an individual or group of individuals who have a vision of where FPC is going as a flavor of Pascal and not just as a mostly "Delphi compatible" implementation of the language.

That is exact the mentality that got us all of the junk expansions.
That's what a formal/structured approach to the design of the language would prevent: the addition of junk. I don't like junk features anymore than you do.

No, the old core people simply got old, and bowed to pressure of newcomers that directly want to implement new language features.
Maybe but, it could also be simply driven by a desire to prevent giving the impression that the language is languishing because no new features are added to it.  I have a feeling that's mostly what the addition of inline variables is all about with Delphi.

That said, back then it was easier to be allround since the project was smaller and more overviewable.
You're right about that, FPC is no small project anymore.  That's one of the reasons I feel a more formal method and clear vision are necessary.

There are lots of ways FPC can be made even better than it is but, it's important to avoid ending in hell while paving the road with good intentions.
Title: Re: Considerations in the decision of adding a feature to the FPC compiler?
Post by: marcov on June 11, 2019, 09:46:45 pm
I forgot two other things.

Very long ago I wrote this about language extensions: https://www.freepascal.org/faq.html#extensionselect

And while not all developers agreed 100%, the central tenet of that, "it must make things possible that are not possible now" must be clear. Short syntax only is not desirable.

I don't think I have seen an NEW interesting new language feature in years.
Title: Re: Considerations in the decision of adding a feature to the FPC compiler?
Post by: kupferstecher on June 11, 2019, 10:01:47 pm
In my opinion the feature list is already too long. It's nearly impossible to know all the possibilities, the syntax options and all side effects of the various features of fpc.
So instead of introducing new features I would suggest to do some tidy up.
E.g.
- The writable typed constant concept as local static variable is counter intuitive, perhaps constants should never be variables.
- Classes, objects, advanced records. Thats one to much.
- Modula-2 syntax for blocks: if ... then ... end; for... do ... end;
- Passing constant values to var parameters should be possible. (Well, this would be backward compatible)
- Parameters as var, out, const, constref... perhaps this could be done better.
- Volatility (I know it was introduced, but as I understand if the compiler really builds on it, this would break existing code)

These are just examples, don't need to discuss them. I also know that there are very good reasons why it is done the way it currently is. Furthermore, breaking backward compatibility is a huge step.

But still I think this, a new dialect, would make the language much more modern than adding random features.
Probably, though,  its not possible to be realized with the existing resources.
Title: Re: Considerations in the decision of adding a feature to the FPC compiler?
Post by: 440bx on June 11, 2019, 10:12:12 pm
Short syntax only is not desirable.
I definitely agree with that.

I don't think I have seen an NEW interesting new language feature in years.
There are plenty of _OLD_ very interesting language features that would fit very nicely in FPC.

The best feature a language can offer is the writing of code that is easy to understand while being able to flag programming errors at compile time.

That's one of the characteristics of Pascal and it can be made even better.

As far as interesting new features are concerned, I think you should get out more. ;)  One of the things that seems to have dawned on some MS programmers is that a program can be created by _describing_ it instead of programming it.  The increase in productivity is quite substantial but, at least at this time, the loss of performance is too high for the method to be implemented beyond user interfaces.

It's a little bit like SQL. You describe the set of data you want, you don't write code to build the set, you just describe it and let the SQL engine figure out how to build it.  Compilers could gain from that approach (which is what Delphi is based on actually but, they never developed it beyond the initial conceptualization.)


ETA:

In my opinion the feature list is already too long. It's nearly impossible to know all the possibilities, the syntax options and all side effects of the various features of fpc.
That is true.  The real problem is that there are a lot of features whose benefit is marginal or too limited.

As you said, some tidying up would be good but, the backwards compatibility requirements make that either difficult or simply not possible.

Title: Re: Considerations in the decision of adding a feature to the FPC compiler?
Post by: Martin_fr on June 11, 2019, 10:47:12 pm
I already think the rate of new (non compatible) syntax adoption in recent years has been to high.
And I still fear the coming of "inline procedure declarations". Though I haven't followed it recently and may be wrong. If memory serves, most of those wanting this, did actually want closures. And that is what it is going to be.
Only to do closures, you do not need to declare procedures inline. Those should be 2 completely independent things. (And the inline declaration should not actually be a thing at all)... 
Title: Re: Considerations in the decision of adding a feature to the FPC compiler?
Post by: BeniBela on June 11, 2019, 11:22:54 pm
A big problem is that too many features are put in the compiler, when they should be in libraries. But the features cannot be  in libraries, since there is no support for proper meta-programming like nim/rust/perl-like macros.  For example
 
Advanced records could have been a single macro that converts all records to objects.

Enumerators/for..in are new.  C++ did not have anything like that, but Qt added its own foreach just using macros.

Or the parametrized  exit(abc) function is a compiler feature. But it could have simply been written as a macro to convert it to begin result := abc; exit; end

There is the case statement, and recently it was extended for strings.  Perl on the other hand does not have a case statement, but there is a Perl module that dynamically rewrites the code to convert a case-like statement to ifs or something.


Or strings/arrays/interfaces as different refcounted types. They could be replaced by records in libraries using (Maciej's) smartpointers.


All compiler features could be replaced by the single compiler  feature of a good macro system
Title: Re: Considerations in the decision of adding a feature to the FPC compiler?
Post by: 440bx on June 11, 2019, 11:54:05 pm
All compiler features could be replaced by the single compiler  feature of a good macro system
That's true but, there is a big disadvantage in using that method, which is, the code you _read_ is not the same as the code that is _compiled_.

In addition to that, you also have to deal with the macro language's "personality", e.g, forgetting to enclose a macro parameter in parentheses can lead to some very subtle, occasional, erratic, hard to track down, bugs.

I believe the programmer and the compiler should see exactly the same code.  It really tends to make things a lot easier that way.



Title: Re: Considerations in the decision of adding a feature to the FPC compiler?
Post by: Leledumbo on June 12, 2019, 01:56:13 am
All compiler features could be replaced by the single compiler  feature of a good macro system
That's true but, there is a big disadvantage in using that method, which is, the code you _read_ is not the same as the code that is _compiled_.

In addition to that, you also have to deal with the macro language's "personality", e.g, forgetting to enclose a macro parameter in parentheses can lead to some very subtle, occasional, erratic, hard to track down, bugs.

I believe the programmer and the compiler should see exactly the same code.  It really tends to make things a lot easier that way.
Indeed this is why I avoid macro driven language. It looks cool in the beginning, but quickly becomes nightmare when it comes to maintenance. However, Nim style macro is acceptable to me, as it works at syntax tree level (tree node manipulation) as opposed to source code level (string -> string tranformation), thus reducing the misleading error messages and such. The problem, FPC syntax tree was not designed for that since the beginning, as how Nim one was designed from the start.
Title: Re: Considerations in the decision of adding a feature to the FPC compiler?
Post by: 440bx on June 12, 2019, 02:51:44 am
Indeed this is why I avoid macro driven language. It looks cool in the beginning, but quickly becomes nightmare when it comes to maintenance. However, Nim style macro is acceptable to me, as it works at syntax tree level (tree node manipulation) as opposed to source code level (string -> string tranformation), thus reducing the misleading error messages and such. The problem, FPC syntax tree was not designed for that since the beginning, as how Nim one was designed from the start.
There are a number of things I like about Nim (disclosure: I only have a passing familiarity with the language) but, I think of Nim as a macro language itself since it doesn't compile to native code, it produces C. 

I can't help thinking of such a setup as two (2) potential sources of problems instead of just one (1). 
Title: Re: Considerations in the decision of adding a feature to the FPC compiler?
Post by: marcov on June 12, 2019, 08:28:10 am
What Leledumbo says, and that also illustrates that one should be very careful with borrowing features. What it ends up being might be different from how it is in a different language.

The classic example is GC. People compare languages with movable, generational GCs designed into the core, and then say "Yes, but you can also whack a GC under C, so it should also be possible for Pascal". NO, not the SAME  GC :_)

Anyway, radical changes are not considered AT ALL anyway.

And personally, I think the undirected drive to modernization is just a pretext for forcing random language experimentation into a project where it doesn't belong.
Title: Re: Considerations in the decision of adding a feature to the FPC compiler?
Post by: PascalDragon on June 12, 2019, 09:14:01 am
Quote
Is there some somewhat "formal"/"structured" process that leads to the decision of including/excluding such features ?

No. If a committer wants to commit them, there is sometimes discussion on maillists. Unfortunately IMHO it gets accepted way too often with the argument "you don't have to use it if you don't like it".
I think the recent discussions on fpc-devel showed that we're still not accepting everything and the kitchen sink. ;)

In my opinion the feature list is already too long. It's nearly impossible to know all the possibilities, the syntax options and all side effects of the various features of fpc.
So instead of introducing new features I would suggest to do some tidy up.
E.g.
- The writable typed constant concept as local static variable is counter intuitive, perhaps constants should never be variables.
- Classes, objects, advanced records. Thats one to much.
- Modula-2 syntax for blocks: if ... then ... end; for... do ... end;
- Passing constant values to var parameters should be possible. (Well, this would be backward compatible)
- Parameters as var, out, const, constref... perhaps this could be done better.
- Volatility (I know it was introduced, but as I understand if the compiler really builds on it, this would break existing code)

These are just examples, don't need to discuss them. I also know that there are very good reasons why it is done the way it currently is. Furthermore, breaking backward compatibility is a huge step.
No. Just no. Both compatibility to TP/Delphi and backwards compatibility are important core concepts of FPC (though the Delphi compatibility got quite a chink with Delphi introducing inline variables %) ) and thus we won't restructure core concepts only because some think that it's too much. Those people can always start a new compiler, cause that's what it would be essentially.
Title: Re: Considerations in the decision of adding a feature to the FPC compiler?
Post by: SymbolicFrank on June 12, 2019, 09:19:47 am
Half my projects end up written in a different language, simply because the lack of support for the library or target. Like bigint, iOS or embedded. Mac is still not complete. And the installation still fails half the time (especially on Linux).

Ok, not as sexy as adding stuff to the environment you actively use to code, but if you want to grow support for the language, the most important feature is that you can start coding your new project right away. If it takes days or weeks of effort before you can start, most people take an alternative development platform. And the few who do make the effort end up as part of the dev team.
Title: Re: Considerations in the decision of adding a feature to the FPC compiler?
Post by: Leledumbo on June 13, 2019, 06:37:09 am
but, I think of Nim as a macro language itself since it doesn't compile to native code, it produces C. 
If you take a look at the generated C code, or how it's generated, it's far from other compilers that transpiles to C, where the C code is usually still human readable. The author treats C like an abstract architecture, hence the generated C code looks more assembly-ish than C converted code.
Title: Re: Considerations in the decision of adding a feature to the FPC compiler?
Post by: 440bx on June 13, 2019, 07:29:45 am
but, I think of Nim as a macro language itself since it doesn't compile to native code, it produces C. 
If you take a look at the generated C code, or how it's generated, it's far from other compilers that transpiles to C, where the C code is usually still human readable. The author treats C like an abstract architecture, hence the generated C code looks more assembly-ish than C converted code.
I'm not familiar enough with Nim to evaluate it properly but, the fact that a separate compiler is involved in the process of getting executable code makes me uncomfortable.  That additional compiler is, at least potentially, another source of problems.
Title: Re: Considerations in the decision of adding a feature to the FPC compiler?
Post by: Leledumbo on June 14, 2019, 01:01:33 pm
I'm not familiar enough with Nim to evaluate it properly but, the fact that a separate compiler is involved in the process of getting executable code makes me uncomfortable.  That additional compiler is, at least potentially, another source of problems.
Indeed, it was a source of attraction by supporting multiple C compilers. It actually has more than 1 backend, the compiler can generate C, C++, Objective-C and JavaScript. Much like FPC with HL codegen backend only plus Pas2JS.
Title: Re: Considerations in the decision of adding a feature to the FPC compiler?
Post by: Akira1364 on June 20, 2019, 05:37:23 am
In general, I think there's certainly a difference between mere syntax extensions and things that make it possible to write better-optimized code than you could previously.

Personally I'm hoping to see some more movement on getting the generic constants feature, shown here: https://bugs.freepascal.org/view.php?id=35140 (https://bugs.freepascal.org/view.php?id=35140) merged sometime soon.
Title: Re: Considerations in the decision of adding a feature to the FPC compiler?
Post by: Thaddy on June 20, 2019, 07:50:19 am
Well, the only thing I really miss is anonymous methods because I have a lot of dual code base between Delphi and FPC.
But that is already in progress. Freepascal has already so many features above and beyond that I can understand the hesitation to add new features that extend the language once more.
Title: Re: Considerations in the decision of adding a feature to the FPC compiler?
Post by: marcov on June 20, 2019, 09:28:30 am
And gettypekind in generics.
Title: Re: Considerations in the decision of adding a feature to the FPC compiler?
Post by: Akira1364 on June 20, 2019, 04:09:33 pm
Well, the only thing I really miss is anonymous methods because I have a lot of dual code base between Delphi and FPC.
But that is already in progress.

It's been in progress for a lot longer than it should have been, I think. Blaise seems to keep disappearing for long stretches of time and then reappearing.

And gettypekind in generics.

I'm pretty sure it works in generics at least in trunk. You can't currently use it in constants or within scanner conditionals though, which you can in Delphi. That said, I actually have a patch I've been working on that enables this, and also adds Delphi-compatible IsManagedType and IsConstValue intrinsics that I might submit soon. (FWIW, adding those intrinsics was actually... very easy, as the compiler has internal methods that provide basically exactly the required functionality.)
Title: Re: Considerations in the decision of adding a feature to the FPC compiler?
Post by: Thaddy on June 20, 2019, 04:24:36 pm
Anonymous methods are not easy to implement. Be graceful some people take the effort. And note: the way Delphi does it with hidden interfaces is not optimal.
Although notationally easy it has a performance hit.
Title: Re: Considerations in the decision of adding a feature to the FPC compiler?
Post by: Akira1364 on June 20, 2019, 04:37:49 pm
And note: the way Delphi does it with hidden interfaces is not optimal.
Although notationally easy it has a performance hit.

I'm aware. I think this is also how the FPC implementation works, unfortunately. Hopefully at some point after it's initially implemented, it can be reworked in a more optimal way for situations where no variables need to be captured and forwarded.
Title: Re: Considerations in the decision of adding a feature to the FPC compiler?
Post by: marcov on June 20, 2019, 05:19:37 pm
I'm pretty sure it works in generics at least in trunk. You can't currently use it in constants or within scanner conditionals though, which you can in Delphi. That said, I actually have a patch I've been working on that enables this, and also adds Delphi-compatible IsMangagedType and IsConstValue intrinsics that I might submit soon. (FWIW, adding those intrinsics was actually... very easy, as the compiler has internal methods that provide basically exactly the required functionality.)

My bad. I indeed meant ismanagedtype and not gettypekind and specialization time elimination of other branches
Title: Re: Considerations in the decision of adding a feature to the FPC compiler?
Post by: PascalDragon on June 21, 2019, 10:07:42 am
I'm pretty sure it works in generics at least in trunk. You can't currently use it in constants or within scanner conditionals though, which you can in Delphi. That said, I actually have a patch I've been working on that enables this, and also adds Delphi-compatible IsMangagedType and IsConstValue intrinsics that I might submit soon. (FWIW, adding those intrinsics was actually... very easy, as the compiler has internal methods that provide basically exactly the required functionality.)

My bad. I indeed meant ismanagedtype and not gettypekind and specialization time elimination of other branches
The specialization time elimination is already done by the compiler if it can do it (so once IsManagedType and IsConstValue are supported that should be immediately picked up as well; GetTypeKind already does that correctly, cause I explicitely tested that):
Code: Pascal  [Select]
  1. generic function GenSwapEndian<T>(aValue: T): T; inline;
  2. begin
  3.   case SizeOf(aValue) of
  4.     1:
  5.       Result := aValue;
  6.     2:
  7.       Result := SwapEndian(UInt16(aValue));
  8.     4:
  9.       Result := SwapEndian(UInt32(aValue));
  10.     8:
  11.       Result := SwapEndian(UInt64(aValue));
  12.   end;
  13. end;
  14.  
  15. procedure Test;
  16. var
  17.   u8: UInt8;
  18.   u16: UInt16;
  19.   u32: UInt32;
  20.   u64: UInt64;
  21. begin
  22.   specialize GenSwapEndian<UInt8>(u8);
  23.   specialize GenSwapEndian<UInt16>(u16);
  24.   specialize GenSwapEndian<UInt32>(u32);
  25.   specialize GenSwapEndian<UInt64>(u64);
  26. end;
  27.  
The assembler code of Test then looks like this (unlike Delphi we also inline correctly  :-[ ):
Code: [Select]
.section .text.n_p$thelloworld_$$_test,"x"
.balign 16,0x90
.globl P$THELLOWORLD_$$_TEST
P$THELLOWORLD_$$_TEST:
# [42] begin
pushl %ebp
movl %esp,%ebp
leal -24(%esp),%esp
# Var u8 located at ebp-4, size=OS_8
# Var u16 located at ebp-8, size=OS_16
# Var u32 located at ebp-12, size=OS_32
# Var u64 located at ebp-24, size=OS_64
# [43] specialize GenSwapEndian<UInt8>(u8);
movb -4(%ebp),%al
# [44] specialize GenSwapEndian<UInt16>(u16);
movzwl -8(%ebp),%eax
shrl $8,%eax
movzwl -8(%ebp),%edx
shll $8,%edx
orl %edx,%eax
# [45] specialize GenSwapEndian<UInt32>(u32);
movl -12(%ebp),%eax
shll $8,%eax
andl $-16711936,%eax
movl -12(%ebp),%edx
shrl $8,%edx
andl $16711935,%edx
orl %edx,%eax
roll $16,%eax
# [46] specialize GenSwapEndian<UInt64>(u64);
pushl -20(%ebp)
pushl -24(%ebp)
call SYSTEM_$$_SWAPENDIAN$QWORD$$QWORD
# [47] end;
movl %ebp,%esp
popl %ebp
ret