Recent

Author Topic: likely bug in variable definitions  (Read 1831 times)

y.ivanov

  • Hero Member
  • *****
  • Posts: 555
Re: likely bug in variable definitions
« Reply #15 on: October 01, 2022, 06:38:34 pm »
Code: Pascal  [Select][+][-]
  1. var
  2.   a: 1 * dword; // ok
  3.   b: 1 * 0 + dword; // ok
  4.   c: 1 * 1 * 0 + dword; // ok
  5.   d: 1 * 1 * ( 1 - 1 ) + dword; // ok
  6.   e: 1 * 1 * ( 16 - 2 * 8 ) + dword; // ok
  7.   f: dword + #0; // Operator is not overloaded: "LongWord" + "Char"
  8. begin
  9.   writeln(a,b,c,d,e,f);
  10. end.

It seems that an expression non-terminal can follow the : into the variable decl. despite the syntax diagrams
https://www.freepascal.org/docs-html/current/ref/refse22.html

As long it evaluates to zero and includes a suitable type identifier - it is OK  :o
"I'm sorry Dave, I'm afraid I can't do that."
—HAL 9000

Bogen85

  • Full Member
  • ***
  • Posts: 146
Re: likely bug in variable definitions
« Reply #16 on: October 01, 2022, 06:47:31 pm »
For me this would not compile, even the ones marked as ok by @y.ivanov

Code: Pascal  [Select][+][-]
  1. program DefinitionBug2;
  2.  
  3. var
  4.   a: 1 * dword;
  5.   b: 1 * 0 + dword;
  6.   c: 1 * 1 * 0 + dword;
  7.   d: 1 * 1 * ( 1 - 1 ) + dword;
  8.   e: 1 * 1 * ( 16 - 2 * 8 ) + dword;
  9.   f: dword + #0;
  10. begin
  11.   writeln(a,b,c,d,e,f);
  12. end.

Code: Text  [Select][+][-]
  1. $ fpc ./lazforum/60776/DefinitionBug2.pas
  2. Free Pascal Compiler version 3.3.1 [2022/10/01] for x86_64
  3. Copyright (c) 1993-2022 by Florian Klaempfl and others
  4. Target OS: Linux for x86-64
  5. Compiling ./lazforum/60776/DefinitionBug2.pas
  6. DefinitionBug2.pas(4,15) Error: Error in type definition
  7. DefinitionBug2.pas(5,19) Error: Error in type definition
  8. DefinitionBug2.pas(6,23) Error: Error in type definition
  9. DefinitionBug2.pas(7,31) Error: Error in type definition
  10. DefinitionBug2.pas(8,36) Error: Error in type definition
  11. DefinitionBug2.pas(9,12) Error: Operator is not overloaded: "LongWord" + "Char"
  12. DefinitionBug2.pas(9,16) Error: Error in type definition
  13. DefinitionBug2.pas(11,12) Error: Can't read or write variables of this type
  14. DefinitionBug2.pas(13) Fatal: There were 8 errors compiling module, stopping
  15. Fatal: Compilation aborted
  16. Error: /home/shared-development/fpc_usr/lib/fpc/3.3.1/ppcx64 returned an error exitcode

Arioch

  • Sr. Member
  • ****
  • Posts: 414
Re: likely bug in variable definitions
« Reply #17 on: October 01, 2022, 06:48:24 pm »
As long it evaluates to zero

hmmm...

not quite...

Code: Pascal  [Select][+][-]
  1.  var a: int64(nil) + qword;  // project1.lpr(3,26) Error: Error in type definition

 >:D

Code: Pascal  [Select][+][-]
  1. var a: qword + ord(0);
this works though

 O:-)

Code: Pascal  [Select][+][-]
  1. var a: qword + qword(nil);
and this too!

 :-*

Code: Pascal  [Select][+][-]
  1. var a: qword(nil) + uint64;
and this

 :-[

but this still doesn't
Code: Pascal  [Select][+][-]
  1. var a: qword(0);

 >:(

дали детям игрушку :-D

FPC win64 of git Fixes_3_2 - probably 3.2.3 or something
« Last Edit: October 01, 2022, 06:50:48 pm by Arioch »

y.ivanov

  • Hero Member
  • *****
  • Posts: 555
Re: likely bug in variable definitions
« Reply #18 on: October 01, 2022, 07:29:40 pm »
@Bogen85
Mine is FPC 3.2.2

But:
Code: Text  [Select][+][-]
  1. DefinitionBug2.pas(9,12) Error: Operator is not overloaded: "LongWord" + "Char"
still means there is an expression evaluator lurking.

дали детям игрушку :-D
Exactly.

And I very much wonder what an expression has to do where a type non-terminal was expected.  :o
"I'm sorry Dave, I'm afraid I can't do that."
—HAL 9000

Arioch

  • Sr. Member
  • ****
  • Posts: 414
Re: likely bug in variable definitions
« Reply #19 on: October 01, 2022, 07:57:12 pm »
And I very much wonder what an expression has to do where a type non-terminal was expected.  :o

if you really do it "very much" - there are soruces.

I do not, though. Not THAT much.

Guess that is rooted somehow in the need to transit "initialized variables" to AST

The token then get some attibutes, like type, value, sizeof/address, mutability (const/var)

Then later some sanity check is applied to make sure different flags and attributes are consistent, like typed const should have non-zero sezeof and address, and untyped one should lack them.

Type expressions and value expressions are intertwined anyway (because of array <set expression> of type) just tend to use different operators (^ in types, * in values).
Also, why you evaluate value-expressions it might include type-casts, especially implicit ones (that very 1 / 2 == 0 in C but = 0.5 in Pascal )
So, perhaps, unified parser emitting both evaluation tree and adjusted data type was a reasonable implementation.

Having both type ident = type-expr and var ident : type-expr FPC had to run that dual-evaluator after both colon and quality.
As initialized vars were introduced - the parser had to be permitted to create both type and value in var context.
And that is how it ended.

Just speculating.

Also, i remember colon is also used in some more places of the language: goto labelsand variant records. I wonder if the latter have some pinnatas too :-)
« Last Edit: October 01, 2022, 07:59:00 pm by Arioch »

y.ivanov

  • Hero Member
  • *****
  • Posts: 555
Re: likely bug in variable definitions
« Reply #20 on: October 01, 2022, 10:15:17 pm »
And I very much wonder what an expression has to do where a type non-terminal was expected.  :o

if you really do it "very much" - there are soruces.

I do not, though. Not THAT much.
I won't dig in the compiler sources. At least not for that. 

Known is that the Pascal is (at least was) a LL(1) grammar, the compiler implements a recursive descent parser and that is the foundation of its lightning speed. BTW it is still advertised as such (FPC).

It seems that has changed, IMO the RDP is not capable of such a flippant behavior. This is a direct violation of the syntax as it was defined.

« Last Edit: October 02, 2022, 01:14:41 am by y.ivanov »
"I'm sorry Dave, I'm afraid I can't do that."
—HAL 9000

PascalDragon

  • Hero Member
  • *****
  • Posts: 4756
  • Compiler Developer
Re: likely bug in variable definitions
« Reply #21 on: October 03, 2022, 02:42:04 pm »
Known is that the Pascal is (at least was) a LL(1) grammar, the compiler implements a recursive descent parser and that is the foundation of its lightning speed. BTW it is still advertised as such (FPC).

It seems that has changed, IMO the RDP is not capable of such a flippant behavior. This is a direct violation of the syntax as it was defined.

FPC's parser is not derived from a grammar, but it's a hand written one. In this case the “problem” is that the type expression for e.g. a variable or a field is done using the same function that parses a normal expression just set to a different mode (namely “parse type” instead of “parse normal body”) - this is done because the two share quite some functionality - , but for some reason it doesn't correctly filter out expressions that shouldn't be part of a type expression.

y.ivanov

  • Hero Member
  • *****
  • Posts: 555
Re: likely bug in variable definitions
« Reply #22 on: October 03, 2022, 05:23:06 pm »
Known is that the Pascal is (at least was) a LL(1) grammar, the compiler implements a recursive descent parser and that is the foundation of its lightning speed. BTW it is still advertised as such (FPC).

It seems that has changed, IMO the RDP is not capable of such a flippant behavior. This is a direct violation of the syntax as it was defined.

FPC's parser is not derived from a grammar, but it's a hand written one.
I wouldn't agree with that. Every formal language have a formal grammar. Syntax diagrams are just other representation of the metalanguage such as BNF. So the parser should be implemented strictly according to (deriving from) that grammar, even it was hand written.

In this case the “problem” is that the type expression for e.g. a variable or a field is done using the same function that parses a normal expression just set to a different mode (namely “parse type” instead of “parse normal body”) - this is done because the two share quite some functionality - , but for some reason it doesn't correctly filter out expressions that shouldn't be part of a type expression.
As you confirm, and also as Arioch speculated, the parser parts for <type> and <expression> somehow intertwined here. And I wonder how one can find enough similarities between the two and put them in one function.

I'm pasting some Pascal BNFs I googled. Of course they're not comprehensive, but are correct as a bedding for FPC, though. How it is possible to merge them?

<expression> BNF:
Code: Text  [Select][+][-]
  1. <expression> ::= <simple expression> | <simple expression> <relational operator> <simple expression>
  2. <relational operator> ::= = | <> | < | <= | >= | > | in
  3. <simple expression> ::= <term> | <sign> <term>| <simple expression> <adding operator> <term>
  4. <adding operator> ::= + | - | or
  5. <term> ::= <factor> | <term> <multiplying operator> <factor>
  6. <multiplying operator> ::= * | / | div | mod | and
  7. <factor> ::= <variable> | <unsigned constant> | ( <expression> ) | <function designator> | <set> | not <factor>
  8. ...

<type> BNF:
Code: Text  [Select][+][-]
  1. <type> ::= <simple type> | <structured type> | <pointer type>
  2. <simple type> ::= <scalar type> | <subrange type> | <type identifier>
  3. <scalar type> ::= (<identifier> {,<identifier>})
  4. <subrange type> ::= <constant> .. <constant>
  5. <type identifier> ::= <identifier>
  6. <structured type> ::= <array type> | <record type> | <set type> | <file type>
  7. <array type> ::= array [<index type>{,<index type>}] of <component type>
  8. <index type> ::= <simple type>
  9. <component type> ::= <type>
  10. <record type> ::= record <field list> end
  11. ...

P.S.
Despite the sub-range was defined in https://www.freepascal.org/docs-html/current/ref/refsu4.html#x26-290003.1.1 as:
Code: Text  [Select][+][-]
  1. <subrange type> ::= <constant> .. <constant>
it is more like:
Code: Text  [Select][+][-]
  1. <subrange type> ::= <constant expression> .. <constant expression>
but that introduces ambiguity with the enumeration type (starts with '(') and the weirdness of:
Code: Pascal  [Select][+][-]
  1. const
  2.   c1 = 1;
  3.   c2 = 2;
  4. type
  5.   t1 = 1 + c2 .. 5; // ok, 3..5
  6.   t2 = c1 + c2 .. 5; // ok, 3..5, see the next line
  7.   t3 = (c1 + c2) .. 5; // Error: Duplicate identifier "c1"
  8.   t4 = 2 * (1 + c2) .. 7; // ok, 6..7, see the next line
  9.   t5 = (1 + c2) * 2 .. 7; // Syntax error, "identifier" expected but "ordinal const" found
  10.   t6 = c1 .. (1 + c2) * 2; // ok, 1..6
  11.   t7 = c1 .. (c1 + c2); // ok, 1..3
« Last Edit: October 04, 2022, 01:43:49 am by y.ivanov »
"I'm sorry Dave, I'm afraid I can't do that."
—HAL 9000

MarkMLl

  • Hero Member
  • *****
  • Posts: 5571
Re: likely bug in variable definitions
« Reply #23 on: October 03, 2022, 08:58:54 pm »
I wouldn't agree with that. Every formal language have a formal grammar. Syntax diagrams are just other representation of the metalanguage such as BNF. So the parser should be implemented strictly according to (deriving from) that grammar, even it was hand written.

I believe that what he means is that it doesn't use a metalanguage such as a YACC description.

However that does leave me with a question: accepting for the moment that Pascal is LL(1), my understanding is that Wirth's earliest compilers- which presumably includes the first Pascal implementations- were recursive /ascent/ rather than recursive /descent/. Would any CS guru like to comment on whether the two approaches are provably equivalent?

MarkMLl
MT+86 & Turbo Pascal v1 on CCP/M-86, multitasking with LAN & graphics in 128Kb.
Pet hate: people who boast about the size and sophistication of their computer.
GitHub repositories: https://github.com/MarkMLl?tab=repositories

PascalDragon

  • Hero Member
  • *****
  • Posts: 4756
  • Compiler Developer
Re: likely bug in variable definitions
« Reply #24 on: October 04, 2022, 01:55:09 pm »
Known is that the Pascal is (at least was) a LL(1) grammar, the compiler implements a recursive descent parser and that is the foundation of its lightning speed. BTW it is still advertised as such (FPC).

It seems that has changed, IMO the RDP is not capable of such a flippant behavior. This is a direct violation of the syntax as it was defined.

FPC's parser is not derived from a grammar, but it's a hand written one.
I wouldn't agree with that. Every formal language have a formal grammar. Syntax diagrams are just other representation of the metalanguage such as BNF. So the parser should be implemented strictly according to (deriving from) that grammar, even it was hand written.

There never was a formal grammar we began with. And the syntax diagrams that exist are derived from what the compiler supports (and even then some are faulty, because the one writing the documentation didn't find out everything that the compiler itself understands).

In this case the “problem” is that the type expression for e.g. a variable or a field is done using the same function that parses a normal expression just set to a different mode (namely “parse type” instead of “parse normal body”) - this is done because the two share quite some functionality - , but for some reason it doesn't correctly filter out expressions that shouldn't be part of a type expression.
As you confirm, and also as Arioch speculated, the parser parts for <type> and <expression> somehow intertwined here. And I wonder how one can find enough similarities between the two and put them in one function.

It's due to things like this (as you noticed further down as well):

Code: Pascal  [Select][+][-]
  1. program trange;
  2.  
  3. const
  4.   MyMin = 42;
  5.   MyMax = 128;
  6.  
  7. var
  8.   r: MyMin - 4 .. MyMax * SizeOf(Int16);
  9. begin
  10.   Writeln(Low(r), ' .. ', High(r));
  11. end.

P.S.
Despite the sub-range was defined in https://www.freepascal.org/docs-html/current/ref/refsu4.html#x26-290003.1.1 as:
Code: Text  [Select][+][-]
  1. <subrange type> ::= <constant> .. <constant>
it is more like:
Code: Text  [Select][+][-]
  1. <subrange type> ::= <constant expression> .. <constant expression>
but that introduces ambiguity with the enumeration type (starts with '(') and the weirdness of:
Code: Pascal  [Select][+][-]
  1. const
  2.   c1 = 1;
  3.   c2 = 2;
  4. type
  5.   t1 = 1 + c2 .. 5; // ok, 3..5
  6.   t2 = c1 + c2 .. 5; // ok, 3..5, see the next line
  7.   t3 = (c1 + c2) .. 5; // Error: Duplicate identifier "c1"
  8.   t4 = 2 * (1 + c2) .. 7; // ok, 6..7, see the next line
  9.   t5 = (1 + c2) * 2 .. 7; // Syntax error, "identifier" expected but "ordinal const" found
  10.   t6 = c1 .. (1 + c2) * 2; // ok, 1..6
  11.   t7 = c1 .. (c1 + c2); // ok, 1..3

These should probably be reported then...

MarkMLl

  • Hero Member
  • *****
  • Posts: 5571
Re: likely bug in variable definitions
« Reply #25 on: October 04, 2022, 02:53:19 pm »
There never was a formal grammar we began with. And the syntax diagrams that exist are derived from what the compiler supports (and even then some are faulty, because the one writing the documentation didn't find out everything that the compiler itself understands).

Speaking as somebody who's done his fair share of technical writing: I sympathise.

Even when working in a company with clear responsibilities, getting comprehensive and intelligible information out of designers and developers- and distilling the edge cases into something that passes as English- can be like wringing blood out of a stone.

MarkMLl
MT+86 & Turbo Pascal v1 on CCP/M-86, multitasking with LAN & graphics in 128Kb.
Pet hate: people who boast about the size and sophistication of their computer.
GitHub repositories: https://github.com/MarkMLl?tab=repositories

y.ivanov

  • Hero Member
  • *****
  • Posts: 555
Re: likely bug in variable definitions
« Reply #26 on: October 04, 2022, 03:11:20 pm »
*snip*
P.S.
Despite the sub-range was defined in https://www.freepascal.org/docs-html/current/ref/refsu4.html#x26-290003.1.1 as:
Code: Text  [Select][+][-]
  1. <subrange type> ::= <constant> .. <constant>
it is more like:
Code: Text  [Select][+][-]
  1. <subrange type> ::= <constant expression> .. <constant expression>
but that introduces ambiguity with the enumeration type (starts with '(') and the weirdness of:
Code: Pascal  [Select][+][-]
  1. const
  2.   c1 = 1;
  3.   c2 = 2;
  4. type
  5.   t1 = 1 + c2 .. 5; // ok, 3..5
  6.   t2 = c1 + c2 .. 5; // ok, 3..5, see the next line
  7.   t3 = (c1 + c2) .. 5; // Error: Duplicate identifier "c1"
  8.   t4 = 2 * (1 + c2) .. 7; // ok, 6..7, see the next line
  9.   t5 = (1 + c2) * 2 .. 7; // Syntax error, "identifier" expected but "ordinal const" found
  10.   t6 = c1 .. (1 + c2) * 2; // ok, 1..6
  11.   t7 = c1 .. (c1 + c2); // ok, 1..3

These should probably be reported then...
Not sure the RDP can handle it without back-tracking ... I have tried this on D7 - it fails in the similar way as FPC. Perhaps we can say they're compatible in the way they fail. ;)
At the other hand gpc can handle it, but it is yacc based, i.e. it is LALR(1) parser.
IMO such an issue is inevitable and the only thing that can be done about it is to correct the documentation, i.e. to put a '(' on a right place into the syntax diagram.
"I'm sorry Dave, I'm afraid I can't do that."
—HAL 9000

Thaddy

  • Hero Member
  • *****
  • Posts: 12200
Re: likely bug in variable definitions
« Reply #27 on: October 04, 2022, 05:26:34 pm »
@PascalDragon
There was a formal grammar published by Borland. (D5-D7) in BNR notation and that was not a refactoring or a back-port. It was the real deal. I did some work for Borland att.
It was only partially correct though, but neigh complete. And it served many purposes and still does. Sloppy engineering.
Reference: appendix A of the Delphi language guide for D7.
I am a bit bemused you are not aware of that? ;)


( At that time, though, most proper Elvises had left the building(s))
« Last Edit: October 04, 2022, 05:40:27 pm by Thaddy »
Manuals, manuals, manuals first.
You have incompetence and sheer incompetence.

MarkMLl

  • Hero Member
  • *****
  • Posts: 5571
Re: likely bug in variable definitions
« Reply #28 on: October 04, 2022, 05:45:13 pm »
@PascalDragon
There was a formal grammar published by Borland. (D5-D7) in BNR notation and that was not a refactoring or a back-port. It was the real deal. I did some work for Borland att.
It was only partially correct though, but neigh complete. And it served many purposes and still does. Sloppy engineering.
Reference: appendix A of the Delphi language guide for D7.
I am a bit bemused you are not aware of that? ;)

But there's a big difference between having a grammar that describes a language, and working from a grammar to build a compiler using e.g. YACC.

You said yourself: "partially correct, nigh complete". You can't build a compiler from that.

MarkMLl
MT+86 & Turbo Pascal v1 on CCP/M-86, multitasking with LAN & graphics in 128Kb.
Pet hate: people who boast about the size and sophistication of their computer.
GitHub repositories: https://github.com/MarkMLl?tab=repositories

marcov

  • Administrator
  • Hero Member
  • *
  • Posts: 10399
  • FPC developer.
Re: likely bug in variable definitions
« Reply #29 on: October 04, 2022, 05:50:04 pm »
@PascalDragon
There was a formal grammar published by Borland. (D5-D7) in BNR notation and that was not a refactoring or a back-port. It was the real deal. I did some work for Borland att.
It was only partially correct though, but neigh complete. And it served many purposes and still does. Sloppy engineering.

The grammar in the manual, that was for documentation purposes only, and was very simplistic and buggy. It was not really a language definition.



 

TinyPortal © 2005-2018