Recent

Author Topic: Boolean32  (Read 1649 times)

Jonas Maebe

  • Hero Member
  • *****
  • Posts: 640
Re: Boolean32
« Reply #15 on: March 23, 2019, 07:06:03 pm »
It says that only the array elements defined by its index-type exist. So in an array[0..0], only element 0 exist. Addressing a non-existent element in an array is an error.
In that case, there are more bugs in FPC as the code snippet below demonstrates.
You are correct that FPC does not generate compile-time errors in all cases when it detects undefined behaviour. I explained why in the other thread: compatibility with older Delphi-versions. This will probably be changed into an error at some point, at the very least for the delphiunicode mode (which corresponds to Delphi 2009+) and for the FPC/ObjFPC modes.

Quote
You can't have it both ways.  If indexing out of range is an error then FPC should not compile the statements in the code snippet above.  If it compiles them then it should produce code that is semantically correct, in this case, address(array) + (n * elementsize).
As explained by the Pascal standard quoted above, that is not the semantics of the array indexing operation in Pascal. It's the semantics of array indexing in C/C++.

Quote
when it decided to unceremoniously mutilate a signed variable into an unsigned one, it did so silently, no warning
Why do you keep repeating this? I must have answered with the explanation about how type checking and type conversions work, the fact that your programs would get hundreds of warnings if we did warn in such scenarios, the comparison with Ada, and the concept of range checking, about five times by now. The program will not be silently go wrong if you enable range checking. That is the whole and sole purpose of range checking: emit errors at run time for cases that cannot be checked at compile time.

Quote
Since you justify _forcing_ variables to be in the declared range of an array, how do you justify that FPC did _not_ turn the indexing numerals 5 and -5 into zero ?  or is mutilation a "feature" reserved for variables only ? 
Indexing an array out of its bounds is an error. The result of erroneous code is undefined. Turning such indices into zero would be therefore be "correct". Not turning them into zero is equally correct (that's why Delphi's behaviour is also correct). It's the difference between undefined (anything can happen at any time) and implementation-defined (the compiler can choose what happens, but it needs to be consistent). Undefined behaviour is unpredictable by its very nature.

That said, I fully agree consistent behaviour trumps inconsistent behaviour any day, regardless of what the standard allows. It's inherent to trying to be compatible to undefined behaviour of another compiler (which we tried in the past, hence the allowed out-of-range constants), and why it's such a bad idea (people will expect that you'll stay compatible to it in all possible forms and future versions, which is perfectly understandable but equally untenable).
« Last Edit: March 23, 2019, 11:30:32 pm by Jonas Maebe »

440bx

  • Hero Member
  • *****
  • Posts: 958
Re: Boolean32
« Reply #16 on: March 23, 2019, 10:40:07 pm »
You are correct that FPC does not generate compile-time errors in all cases when it detects undefined behaviour. I explained why in the other thread: compatibility with older Delphi-versions. This will probably be changed into an error at some point, at the very least for the delphiunicode mode (which corresponds to Delphi 2009+) and for the FPC/ObjFPC modes.
You can't quote standards then wrap them in a sheet of Delphi compatibility to justify bugs.  Obviously no standard can be used to redefine proven mathematical theory and justify the totally erroneous decision that zero (0) is a positive-only number.

In the particular case we are talking about the only appropriate error message the compiler could emit would be something along the lines of "Error: per creative interpretation of a Pascal standard, the compiler is going to put your 32bit signed variable into a 64bit register without extending the sign because zero (0) is a member of Pascal unsigned types."

At the very least, emphasis on least, FPC should not compile a statement indexed by a constant that is out of range.  That, is a bug and it cannot be hidden under the Delphi compatibility rug because Delphi doesn't compile it and it obviously violates the Pascal standard too.

As explained by the Pascal standard quoted above, that is not the semantics of the array indexing operation in Pascal. It's the semantics of array indexing in C/C++.
The standards passage you quoted does not say anything about how arrays and their elements are arranged in memory and how the address of said elements are determined.  Its main point is the determination of the number of elements in the array and their applicable range.  I will note too, that it doesn't say anything at all about the type of the range, which you have been using to justify that bug in FPC (or more accurately and worse, claim it is not a bug.)

Why do you keep repeating this? I must have answered with the explanation about how type checking and type conversions work, the fact that your programs would get hundreds of warnings if we did warn in such scenarios, the comparison with Ada, and the concept of range checking, about five times by now. The program will not be silently go wrong if you enable range checking.
I keep repeating it because you are grossly misusing range and type checking to justify FPC's incorrect code generation.  An array whose range consists of constants cannot always be assigned a type unambiguously and, when the range is 0..0 then no type can be assigned to it.  Assigning a type to it, is mathematically incorrect, forcing that incorrect type onto a variable that _does_ have a declared type is beyond incorrect, it's dismal.
 
That is the whole and sole purpose of range checking: emit errors at run time for cases that cannot be checked at compile time.
The compiler didn't do any range checking (it wasn't asked to do any either.) but, it turned a 32bit signed integer into a 64bit unsigned integer.  No definition of range checking covers that kind of behavior.

If the compiler had generated code and compared the value (after properly sign extending the 32bit value into the 64bit register) against zero (0), that would be range checking.  Mutilating variables is not range checking.

Indexing an array out of its bounds is an error.
That almost sounds reasonable but, then the existence of open arrays is an error in itself.  Those array don't even have a declared range much less a type associated with it.  To make matters even worse, their number of elements can vary at runtime.  That definitely violates the standards passage you quoted.

And if it is an error then FPC should not compile a reference to an array with a declared range that is using a constant that is out of the range, yet it does (and as stated before, Delphi doesn't.)

The result of erroneous code is undefined. Turning such indices into zero would be therefore be "correct". Not turning them into zero is equally correct (that's why Delphi's behaviour is also correct). It's the difference between undefined (anything can happen at any time) and implementation-defined (the compiler can choose what happens, but it needs to be consistent). Undefined behaviour is unpredictable by its very nature.
You cannot, as you are attempting to, claim that AnArray[SomeIndex] is erroneous code.  If it were "erroneous" as you pretend it is then, indexing open arrays is erroneous (by the definition you are attempting to slide, not mine.)  With a "normal" array, it simply may or may not be correct.  The compiler's job is to compute the address referenced _correctly_, emphasis on _correctly_ which FPC fails to do.   And it fails to do that, not because it didn't generate the correct code to calculate the address but, because it mutilated the indexing variable.

It's inherent to trying to be compatible to undefined behaviour of another compiler (which we tried in the past, hence the allowed out-of-range constants), and why it's such a bad idea (people will expect that you'll stay compatible to it in all possible forms and future versions, which is perfectly understandable but equally untenable).
You can't hide that bug behind the incorrect claim of "undefined behavior".  If it were "undefined" as you pretend it is, why do most compilers not only get it right but, do it the same way.   

FPC had a simple task, take the address of the array add to that (index * elementsize) and it would have gotten it right had it not "decided" to turn a signed type into an unsigned type thereby generating an address that was not in the range of the array's allocated memory.   

When a compiler generates the wrong address for an array element, that is a bug.  That's what FPC does in that case and there is nothing "erroneous" in the statement "AnArray[AnIndex]", that is, until FPC decides to chop the index's sign because the index range is "positive" (great, got to inform mathematicians around the world that zero is positive and correct "a few" books - no doubt Carl Friedrich Gauss would have been impressed.)

All that said, it's as unfortunate as it is evident that the bug is here to stay.

Let's agree to disagree.
« Last Edit: March 23, 2019, 11:12:11 pm by 440bx »
using FPC v3.0.4 and Lazarus 1.8.2 on Windows 7 64bit.

ASBzone

  • Full Member
  • ***
  • Posts: 226
  • Automation leads to relaxation...
    • BrainWaveCC Utilities
Re: Boolean32
« Reply #17 on: March 24, 2019, 09:01:09 pm »
It says that only the array elements defined by its index-type exist. So in an array[0..0], only element 0 exist. Addressing a non-existent element in an array is an error.
In that case, there are more bugs in FPC as the code snippet below demonstrates .....

Doesn't just simply fall into the category of undefined behavior?  As in: cannot be relied upon in any specific way, because the output is not defined?
-ASB: https://www.BrainWaveCC.com

Lazarus v2.0.3 r61485 / FPC v3.2.0-beta-r42306 (via FpcUpDeluxe) -- Windows 64-bit install w/32-bit cross-compile
Primary System: Windows 10 Pro x64, Version 1903 (Build 18362.116)
Other Systems: Windows 10 Pro x64, Version 1809 or greater

ASBzone

  • Full Member
  • ***
  • Posts: 226
  • Automation leads to relaxation...
    • BrainWaveCC Utilities
Re: Boolean32
« Reply #18 on: March 24, 2019, 09:23:58 pm »
Based on your last statement compiling case 1 and case 2 are errors yet, FPC compiles them (and it should not), it does emit a warning, that's nice but that doesn't justify compiling those erroneous statements.
...
It seems FPC is rather choosy and unpredictable when it comes to complying with Pascal standards.

With range checking on, FPC does not compile that.
-ASB: https://www.BrainWaveCC.com

Lazarus v2.0.3 r61485 / FPC v3.2.0-beta-r42306 (via FpcUpDeluxe) -- Windows 64-bit install w/32-bit cross-compile
Primary System: Windows 10 Pro x64, Version 1903 (Build 18362.116)
Other Systems: Windows 10 Pro x64, Version 1809 or greater

440bx

  • Hero Member
  • *****
  • Posts: 958
Re: Boolean32
« Reply #19 on: March 24, 2019, 11:52:16 pm »
With range checking on, FPC does not compile that.
That demonstrates that FPC is mixing apples and oranges.  Range checking isn't meant to change the compile time data type of a variable.  It's purpose is to tell the compiler to generate code to verify that a _variable_ is in the specified range at _run time_ not compile time (since it obviously cannot do it at compile time when a variable is used).  Pascal's strong type checking is neither associated nor dependent in any way on any runtime feature, such as range checking.

In a definition such as:
Code: Pascal  [Select]
  1. type
  2.   TRange = 0..0;
and in any range definition in Pascal, the lower and upper bounds are known and they limit the acceptable values of the underlying type, which the compiler is forced to _assume_. In this particular example, it is forced to assume the underlying type is: integer.

now, in a statement such as:
Code: Pascal  [Select]
  1. AnArray[-5]
that's, first and foremost, a data type violation.  No correctly implemented Pascal compiler would compile that.  Range checking is not necessary for the compiler to determine there is a data type violation there. That is as incorrect as
Code: Pascal  [Select]
  1. AnArray["somestring"]

Now when using a variable instead of a constant, in a statement such as:
Code: Pascal  [Select]
  1. var
  2.   AnIndex : integer;
  3.   ..
  4.   ..
  5.   AnArray[AnIndex]
the type of AnIndex is assignment compatible with the compiler _assumed_ type of the range, therefore, the statement is perfectly valid.  Of course, when using a variable, the compiler does not have enough information at compile time to enforce the range boundaries.  That's when range checking comes into play, when enabled the compiler can generate code to verify that the variable, in addition to being assignment compatible - which it checked at compile time against the _assumed_ underlying type of the range - holds a value that is in the range.

The crucial thing to understand is that, in no way does that change the code that is generated.   The compiler generates the same code to calculate what address the expression refers to and, when range checking is enabled,  it also generates code to enforce the range boundaries.  The claim that the code is erroneous when using a variable is false but it is true when using a constant that is not in the range yet, FPC compiles it, that by itself is a compiler bug.

In that specific case, FPC compiles code which is erroneous (AnArray[-5]) and generates erroneous code for a valid Pascal statement (AnArray[AnIndex]).

Doesn't just simply fall into the category of undefined behavior?  As in: cannot be relied upon in any specific way, because the output is not defined?
The claim that there is something undefined is yet another instance of mixing apples and oranges.  The result, when the variable is not in the range is undefined but, there is nothing undefined as far as code generation is concerned.

An example will make this clearer
Code: Pascal  [Select]
  1. var
  2.   AShortInt : shortint;
  3.   AnInteger, AnotherInteger : integer;
  4.  
  5. begin
  6.   AnInteger      := 240;
  7.   AnotherInteger := 240;
  8.   AShortInt := AnInteger + AnotherInteger;
  9.  
  10.   etc
  11.  
The statement
Code: Pascal  [Select]
  1.   AShortInt := AnInteger + AnotherInteger;
isn't wrong/erroneous or invalid in any way.  The compiler generates code to add the two variables AnInteger + AnotherInteger and stores the result in AShortInt. The one thing that is legitimately undefined is what the result is going to be since the sum causes an overflow but, ensuring there is no overflow is the programmer's responsibility, not the compiler's and, much less a justification for the compiler to get "creative" and truncate the values stored in AnInteger and AnotherInteger to force the result to fit in a shortint. 

When overflow checking is off, the result will be whatever bit pattern is generated by the sum truncated to a shortint but, the compiler does not generate different code when overflow checking is on.  It simply adds instructions to verify that the result is in the range of the variable and, it does so only if the programmer requests it. 

That's exactly what it should do - and rather unfortunately isn't doing - when indexing an array with a variable.

When variables are used to access memory, it is the programmer's responsibility to ensure that those references point to the location they are supposed to reference.  It is most definitely not the compiler's prerogative to mutilate variables to force-fit references in a range it incorrectly determined.

Unfortunately, nothing short of a miracle is going to cause that bug to be corrected.  Maybe the way to have it fixed is go to church or Lourdes.








« Last Edit: March 25, 2019, 12:17:33 am by 440bx »
using FPC v3.0.4 and Lazarus 1.8.2 on Windows 7 64bit.