Lazarus

Free Pascal => General => Topic started by: jc99 on June 13, 2015, 12:00:16 am

Title: Out Parameter - Test program - funny behaviour
Post by: jc99 on June 13, 2015, 12:00:16 am
I have a program/problem:
see: http://bugs.freepascal.org/view.php?id=28279
Code: [Select]
program PrjTestParameter;

{$IFDEF MSWINDOWS}
{$APPTYPE CONSOLE}
{$ENDIF}

{ $Define WorkAround1}
{ $Define WorkAround2}
{ $Define WorkAround3}

Function Test({const}Line:String;out Part1:string;{$ifndef WorkAround2} out{$else}var{$endif} Part2:string):Boolean;{$ifdef WorkAround3} inline;{$endif}

begin
  result := length(Line)>1;
  Part1:= copy(Line,1,1);
  Part2:=copy(Line,2,length(Line)-1);
end;

var
  BaseLine: String;
  OneChar: string;
  Baseline2: string;

begin
  BaseLine := 'ABCDEFGHIJKLMNOP';
  while Test(Baseline,OneChar,Baseline) do
    begin
     writeln(Onechar);
{$ifdef WorkAround1}
     BaseLine2:=BAseline;
{$endif}
    end;
  readln;
end.

In my opinion this should work.
If it is wrong then Workaround1 (not 2, sorry my mistake) should not make it work.

Both workarounds are not satisfying, the second gives a hint that a variable isn't initialized. The first uses an unnecessary variable.

Edit:
In this form, it may be obvious that it is not working ...
but think of it that the procedure is in one unit and the call is in another unit. ... and it's a big project, lot's of files, ton's of code ...
Then think that the procedure started as
Code: [Select]
Function Test(Line:String;var Part1,Part2:string):Boolean;everything was fine.
The Story goes:
Then a new compiler came and tells you that some variables are not initialized. So you look at the code and say "Oh, yes, these paramaters are only written to so they should be out-Parameters". You change the var to out, compile the program, no error no warning, no hint.
Program running fine. 'A' as baseline works, 'AB' as baseline works.
But then after releasing the Prog somehow you have to do 'ABC' as baseline and BOOM, where is my B and C.

So my point is:
This should either be working, or there should be a Warning that it may be not working.

Edit:
Found Workaround3: declare the function inline
Title: Re: Out Parameter - Test program - funny behaviour
Post by: Basile B. on June 13, 2015, 03:46:36 am
The problem is that when you use the out parameter storage class, the value you pass is implicitly set to its default value (empty string). example:

Code: [Select]
program Project1;

var
  str: string;

procedure MagicReset(out value: string);
begin
end;

begin
  str := 'wxcvbn,;:!';
  MagicReset(str);
  writeln(str);
  readln;
end.   

But, in your example, when Baseline is magically reset, since ObjectPascal string are reference counted, this reference count is reset to 0. While actually it shouldn't. That's a bug.

read more here, Delphi has (had?) the same problem:
http://alex.ciobanu.org/?p=48
Title: Re: Out Parameter - Test program - funny behaviour
Post by: Martin_fr on June 13, 2015, 05:56:04 am
About your remark in the linked bug:
Code: [Select]
When debugging you will see, that at the end of the first call of test Line still has 'ABCD...', Part1 := 'A', Part2 := 'BCD..'Assuming that statement was checked with "const" in code. (const kind of makes the string a pchar, not guaranteeing any ref count)

The debugger does not check if the variable is valid. A string variable is a pointer, and a "const string" can point to the remains of a string. That is it can point to freed memory, that has not yet been overridden and therefore looks like the string you expect.

In case if there is no "const" then it depends on the refcount. (see below)

Actually there is a bug. However that is probably not what you experienced. Because this bug passes in an invalid/trashed string.
--------------

The reason you may (or may not) see the "correct" string in "Line" is the ref count of the variable "Baseline". In the first iteration of the loop, you have a constant (special ref count, afaik -1), that works different. In the 2nd iteration of the loop the refcount should be 1, and things may go wrong.
However any slight change of the code (or me mis-reading it) and the refcount is 2 or higher, and it behaves more like you want (at least it may, no guarantee here).

----------
Even if the issue I mentioned earlier is fixed, your code is still not good.
That is the value of "Line" still depends on the order of argument evaluation. And afaik in your code it will first be set to empty string.
(Well that is assumed the order of param eval is documented, and therefore guaranteed. IIRC it is, but not sure)

I agree your code looks good at first, and it takes a lot of detailed knowledge to find why it does not work.
As I said *IF* the order of param eval is documented (and in this case documented last param is evaled first (at least for pascal calling convention).
1) out param is evaluated => variable is set to empty string.
   **IF** refcount is 1, then it affects the next param, otherwise not.
2) Line is evaluated. This may be the empty string.

Title: Re: Out Parameter - Test program - funny behaviour
Post by: jc99 on June 13, 2015, 08:53:26 am
@Martin_fr
Quote
Assuming that statement was checked with "const" in code. (const kind of makes the string a pchar, not guaranteeing any ref count)
No it behaves like this in both ways (with and without "const")

@BBasile
Quote
read more here, Delphi has (had?) the same problem:
In Delphi (XE & XE2) it is different, the code does not work at all,
the parameters are reset right away. So you have no problem debugging it.

My point is:
Since the compiler gives so much Hints about not initialized Variables, it should give a Warning when the same parameter is passed to Procedure/Function twice in different manners.
The better solution would be to make the thing work.
1. always evaluate out Parameter last/first (same way)
2. increment the counter, when the parameter is used
or do the Initialization only when it's not initialized.

BUT:
Code: [Select]
  BaseLine := 'ABC';
  if test(BaseLine,Onechar,BaseLine) then
    writeln('1: ',Onechar);

  if test(BaseLine,Onechar,BaseLine) then
    writeln('2: ',Onechar);
(Code fails on second call) in this case it should be obvious to the compiler, that the parameter is used again, but it still fails.
Title: Re: Out Parameter - Test program - funny behaviour
Post by: jc99 on June 13, 2015, 09:01:17 am
Quote
Even if the issue I mentioned earlier is fixed, your code is still not good.
That is the value of "Line" still depends on the order of argument evaluation. And afaik in your code it will first be set to empty string.
(Well that is assumed the order of param eval is documented, and therefore guaranteed. IIRC it is, but not sure)
I switched the parameters in the test Function (out,out,const/direct),
and the program/compiler still behaves this way.
Title: Re: Out Parameter - Test program - funny behaviour
Post by: Basile B. on June 13, 2015, 09:08:11 am
My point is more that it shouldn't be allowed at all to use the same parameter as in and out. The fact is that IRL everyone would have written the function:

Code: [Select]
function(var Line: string; out current char): boolean;
It's bad luck you've been faced to this crazy corner case.
Though, interesting to analyze. The link i've put in my first answer EXACTLY describes the same thing.

Title: Re: Out Parameter - Test program - funny behaviour
Post by: jc99 on June 13, 2015, 09:25:36 am
My point is more that it shouldn't be allowed at all to use the same parameter as in and out. The fact is that IRL everyone would have written the function:

Code: [Select]
function(var Line: string; out current char): boolean;
It's bad luck you've been faced to this crazy corner case.
Though, interesting to analyze. The link i've put in my first answer EXACTLY describes the same thing.
1. I try to make a different point: If it is allowed, it should work. Or if it isn't, there should be a warning.
2. Your answer DOES describe the same thing. What I tried to point out that dpc and fpc handle the code differently.
in Delphi the program fails on first call. In FPC the first call seems OK and the program fails on second call.
Title: Re: Out Parameter - Test program - funny behaviour
Post by: jc99 on June 14, 2015, 01:31:47 pm
I know it's not good to reply to your own diskusion, but there is new info:
Subject closed:
Quote from: Jonas_Maebe link=http://bugs.freepascal.org/view.php?id=28279
Other people have already answered in the forum. See also the discussion on the fpc mailing list: http://lists.freepascal.org/pipermail/fpc-devel/2015-June/thread.html ("ref count issue with out param"). Your remark regarding the warning (or rather, why a warning that always works is impossible to add) is addressed in http://lists.freepascal.org/pipermail/fpc-devel/2015-June/035706.html
I am not so happy that the issue is marked resolved.
Because (i hope) there is an ongoing discusion about this.
Nobody seems to care about the point I try to make:
You start with a
Code: [Select]
function foo(aPar1:TSomething;var aPar2:TSomething):TSomethingElse;
You test is, release it.--> Valid.

SomeOne (else) uses your function.
Code: [Select]
while foo(s1,s1) do [...]
He tests his code, still valid.

Years later the compiler has a new feature (Hints and out-params)
You came across your old function
and since you only write to aPar2 (and you want get rid of the annoying hint that the variable is not initialized.
you change the var to out.
by this you make someone else's code WRONG.
I think that some deserves a warning that his formerly valid code may have problems now.

I am fully aware that you can not warn everybody with everything. But a warning when a parameter is obviously handed twice (or more) to a routine with one of which is an "out" would be a start.

[...]
Found Workaround3: declare the function inline
And also a function should not change it's (obvious) behavior when declared inline (or not). If you are a good programmer then you normally put your "inline" in a compiler
-def like
Code: [Select]
function foo([...]):TSomething; {$IFDEF SUPPORTS_INLINE}inline;{$ENDIF}

The next point is:
You use this function:
Code: [Select]
var p1,p2:string;
[...]
  p1:='ABCDE';
  while foo(p1,p2) do
     p1:=p2;
[...]
That is obviously VALID code.
then someone ads a new fancy optimization to the compiler
Namely: when a variable is only used to fill a parameter it can be replaced with that variable, to get rid of an obvious unnecessary copy.
The formerly valid code starts to behave erratic. which is a very unpleasant thing, if you try to debug a large project.

At some point
Code: [Select]
i := i + 1;
is not valid anymore because you use variable i as an input and also as a target of that code.
(just kidding)
Title: Re: Out Parameter - Test program - funny behaviour
Post by: Thaddy on June 15, 2015, 05:52:33 pm
Since I noticed from the discussions that only some DXE's were tested for reference:
D7 has the same  behavior  The out parameter is initialized to nil. Accordingly the output is nothing.
Only with the var parameter, D7 outputs the separate characters.
If you use the same variable as input for both parameters it gets reset to nil.
So from a delphi compatibility  point of view, you may hope for documentation of this behavior but it won't ever be "fixed" in any other way.
Title: Re: Out Parameter - Test program - funny behaviour
Post by: jc99 on June 16, 2015, 04:13:28 am
Since I noticed from the discussions that only some DXE's were tested for reference:
D7 has the same  behavior  The out parameter is initialized to nil. Accordingly the output is nothing.
Only with the var parameter, D7 outputs the separate characters.
DXE2 was only the newest Compiler availible for me, If i wanted I could Do a synopsis from tp3 over bp7  via d2, d4 to dxe2 where i stoped buying New compilers. My Code also developed through the different versions. BTW Bp7 was my favorite for a long time.
But as You See above there was a link to a Delphi Blog that they try to fix that in newer versions.
So i only tried the newer versions.
The reason i prefered Pascal over C, C++, C#, Java (and all the other New shit) was the structure and encapulation. And Not have to think about, oh when i Do this then the reference count is Not Set correctly ... To me that is against the Basic philosophy of Pascal.


Title: Re: Out Parameter - Test program - funny behaviour
Post by: Thaddy on June 16, 2015, 08:19:12 am
Mm, but given that you access the same variable, the same memory location if you wish, and pass this location in two (three) contradictory ways in your routine, at best your code should never compile. It should probably generate an error.
Furthermore, the order in which the parameters are evaluated by a Pascal compiler is probably implementation specific in the case of out. Any differences between FPC and Delphi can be explained in this way: Given that an out parameter is defined as only valid on exit, compiler logic can assume it can optimize certain code by changing the order in which parameters are used.
The value of the out parameter is by definition undefined when passed in, you really can not expect anything else. The compiler can only turn this into proper code by initializing it to a default, i.e. nil, in the proccedure body. This behavior is documented for Delphi and FPC alike. This means that if the same memory location is referenced it also affects the by value or const parameter in a contradictory way. Your code should never work. The compiler should never be compromized by handling a refcount because of an out parameter, but the programmer should probably be warned of his error. In my opinion this is, as the majority in the discussion opiniate or state, user error.

The valid point  is not that your code is correct, it isn't, but that this construction should be caught by the compiler as an error, much in the same way as trying to change a parameter passed in as const generates a compile time error.

Not to put too fine a point on it, but the latter is effectively exactly what you are doing in part of your example. The difference is that this can only be caught at runtime.
Title: Re: Out Parameter - Test program - funny behaviour
Post by: jc99 on June 17, 2015, 07:04:23 am
Then I have to remind you of the topic. It is NOT about fooling the Compiler with all possible variants and waiting for the crash. It is about out parameter (replacing the var-parameter). Like const it is a way to tell the compiler what you want to do with it.
all three are by reference and should be carefully used by the programmer.
--
Now I have an atomic function/procedure :
1 in value
3 out value. (1 fct-result, 2 var or out)
What is than the right way to do it ?
--
BTW.
the whole thing is working, when the value after the call is READ and put to another variable.
This could be the lever to a solution if someone is willing.
Wouldn't it be a good thing, to get this working ?
Why is the out parameter initialized in the first place ? When you try a read-access you defintly get a warning. For a good programmer that should be enough to find his error.

Title: Re: Out Parameter - Test program - funny behaviour
Post by: Thaddy on June 17, 2015, 07:25:25 am
The variable is initialized because it is on the stack. The stack is dirty by definition. To get predictable, sensible results the variable needs to be predictable, i.e. initialized to a known value.
Same goes if the variable is optimized to be in a register. The register is dirty by default, to get predictable results.... etc etc.
Nothing should be assumed about the initial content of an out variable before it is passed or the procedure or function returns. That is also by definition.
And yes, of course the code is working if you assign the output to another, different, variable. That is correct coding. That's the whole point: if you assign it to the same variable you used as const or by value you are in trouble. That's the mistake.
program testout;
{$APPTYPE CONSOLE}
{$IFDEF FPC}{$MODE OBJFPC}{$ENDIF}
procedure test(const a:integer;out b:integer);
begin
 inc(b);
end;
var
 c:integer;
begin
  c := 100;
  test(c,c);
  writeln(c);
  readln;
end.

const is not obeyed and as such the program produces nonsense.
Title: Re: Out Parameter - Test program - funny behaviour
Post by: jc99 on June 17, 2015, 07:15:44 pm
I am sorry, you are missing the point again:
When you compile your program you get:
Code: [Select]
Projekt kompilieren, Ziel: project1.exe: Erfolg, Warnungen: 1, Hinweise: 1
project1.lpr(6,8) Warning: Variable "b" does not seem to be initialized
project1.lpr(4,22) Hint: Parameter "a" not used
--> you get enough hints, where to debug, when it's not working.
BTW: the result is 101, and not 1   
------------
But if you do:
Code: [Select]
procedure test(const a:integer;out b:integer);
begin
  b := a + 1;
end;
no Hints, compiles fine, output : 101 --> seems OK
------------
But this is still not the point:
Start with 
Code: [Select]
procedure test(a:integer;var b:integer);
begin
  b := a + 1;
end;

var
 c,d:integer;
begin
  c := 100;
  test(c,c);
  writeln(c);
  test(c,d);
  writeln(d);
  readln;
end.   
How to do it right ? [Edit] Without hints {%H-} does not count because you have to do it on every call.
Expected output:
101
102

Ps: A sane programmer would solve this problem with a function, or use inc(), but it's just for demonstration.
Title: Re: Out Parameter - Test program - funny behaviour
Post by: Thaddy on June 20, 2015, 10:21:14 pm
The problem is that a variable passed as const changes value. That is nonsense, not the 101 value. That one is debatable.
Title: Re: Out Parameter - Test program - funny behaviour
Post by: jc99 on June 21, 2015, 06:23:37 am
The problem is that a variable passed as const changes value. That is nonsense, not the 101 value. That one is debatable.
No it isn't. And you have to keep that in mind.

But it's no problem if you keep the IPO-scheme.

const in a Procedure/function declaration only mean that it is passed to the function by reference( a pointer) and you get a warning when you try to write to the memory it is pointing to by that variable. It does NOT mean the memory it is poining to is WRITE-PROTECTED somehow. Something else, even another thread could change that part of the memory.
out in a p/f- declaration should only mean, by reference, and has to be written first, not more not less.
About everything else the programmer has to take care of.
e.G:
Code: [Select]
   pa1 := @a;
   pa2 := @a;
   test(pa1^,pa2^);
Title: Re: Out Parameter - Test program - funny behaviour
Post by: Leledumbo on June 21, 2015, 11:44:27 am
const in a Procedure/function declaration only mean that it is passed to the function by reference( a pointer)
Not anymore since 2.6.0 (http://wiki.lazarus.freepascal.org/FPC_New_Features_2.6.0#Constref_parameter_modifier).
Title: Re: Out Parameter - Test program - funny behaviour
Post by: jc99 on June 21, 2015, 02:33:35 pm
Not anymore since 2.6.0 (http://wiki.lazarus.freepascal.org/FPC_New_Features_2.6.0#Constref_parameter_modifier).
Right, now you have the new constref-modifier to go strictly by reference. But is the rest still true, or has it also changed ?
Title: Re: Out Parameter - Test program - funny behaviour
Post by: Thaddy on June 24, 2015, 12:19:22 pm
It is still true that you are writing bogus code and it is still true that const parameter passing means that that specific parameter should not EVER change value inside of a procedure or function.

I won't spend more time on this. It's user error plain and simple.
Title: Re: Out Parameter - Test program - funny behaviour
Post by: jc99 on June 25, 2015, 12:45:57 am
It is still true that you are writing bogus code and it is still true that const parameter passing means that that specific parameter should not EVER change value inside of a procedure or function.
1. May I remind you that this discussion is about out-modifier ?
2. Is it just your opinion, or have you actual prove this ?
this would mean the value of that parameter is actual copied to a new memory-location. Which is actually done when you pass a simple parameter without any modifier.

I think you really didn't understand the reason for the introduction of the const-modifier in the first place.
... and even the compiler proves you wrong.
It's working just the way i think it should work. Except for object/strings (non simple variables) where it works once but not twice.
BTW. I have no problem this or the other way, it only should be consistent.
or since we now have an constref-modifier it would be good to have outref-modifier.

Doesn't anyone find it annoying when you use a API-function which is declared with var parameter, that you get these unnecessary 'Variable not initialized'-hints ?
When the out-parameter would be working some of the var-declaration in the API/LCL/FCL could be replaced with out/outref-declaration so you get only these hints which are actually helpful.

Wouldn't that be a good thing ?
Title: Re: Out Parameter - Test program - funny behaviour
Post by: Thaddy on June 25, 2015, 05:20:59 pm
I presented you the proof on a plate. It is not a side effect, it is user error. Now stop this charade.
Title: Re: Out Parameter - Test program - funny behaviour
Post by: jc99 on June 26, 2015, 12:58:13 am
I presented you the proof on a plate. It is not a side effect, it is user error. Now stop this charade.
Sorry I can't because the proof you gave, proofed ME right. Because the compiler DID gave hints & a warning.
And it outputs the values I'd expect !
So you find it OK when a program compile without any hints, when someone does such an obvious error ?
It' also seems everybody else is OK with functions giving different results, whether they are declared inline or not.
... I wish those a happy debugging-time, I'll use my time for something more useful ...
PS: As I said before. I know how to workaround this. And I will start ignoring the hints, because they are not useful to me.
Title: Re: Out Parameter - Test program - funny behaviour
Post by: jc99 on June 26, 2015, 04:28:36 pm
It is still true that you are writing bogus code and it is still true that const parameter passing means that that specific parameter should not EVER change value inside of a procedure or function.
About your opinion about the const-modifyer:
READ THE WIKI: http://wiki.freepascal.org/const
Quote from: ? link=http://wiki.freepascal.org/const
The declaration const in a Pascal program is used to inform the compiler that certain identifiers which are being declared are constants, that is, they are initialized with a specific value at compile time as opposed to a variable which is initialized at run time.

In some Pascal compilers, the Const declaration is used to define variables which are initialized at compile time to a certain specific value, and that the variables so defined can change as the program executes. This can be used for initializing arrays at compile time as opposed to setting values when the program is executed.
So please could you stop YOUR charade !
TinyPortal © 2005-2018