Out Parameter - Test program - funny behaviour

[jc99]

I have a program/problem:
see: http://bugs.freepascal.org/view.php?id=28279

Code: [Select]

program PrjTestParameter;

{$IFDEF MSWINDOWS}
{$APPTYPE CONSOLE}
{$ENDIF}

{ $Define WorkAround1}
{ $Define WorkAround2}
{ $Define WorkAround3}

Function Test({const}Line:String;out Part1:string;{$ifndef WorkAround2} out{$else}var{$endif} Part2:string):Boolean;{$ifdef WorkAround3} inline;{$endif}

begin
  result := length(Line)>1;
  Part1:= copy(Line,1,1);
  Part2:=copy(Line,2,length(Line)-1);
end;

var
  BaseLine: String;
  OneChar: string;
  Baseline2: string;

begin
  BaseLine := 'ABCDEFGHIJKLMNOP';
  while Test(Baseline,OneChar,Baseline) do
    begin
     writeln(Onechar);
{$ifdef WorkAround1}
     BaseLine2:=BAseline;
{$endif}
    end;
  readln;
end.
In my opinion this should work.
If it is wrong then Workaround1 (not 2, sorry my mistake) should not make it work.

Both workarounds are not satisfying, the second gives a hint that a variable isn't initialized. The first uses an unnecessary variable.

Edit:
In this form, it may be obvious that it is not working ...
but think of it that the procedure is in one unit and the call is in another unit. ... and it's a big project, lot's of files, ton's of code ...
Then think that the procedure started as

Code: [Select]

Function Test(Line:String;var Part1,Part2:string):Boolean;everything was fine.
The Story goes:
Then a new compiler came and tells you that some variables are not initialized. So you look at the code and say "Oh, yes, these paramaters are only written to so they should be out-Parameters". You change the var to out, compile the program, no error no warning, no hint.
Program running fine. 'A' as baseline works, 'AB' as baseline works.
But then after releasing the Prog somehow you have to do 'ABC' as baseline and BOOM, where is my B and C.

So my point is:
This should either be working, or there should be a Warning that it may be not working.

Edit:
Found Workaround3: declare the function inline

[Basile B.]

The problem is that when you use the out parameter storage class, the value you pass is implicitly set to its default value (empty string). example:

Code: [Select]

program Project1;

var
  str: string;

procedure MagicReset(out value: string);
begin
end;

begin
  str := 'wxcvbn,;:!';
  MagicReset(str);
  writeln(str);
  readln;
end.   

But, in your example, when Baseline is magically reset, since ObjectPascal string are reference counted, this reference count is reset to 0. While actually it shouldn't. That's a bug.

read more here, Delphi has (had?) the same problem:
http://alex.ciobanu.org/?p=48

[Martin_fr]

About your remark in the linked bug:

Code: [Select]

When debugging you will see, that at the end of the first call of test Line still has 'ABCD...', Part1 := 'A', Part2 := 'BCD..'Assuming that statement was checked with "const" in code. (const kind of makes the string a pchar, not guaranteeing any ref count)

The debugger does not check if the variable is valid. A string variable is a pointer, and a "const string" can point to the remains of a string. That is it can point to freed memory, that has not yet been overridden and therefore looks like the string you expect.

In case if there is no "const" then it depends on the refcount. (see below)

Actually there is a bug. However that is probably not what you experienced. Because this bug passes in an invalid/trashed string.
--------------

The reason you may (or may not) see the "correct" string in "Line" is the ref count of the variable "Baseline". In the first iteration of the loop, you have a constant (special ref count, afaik -1), that works different. In the 2nd iteration of the loop the refcount should be 1, and things may go wrong.
However any slight change of the code (or me mis-reading it) and the refcount is 2 or higher, and it behaves more like you want (at least it may, no guarantee here).

----------
Even if the issue I mentioned earlier is fixed, your code is still not good.
That is the value of "Line" still depends on the order of argument evaluation. And afaik in your code it will first be set to empty string.
(Well that is assumed the order of param eval is documented, and therefore guaranteed. IIRC it is, but not sure)

I agree your code looks good at first, and it takes a lot of detailed knowledge to find why it does not work.
As I said *IF* the order of param eval is documented (and in this case documented last param is evaled first (at least for pascal calling convention).
1) out param is evaluated => variable is set to empty string.
   **IF** refcount is 1, then it affects the next param, otherwise not.
2) Line is evaluated. This may be the empty string.

[jc99]

@Martin_fr

Assuming that statement was checked with "const" in code. (const kind of makes the string a pchar, not guaranteeing any ref count)

No it behaves like this in both ways (with and without "const")

@BBasile

read more here, Delphi has (had?) the same problem:

In Delphi (XE & XE2) it is different, the code does not work at all,
the parameters are reset right away. So you have no problem debugging it.

My point is:
Since the compiler gives so much Hints about not initialized Variables, it should give a Warning when the same parameter is passed to Procedure/Function twice in different manners.
The better solution would be to make the thing work.
1. always evaluate out Parameter last/first (same way)
2. increment the counter, when the parameter is used
or do the Initialization only when it's not initialized.

BUT:

Code: [Select]

  BaseLine := 'ABC';
  if test(BaseLine,Onechar,BaseLine) then
    writeln('1: ',Onechar);

  if test(BaseLine,Onechar,BaseLine) then
    writeln('2: ',Onechar);
(Code fails on second call) in this case it should be obvious to the compiler, that the parameter is used again, but it still fails.

[jc99]

Even if the issue I mentioned earlier is fixed, your code is still not good.
That is the value of "Line" still depends on the order of argument evaluation. And afaik in your code it will first be set to empty string.
(Well that is assumed the order of param eval is documented, and therefore guaranteed. IIRC it is, but not sure)

I switched the parameters in the test Function (out,out,const/direct),
and the program/compiler still behaves this way.

[Basile B.]

My point is more that it shouldn't be allowed at all to use the same parameter as in and out. The fact is that IRL everyone would have written the function:

Code: [Select]

function(var Line: string; out current char): boolean;
It's bad luck you've been faced to this crazy corner case.
Though, interesting to analyze. The link i've put in my first answer EXACTLY describes the same thing.

[jc99]

My point is more that it shouldn't be allowed at all to use the same parameter as in and out. The fact is that IRL everyone would have written the function:

Code: [Select]

function(var Line: string; out current char): boolean;
It's bad luck you've been faced to this crazy corner case.
Though, interesting to analyze. The link i've put in my first answer EXACTLY describes the same thing.

1. I try to make a different point: If it is allowed, it should work. Or if it isn't, there should be a warning.
2. Your answer DOES describe the same thing. What I tried to point out that dpc and fpc handle the code differently.
in Delphi the program fails on first call. In FPC the first call seems OK and the program fails on second call.

[jc99]

I know it's not good to reply to your own diskusion, but there is new info:
Subject closed:

Other people have already answered in the forum. See also the discussion on the fpc mailing list: http://lists.freepascal.org/pipermail/fpc-devel/2015-June/thread.html ("ref count issue with out param"). Your remark regarding the warning (or rather, why a warning that always works is impossible to add) is addressed in http://lists.freepascal.org/pipermail/fpc-devel/2015-June/035706.html

I am not so happy that the issue is marked resolved.
Because (i hope) there is an ongoing discusion about this.
Nobody seems to care about the point I try to make:
You start with a

Code: [Select]

function foo(aPar1:TSomething;var aPar2:TSomething):TSomethingElse;
You test is, release it.--> Valid.

SomeOne (else) uses your function.

Code: [Select]

while foo(s1,s1) do [...]
He tests his code, still valid.

Years later the compiler has a new feature (Hints and out-params)
You came across your old function
and since you only write to aPar2 (and you want get rid of the annoying hint that the variable is not initialized.
you change the var to out.
by this you make someone else's code WRONG.
I think that some deserves a warning that his formerly valid code may have problems now.

I am fully aware that you can not warn everybody with everything. But a warning when a parameter is obviously handed twice (or more) to a routine with one of which is an "out" would be a start.

[...]
Found Workaround3: declare the function inline

And also a function should not change it's (obvious) behavior when declared inline (or not). If you are a good programmer then you normally put your "inline" in a compiler
-def like

Code: [Select]

function foo([...]):TSomething; {$IFDEF SUPPORTS_INLINE}inline;{$ENDIF}

The next point is:
You use this function:

Code: [Select]

var p1,p2:string;
[...]
  p1:='ABCDE';
  while foo(p1,p2) do
     p1:=p2;
[...]
That is obviously VALID code.
then someone ads a new fancy optimization to the compiler
Namely: when a variable is only used to fill a parameter it can be replaced with that variable, to get rid of an obvious unnecessary copy.
The formerly valid code starts to behave erratic. which is a very unpleasant thing, if you try to debug a large project.

At some point

Code: [Select]

i := i + 1;
is not valid anymore because you use variable i as an input and also as a target of that code.
(just kidding)

[Thaddy]

Since I noticed from the discussions that only some DXE's were tested for reference:
D7 has the same  behavior  The out parameter is initialized to nil. Accordingly the output is nothing.
Only with the var parameter, D7 outputs the separate characters.
If you use the same variable as input for both parameters it gets reset to nil.
So from a delphi compatibility  point of view, you may hope for documentation of this behavior but it won't ever be "fixed" in any other way.

[jc99]

Since I noticed from the discussions that only some DXE's were tested for reference:
D7 has the same  behavior  The out parameter is initialized to nil. Accordingly the output is nothing.
Only with the var parameter, D7 outputs the separate characters.

DXE2 was only the newest Compiler availible for me, If i wanted I could Do a synopsis from tp3 over bp7  via d2, d4 to dxe2 where i stoped buying New compilers. My Code also developed through the different versions. BTW Bp7 was my favorite for a long time.
But as You See above there was a link to a Delphi Blog that they try to fix that in newer versions.
So i only tried the newer versions.
The reason i prefered Pascal over C, C++, C#, Java (and all the other New shit) was the structure and encapulation. And Not have to think about, oh when i Do this then the reference count is Not Set correctly ... To me that is against the Basic philosophy of Pascal.

[Thaddy]

Mm, but given that you access the same variable, the same memory location if you wish, and pass this location in two (three) contradictory ways in your routine, at best your code should never compile. It should probably generate an error.
Furthermore, the order in which the parameters are evaluated by a Pascal compiler is probably implementation specific in the case of out. Any differences between FPC and Delphi can be explained in this way: Given that an out parameter is defined as only valid on exit, compiler logic can assume it can optimize certain code by changing the order in which parameters are used.
The value of the out parameter is by definition undefined when passed in, you really can not expect anything else. The compiler can only turn this into proper code by initializing it to a default, i.e. nil, in the proccedure body. This behavior is documented for Delphi and FPC alike. This means that if the same memory location is referenced it also affects the by value or const parameter in a contradictory way. Your code should never work. The compiler should never be compromized by handling a refcount because of an out parameter, but the programmer should probably be warned of his error. In my opinion this is, as the majority in the discussion opiniate or state, user error.

The valid point  is not that your code is correct, it isn't, but that this construction should be caught by the compiler as an error, much in the same way as trying to change a parameter passed in as const generates a compile time error.

Not to put too fine a point on it, but the latter is effectively exactly what you are doing in part of your example. The difference is that this can only be caught at runtime.

[jc99]

Then I have to remind you of the topic. It is NOT about fooling the Compiler with all possible variants and waiting for the crash. It is about out parameter (replacing the var-parameter). Like const it is a way to tell the compiler what you want to do with it.

• var - for in and out. (has to be initialized, value might be used, mother of all)
• const - for in only. (has to be initialized, write-access is warned)
• out - for out only. (no need to be initilialized, read-access is warned)

all three are by reference and should be carefully used by the programmer.
--
Now I have an atomic function/procedure :
1 in value
3 out value. (1 fct-result, 2 var or out)
What is than the right way to do it ?
--
BTW.
the whole thing is working, when the value after the call is READ and put to another variable.
This could be the lever to a solution if someone is willing.
Wouldn't it be a good thing, to get this working ?
Why is the out parameter initialized in the first place ? When you try a read-access you defintly get a warning. For a good programmer that should be enough to find his error.

[Thaddy]

The variable is initialized because it is on the stack. The stack is dirty by definition. To get predictable, sensible results the variable needs to be predictable, i.e. initialized to a known value.
Same goes if the variable is optimized to be in a register. The register is dirty by default, to get predictable results.... etc etc.
Nothing should be assumed about the initial content of an out variable before it is passed or the procedure or function returns. That is also by definition.
And yes, of course the code is working if you assign the output to another, different, variable. That is correct coding. That's the whole point: if you assign it to the same variable you used as const or by value you are in trouble. That's the mistake.
program testout;
{$APPTYPE CONSOLE}
{$IFDEF FPC}{$MODE OBJFPC}{$ENDIF}
procedure test(const a:integer;out b:integer);
begin
 inc(b);
end;
var
 c:integer;
begin
  c := 100;
  test(c,c);
  writeln(c);
  readln;
end.

const is not obeyed and as such the program produces nonsense.

[jc99]

I am sorry, you are missing the point again:
When you compile your program you get:

Code: [Select]

Projekt kompilieren, Ziel: project1.exe: Erfolg, Warnungen: 1, Hinweise: 1
project1.lpr(6,8) Warning: Variable "b" does not seem to be initialized
project1.lpr(4,22) Hint: Parameter "a" not used
--> you get enough hints, where to debug, when it's not working.
BTW: the result is 101, and not 1   
------------
But if you do:

Code: [Select]

procedure test(const a:integer;out b:integer);
begin
  b := a + 1;
end;
no Hints, compiles fine, output : 101 --> seems OK
------------
But this is still not the point:
Start with 

Code: [Select]

procedure test(a:integer;var b:integer);
begin
  b := a + 1;
end;

var
 c,d:integer;
begin
  c := 100;
  test(c,c);
  writeln(c);
  test(c,d);
  writeln(d);
  readln;
end.   
How to do it right ? [Edit] Without hints {%H-} does not count because you have to do it on every call.
Expected output:
101
102

Ps: A sane programmer would solve this problem with a function, or use inc(), but it's just for demonstration.

[Thaddy]

The problem is that a variable passed as const changes value. That is nonsense, not the 101 value. That one is debatable.