Recent

Author Topic: TProcess.Execute behavior  (Read 12642 times)

lagprogramming

  • Full Member
  • ***
  • Posts: 159
TProcess.Execute behavior
« on: May 30, 2015, 04:14:26 pm »
   Setting the value of TProcess.Executable to an executable_file without path(directory) and setting TProcess.CurrentDirectory to the executable_file's directory might raise an error. Strangely to me, the error appears only if the application runs within Lazarus. If I run the application outside Lazarus then everything is fine.
   Example:
   TProcess.Executable:='/home/user/appdir/executablefile';TProcess.CurrentDirectory:='/home/user/appdir/';TProcess.Execute;//Works fine.
   TProcess.Executable:='executablefile';TProcess.CurrentDirectory:='/home/user/appdir/';TProcess.Execute;//Raises an exception if I run the application inside Lazarus, but works fine if I run the application outside Lazarus. I talk about the same binary files involved. Most probably this is because "TProcess.Executable" doesn't look for the executablefile within the TProcess.CurrentDirectory. A patch is attached that workarounds this. I consider it a workaround because I don't know why the same application file runs fine outside Lazarus, but gives an error if I run it with F9 inside Lazarus. Also, I know that I want this behaviour(searching within TProcess.CurrentDirectory) but I'm not sure that TProcess.Execute should do that. Also, should TProcess.CurrentDirectory be the last directory to look in or not, maybe it should be the first one.


   As a side-note looking at fina.inc.IncludeTrailingPathDelimiter I've noticed that
{$ifdef SYSUTILSUNICODE}
...
{$else SYSUTILSUNICODE}
...
{$endif SYSUTILSUNICODE}
   Has no greyed text within Lazarus(stable 1.4.0). It's as if Lazarus says that both "then" and "else" codes would be executed. I might be tired now but I think there should be some greyed text inside.

Edit: Obviously I forgot the attachment.  %)
« Last Edit: May 30, 2015, 04:17:40 pm by lagprogramming »

marcov

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 7453
Re: TProcess.Execute behavior
« Reply #1 on: May 30, 2015, 04:25:58 pm »
TProcess is not a shell, it is documented that it only searched in the PATH.

In general, search for binaries on *nix doesn't include the current directory (like in Windows) because that is considered a security risk.

Basile B.

  • Guest
Re: TProcess.Execute behavior
« Reply #2 on: May 30, 2015, 05:44:05 pm »
TProcess.CurrentDirectory is not a search path, it's the cwd for the application that's gonna be launched. It's like if you cd to that path and then starts the application. If in the remote application you refer to a filename without path, then it can be found thanks to this.

If you strip the TProcess.exectable path off then the file must be found in one of the environment variable.
In short if in konsole you can type 'myApp' without the path and it's executed then you can strip the path.

lagprogramming

  • Full Member
  • ***
  • Posts: 159
Re: TProcess.Execute behavior
« Reply #3 on: May 31, 2015, 04:16:16 pm »
   The fact that an error appears if I run the application within Lazarus and the error doesn't appear when I run it outside Lazarus makes me think that something might be wrong somewhere. The same files are used, in the same directories. What you two have written is right, so I might not looking at the right place. I abort the digging process because for now I have to use the latest stable version of Lazarus, which compiles and runs ok with an ancient version of fpc trunk(something like svn 30101). New versions of FPC trunk(now ~30940) make latest stable Lazarus give an error when trying to run a new project(default empty form): "Unable to stream Form1...". Latest trunk versions of Fpc/Lazarus work together fine but it's not the combination I'm looking for, neither the combination I've used for testing.
   Anyway, until testing using latest svn versions of both Fpc/Lazarus...there's no point in continuing, now.
   Thank you for replies.

marcov

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 7453
Re: TProcess.Execute behavior
« Reply #4 on: May 31, 2015, 04:20:29 pm »
Maybe the difference is the actual current directory before executing?

lagprogramming

  • Full Member
  • ***
  • Posts: 159
Re: TProcess.Execute behavior
« Reply #5 on: May 31, 2015, 10:09:32 pm »
   This is a long reply so first of all:
1. I'm satisfied with current behaviour of TProcess.Execute and If I would be a main developer I would not implement the attached patch in trunk for more than just a single reason.
2. You should not invest any resources in this anomaly. Have you seen the following link!?
http://bugs.freepascal.org/view.php?id=27670
   Not discouraged yet, read twice paragraph E :). I would not try to reproduce this situation.


   "Maybe the difference is the actual current directory before executing?"
   This is what I've suspected, too. Maybe this post needs a brief introduction in chronological order:
A. Apparently Lazarus doesn't always rebuild itself properly. I think I've removed the influence of operating system file cache.(* may be read at the end of the post)
B. Assuming that the forum post anomaly appears from a programming mistake done by me, well, I don't directly use "chdir" inside the application that calls other executables. This means that current working directory would be changed only by LCL/FCL/RTL/LAZARUS_IDE_THROUGH_A_PATH_MODIFIER_ROUTINE,LOCAL_CONDITIONAL_COMPILATION.... I don't know what other reasons may be involved. For my own knowledge, is there a conditional based on running an application within Lazarus, something like "csDesigning in ComponentState"? "csDesigning in ComponentState" doesn't say if the application runs within Lazarus.
C. The exception is risen within TProcess.Execute, but this doesn't mean that the problem is inside that routine.
D. Maybe one core developer, during the history of Fpc or Lazarus development, made a confusion regarding curent directory usage. Together with the initial post, the first two replies of this forum post clearly state that such automation is not welcome. I agree with the designed behaviour, but it's an easy to make misinterpretation(or expectation). This means that there is a possibility that a routine like "fileexists", "directoryexists", "findfirst" or whatever else, might change the working directory internally without reverting back. This is a possibility, not something certain. It looks unlikely to me but I've seen such confusions. Look at lazarus/lcl/graphtype.pp.TRawImageDescription.Init_BPP*. It's just an example where copy/paste(see the comment at procedure "TRawImageDescription.Init_BPP1(AWidth, AHeight: integer);") might lead to misinterpretations/inconsistencies. I've mentioned a 24bit related comment, but for some people bitsperpixel(bpp), pixelformat(pf),... what means "depth" variable inside there!? It's just an example of another not so likely situation that may lead to confusion or misunderstanding. Lazarus has a history of LCL graphics related issues, proof that there is a possibility that a simmilar situation might repeat with fpc.
E. Due to requirements(OS, FPC(svn ~30100)/Lazarus(1.4.0) versions involved...), paragraph A can't be reproduced easily by other people. Without a Lazarus stable release paragraph "A" can't be verified. Also, the computer I use to do the tests is known to have "personality", meaning that I always keep into account variations from required/expected/considered_normal behaviours.



(*) What follows here is a long and boring explanation for paragraph A, nothing more.
   A different approach to updating Lazarus is presented in the following link(you don't have to read it because I'll resume what I consider is important)
http://forum.lazarus.freepascal.org/index.php/topic,28519.0.html
   The presented Lazarus update process consists in running an external tool that creates/overwrites/modifies some files and removes some files/directories. The presentation runs an external application from the Lazarus tool menu, meaning that Lazarus is running. After the external tool does it's job the user rebuilds Lazarus. It's a very easy approach for the Lazarus user, but there is a hard to notice anomaly, an annomally that doesn't bother anyone except for those reading this part of the forum post :). If a user modifies his stable release with a development(svn) version and then reverts back to the stable release, even though the source files are identical, no matter how many times the user rebuilds Lazarus, when Lazarus starts, it will show the development version on the splash screen, not the stable one(1.4.0 in the original post). The user may rebuild Lazarus clean as many times as he wants, as long as it's rebuilt using the "Tools" menu it will show a development version at splash screen, not the real version of the source files/directories. If the user closes Lazarus and rebuilds it using "make clean all" in a terminal window, the stable version will appear on the splash screen.
   Initially I thought the problem was that Lazarus opened some files before running the external tool, Linux caches those files and uses them during child processes. The fact that the user can close Lazarus, manually start it again using startlazarus, rebuild it clean using an external application(fpc) and still have an obsolete information on the splash screen makes me wonder. It's not a problem of OS cache, it's not an OS problem of data read within files, it's not a particular issue with a particular source file modification, maybe the problem lies somewhere in the way Lazarus is designed to rebuilding itself, or maybe it's something fpc related.
   Maybe the source of the anomaly has been fixed in the meantime but I'm unable to verify it because I need a latest fpc trunk compilable and runnable Lazarus stable version. I can't compare two svn Lazarus versions. I'm the only one that noticed this situation and I think I've done something good at documenting it and now we should let it go. If the situation gets worse I(or somebody else) will revive this forum post.
   Thank you very much. If I consider I find something useful I'll append that to this post.

marcov

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 7453
Re: TProcess.Execute behavior
« Reply #6 on: May 31, 2015, 10:18:06 pm »
 I  don't have time to plough through that, but I made the current directory mark because Delphi also suffers from it.

There it can depend on the way you open a project (using most recent projects or navigating with file->open (project)  ). The latter way changes dir to the dproj the dir, the most recent way not.


lagprogramming

  • Full Member
  • ***
  • Posts: 159
Re: TProcess.Execute behavior
« Reply #7 on: September 07, 2015, 05:19:48 pm »
   I'm not sure what's going on but I think that fpc might return a wrong exit code if it fails to build itself during the second or third cycle, or maybe there is a TProcess.Exitcode related problem... I want to investigate further, in order to be sure that the exit code is returned as expected I'd like an alternative to TProcess.Exitcode.
   I'm looking for a command in terminal, something like: command_that_shows_exitcode "fpc parameters". Obviously, the command must not be fpc related. Alternatives for both linux and windows are appreciated.
   Edit: I use "echo $?" in linux.
« Last Edit: September 07, 2015, 07:38:52 pm by lagprogramming »

lagprogramming

  • Full Member
  • ***
  • Posts: 159
Re: TProcess.Execute behavior
« Reply #8 on: September 09, 2015, 03:09:10 pm »
   I've manged to reduce this to an example. Exitcode has a different value than "echo $?" for the same command. I've looked at Lazarus's internal console window and it shows the error like it does when I run in a terminal.

Code: [Select]
//Add "process" to "uses"
procedure TForm1.Button1Click(Sender: TObject);
const localexecutable='make';
      fpcsources='/home/user/fpc/';//valid fpc source directory value
var localprocess:tprocess;
begin
  chdir(fpcsources);
  localprocess:=tprocess.Create(nil);
  localprocess.Executable:=localexecutable;

  //The parameters are intentionally wrong because we want an error
  localprocess.Parameters.Add('crossall');
  localprocess.Parameters.Add('crossinstall');
  localprocess.Parameters.Add('OS.TARGET=win32');
  localprocess.Parameters.Add('CPU.TARGET=i386');
  localprocess.Parameters.Add('INSTALL_PREFIX=none');
  localprocess.Options:=localprocess.Options+[poWaitOnExit];
  localprocess.Execute;

  //The caption will become 0, while "echo $?" in a terminal will show "2". "2" is the right value. Tested with linux-x86_64.
  button1.Caption:=inttostr(localprocess.ExitCode);
  localprocess.Free;
end;

   Something is not right and I can't figure out where is the problem.

marcov

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 7453
Re: TProcess.Execute behavior
« Reply #9 on: September 09, 2015, 03:56:16 pm »
You can check the error code that make returns using strace <fpcapp>

lagprogramming

  • Full Member
  • ***
  • Posts: 159
Re: TProcess.Execute behavior
« Reply #10 on: September 09, 2015, 06:31:02 pm »
   "strace make clean crossall crossinstall OS.TARGET=win32 CPU.TARGET=i386 INSTALL_PREFIX=none"
   ends up with "+++ exited with 2 +++" which is what I expect.
   It's like, when executing the same command using TProcess the ExitCode is not updated.
   The above code has been further simplified by executing "fpc --help". Both strace and "echo $?" show exit code to be 1. The caption of the label shows zero.

Code: [Select]
procedure TForm1.Button1Click(Sender: TObject);
//Requires "process" within "uses"
const localexecutable='fpc';
var localprocess:tprocess;
begin
  localprocess:=tprocess.Create(nil);
  localprocess.Executable:=localexecutable;

  //The parameter is intentionally wrong because we want an error
  localprocess.Parameters.Add('--help');
  localprocess.Options:=localprocess.Options+[poWaitOnExit];
  localprocess.Execute;

  //The caption will become 0, while "echo $?" in a terminal will show "1". "1" is the right value. Tested with linux-x86_64.
  button1.Caption:=inttostr(localprocess.ExitCode);
  localprocess.Free;
end;
   The only difference between executing the latest stable fpc and a recent trunk fpc version is the text output, not the message code.
   If you don't have clue then I'll take a closer look at "PeekExitStatus" calls, but I hope it won't be necessary.

Leledumbo

  • Hero Member
  • *****
  • Posts: 8108
  • Programming + Glam Metal + Tae Kwon Do = Me
Re: TProcess.Execute behavior
« Reply #11 on: September 09, 2015, 07:51:35 pm »
It's like, when executing the same command using TProcess the ExitCode is not updated.
   The above code has been further simplified by executing "fpc --help". Both strace and "echo $?" show exit code to be 1. The caption of the label shows zero.
Use ExitStatus instead of ExitCode.

lagprogramming

  • Full Member
  • ***
  • Posts: 159
Re: TProcess.Execute behavior
« Reply #12 on: September 10, 2015, 12:24:26 am »
   1. What's the difference between ExitStatus and ExitCode within FreePascal? Why ExitStatus and not ExitCode?
   2. The pascal code I'm interested should be used to build crossplatform binaries that call external applications. Is ExitStatus safer than ExitCode for this purpose?

  P.S.  Once more, Leledumbo's single answer connects questions that I ask in different forum threads(topics). I highly appreciate Marcov's support, too.

molly

  • Hero Member
  • *****
  • Posts: 2345
Re: TProcess.Execute behavior
« Reply #13 on: September 10, 2015, 01:09:07 am »
Quote
1. What's the difference between ExitStatus and ExitCode within FreePascal? Why ExitStatus and not ExitCode?
2. The pascal code I'm interested should be used to build crossplatform binaries that call external applications. Is ExitStatus safer than ExitCode for this purpose?
In principle they are the same, but with an exception (or actually two).

One is that an exitcode can be 'customized' by using terminate (in which case the value doesn't have to be the same) and the second one being that it depends on how tprocess unit was compiled (tprocess can be compiled without getexitcode support (reason for me is unknown, perhaps deprecated ?).

Short answer: Use exitstatus, with/without additional analyzing standard error output.

lagprogramming

  • Full Member
  • ***
  • Posts: 159
Re: TProcess.Execute behavior
« Reply #14 on: September 10, 2015, 08:28:45 am »
Thank you Molly.
I'll use ExitStatus instead of ExitCode.