Recent

Author Topic: Best approach to create a *.exe  (Read 936 times)

GreatCorn

  • New Member
  • *
  • Posts: 22
  • A musician, programmer, formerly a game dev.
Best approach to create a *.exe
« on: June 01, 2020, 09:02:10 pm »
Sorry, I need some advice for this. I've been doing something like a command-oriented language interpreter on Pascal for another app and came to a dilemma. What would be the best practice to turn it into a compiler? 'Build' a code written in it into an .exe file (for now I'm working Windows only)? My goal is to have a single .exe file that is able to execute that code, that's in the best result. A worse result would be to rely on any other files, but if it comes to only that, I guess I'll have to accept it.
I've got ideas, but they have their problems:
  • Translating the code to Pascal and using fpc.exe to build it (that's relying on a copy of fpc.exe and some units probably);
  • Embedding the code into the interpreter itself, creating a link with parameters pointing to the code file (relies on the code file and creates a link instead of a program, however, there could be a possibility to embed the code in a way that I don't know of);
  • Embedding a premade program into the interpreter to extract (still relies on the code file);
  • Bitwise .exe file building (God why).
I apologize if I'm just being stupid or don't understand the problem itself in its entirety. Thank you for your time.
-Corn

jamie

  • Hero Member
  • *****
  • Posts: 3141
Re: Best approach to create a *.exe
« Reply #1 on: June 01, 2020, 09:50:29 pm »
What you are asking is like a run time with a attached script type language or maybe even in byte code level to make it run faster..

 This is what I would do, Write an engine exe that when initiated will read the resources it its own file looking for a RAW type resource name which would be your program. Read it in and offer it to your engine
inside..
 of course you need to also make another program that will set all of this up for you as a single EXE file..

 so you need a tool for example that can create the resource and bind the engine EXE and the resource together to form a single EXE with the attached data.

The only true wisdom is knowing you know nothing

GreatCorn

  • New Member
  • *
  • Posts: 22
  • A musician, programmer, formerly a game dev.
Re: Best approach to create a *.exe
« Reply #2 on: June 01, 2020, 10:25:13 pm »
This is what I would do, Write an engine exe that when initiated will read the resources it its own file looking for a RAW type resource name which would be your program. Read it in and offer it to your engine
inside..
That's an interesting approach, didn't think of that. But that lead me to thinking... What if I could try appending the code at the end of a raw exe? I will have to study its structure to see if doing so would lead to any errors or problems, but so far when I tested it by adding it via Notepad++, nothing out of the ordinary happened.
of course you need to also make another program that will set all of this up for you as a single EXE file..
Or, if possible, I could try using the interpreter itself, just copying it. Thanks for the idea. Are there any issues with appending plain text at the end of a .exe though?
-Corn

GreatCorn

  • New Member
  • *
  • Posts: 22
  • A musician, programmer, formerly a game dev.
Re: Best approach to create a *.exe
« Reply #3 on: June 01, 2020, 10:58:06 pm »
Ran into an issue when trying to actually implement this... The program trying to read itself (using Assign, Reset, etc) gets an access denied error. Trying to use DOS' Exec and then running cmd with copy source.exe destination.exe does the job flawlessly tho

EDIT: Windows.CopyFile does the job for now
« Last Edit: June 01, 2020, 11:04:37 pm by GreatCorn »
-Corn

lucamar

  • Hero Member
  • *****
  • Posts: 2930
Re: Best approach to create a *.exe
« Reply #4 on: June 01, 2020, 11:04:55 pm »
Are there any issues with appending plain text at the end of a .exe though?

Not many and, in fact, that was a quick way to add "resources" to your exe in the old days; for example, that's how PKZIP (and similar tools) used to build its auto-extracting archives.

Nowadays, though, you might easily run into problems with antivirus, anti-malware, etc. programs so one approach to solve those might be to include the script in your EXE as a real resource. Look for tools that can add new resources to already linked EXEs, like ResourceHacker and alike.
Turbo Pascal 3 CP/M - Amstrad PCW 8256 (512 KB !!!) :P
Lazarus 2.0.8/FPC 3.0.4 - 32/64 bits on:
(K|L|X)Ubuntu 12..18, Windows XP, 7, 10 and various DOSes.

GreatCorn

  • New Member
  • *
  • Posts: 22
  • A musician, programmer, formerly a game dev.
Re: Best approach to create a *.exe
« Reply #5 on: June 01, 2020, 11:28:11 pm »
Nowadays, though, you might easily run into problems with antivirus, anti-malware, etc. programs so one approach to solve those might be to include the script in your EXE as a real resource. Look for tools that can add new resources to already linked EXEs, like ResourceHacker and alike.
Yeah, nowadays the antiviruses can be a bit of a problem even without doing hacky stuff like this. I have ResourceHacker, but I'd have to figure out how to implement its function in Pascal. It's not too hard I guess, especially when the target file is a text file.
Actually, while Windows.CopyFile solved the problem with copying I still need for the program to read itself.. Are there any methods to do so without causing an Access Denied EInOutError?

EDIT: I could try using other methods like TFileStream, etc, but I'm using FPC while trying to maintain as low space usage as possible
« Last Edit: June 01, 2020, 11:30:57 pm by GreatCorn »
-Corn

lucamar

  • Hero Member
  • *****
  • Posts: 2930
Re: Best approach to create a *.exe
« Reply #6 on: June 02, 2020, 02:43:09 am »
Actually, while Windows.CopyFile solved the problem with copying I still need for the program to read itself.. Are there any methods to do so without causing an Access Denied EInOutError?

Let's say you do:
Code: Text  [Select][+][-]
  1. copy /b stub.exe+textfile.txt final.exe
or the equivalent in code. That would produce your combo exe+text.

When you run the resulting exe what it should do, IIRC, to get the text is read its own header (depending on your program by opening ParamStr[0] or Application.ExeName) to get the length of the executable part (the "stub.exe") and then read the file from that position (+1) to the real end of the file.

Since you don't need any other info you can get the format of that header from for example Wikipedia: DOS MZ executable and Portable Executable to see which fields you need to read to get the executable's length (or get a pointer to better descriptions).

HTH
« Last Edit: June 02, 2020, 02:48:51 am by lucamar »
Turbo Pascal 3 CP/M - Amstrad PCW 8256 (512 KB !!!) :P
Lazarus 2.0.8/FPC 3.0.4 - 32/64 bits on:
(K|L|X)Ubuntu 12..18, Windows XP, 7, 10 and various DOSes.

PascalDragon

  • Hero Member
  • *****
  • Posts: 1768
  • Compiler Developer
Re: Best approach to create a *.exe
« Reply #7 on: June 02, 2020, 09:53:28 am »
Nowadays, though, you might easily run into problems with antivirus, anti-malware, etc. programs so one approach to solve those might be to include the script in your EXE as a real resource. Look for tools that can add new resources to already linked EXEs, like ResourceHacker and alike.
Yeah, nowadays the antiviruses can be a bit of a problem even without doing hacky stuff like this. I have ResourceHacker, but I'd have to figure out how to implement its function in Pascal. It's not too hard I guess, especially when the target file is a text file.
Actually, while Windows.CopyFile solved the problem with copying I still need for the program to read itself.. Are there any methods to do so without causing an Access Denied EInOutError?

EDIT: I could try using other methods like TFileStream, etc, but I'm using FPC while trying to maintain as low space usage as possible

I would suggest you to use resources together with TResourceStream (it's usage is for example described here). Alternatively you can directly use FindResource and friends (they are provided by the System unit). It's built after Windows' Resource API, but it's implemented in a cross platform way in FPC.

Additional point in favour of using resources: you can safely code sign your executable.

Thaddy

  • Hero Member
  • *****
  • Posts: 10213
Re: Best approach to create a *.exe
« Reply #8 on: June 02, 2020, 11:12:09 am »
Are there any issues with appending plain text at the end of a .exe though?

Not many and, in fact, that was a quick way to add "resources" to your exe in the old days; for example, that's how PKZIP (and similar tools) used to build its auto-extracting archives.
Well... Many, because of manifests and certificates and code signing in general. That would destroy the manifest's intention, the certificate and thus the signed code, because it relies on a size and hash too....
For your own code that does not matter too much if at all, but when you try to add or change code or data to a signed program it will not execute at all in modern Windows.. (Ahhh the days of XP have long gone....) and since resources are data, same effect...

For good measure: not Windows 7+ only, but e.g. also Android and all Apple OS's and secure linux.
« Last Edit: June 02, 2020, 11:17:28 am by Thaddy »
I am more like donkey than shrek

Mr.Madguy

  • Hero Member
  • *****
  • Posts: 508
Re: Best approach to create a *.exe
« Reply #9 on: June 02, 2020, 01:17:26 pm »
I experimented with launching app from resource, like old versions of ProcessExplorer did. Problem is - since some moment anti-viruses started to hate CreateProcess. Use ShellExecute instead. This example also has clunky work with command line arguments, because anti-viruses hated GetCommandLine. Another problem - res file has to be manually deleted in order for it to be rebuilt, if exe files are updated.

P.S. You can also add data directly into PE sections. You can easily access this data, because hInstance is actually just address of MZ+PE header. I used this tech in some very old project, that was used to add special stub into application, that would be able to enable/disable/terminate this application via network. And I also patched NFS4 to run on WinXP, because it had faulty PE section headers. :-[
Code: Pascal  [Select][+][-]
  1. program Launcher;
  2.  
  3. {$R 'Prog.res' 'Prog.rc'}
  4.  
  5. uses
  6.   Windows,
  7.   SysUtils,
  8.   ShellAPI,
  9.   Classes;
  10.  
  11. {$R *.res}
  12.  
  13. type
  14.   TIsWOW64Process = function(hProcess:THandle;var IsWOW64:Boolean):Boolean;stdcall;
  15.  
  16. const
  17.   FileName = 'Prog.exe';
  18.   Res32 = 'Prog32';
  19.   Res64 = 'Prog64';
  20.  
  21. function GetTempFolder: String;
  22. var
  23.   Buffer: array [0..MAX_PATH+1] of WideChar;
  24. begin
  25.   GetTempPath(MAX_PATH, Buffer);
  26.   Result := IncludeTrailingPathDelimiter(String(Buffer));
  27. end;
  28.  
  29. function Is64Process:Boolean;
  30.   var Kernel:HMODULE;IsWOW64Process:TIsWOW64Process;Temp:Boolean;
  31. begin
  32.   Result := false;
  33.   Kernel := LoadLibrary('kernel32');
  34.   if Kernel = 0 then Exit;
  35.   IsWOW64Process := GetProcAddress(Kernel, 'IsWow64Process');
  36.   if not Assigned(IsWow64Process) then Exit;
  37.   IsWOW64Process(GetCurrentProcess, Temp);
  38.   Result := Temp;
  39.   FreeLibrary(Kernel);
  40. end;
  41.  
  42. var ResName, FilePath, Params:String;
  43.   Source:TResourceStream;Dest:TStream;
  44.   I:Integer;StartupInfo:TStartupInfo;
  45.   ProcessInfo:TProcessInformation;
  46.  
  47. begin
  48.   if Is64Process then begin
  49.     ResName := Res64;
  50.   end
  51.   else begin
  52.     ResName := Res32;
  53.   end;
  54.   FilePath := GetTempFolder + FileName;
  55.   Source := TResourceStream.Create(hInstance, ResName, RT_RCDATA);
  56.   Dest := TFileStream.Create(FilePath, fmCreate);
  57.   Dest.CopyFrom(Source, 0);
  58.   Dest.Write(Source.Memory, Source.Size);
  59.   Dest.Free;
  60.   Source.Free;
  61.   Params := '"'+FilePath+'"';
  62.   for I := 1 to ParamCount do begin
  63.     Params := Params + ' ' + ParamStr(I);
  64.   end;
  65.   GetStartupInfo(StartupInfo);
  66.   if CreateProcess(PWideChar(FilePath),
  67.     PWideChar(Params),
  68.     nil,
  69.     nil,
  70.     false,
  71.     CREATE_UNICODE_ENVIRONMENT,
  72.     nil,
  73.     PWideChar(GetCurrentDir),
  74.     StartupInfo,
  75.     ProcessInfo) then begin
  76.       if ProcessInfo.hProcess <> 0 then begin
  77.         WaitForSingleObject(ProcessInfo.hProcess, INFINITE);
  78.       end;
  79.   end;
  80.   DeleteFile(FilePath);
  81. end.
  82.  
Prog.rc:
Code: Pascal  [Select][+][-]
  1. Prog32 RCDATA ..\Prog\Win32\Release\Prog.exe
  2. Prog64 RCDATA ..\Prog\Win64\Release\Prog.exe
  3.  
« Last Edit: June 02, 2020, 01:28:37 pm by Mr.Madguy »
DynamicData 3.0 is released!
Since now development is frozen - only optimization passes will be made at some point.
Lack of multiple inheritance turns it into abomination.

PascalDragon

  • Hero Member
  • *****
  • Posts: 1768
  • Compiler Developer
Re: Best approach to create a *.exe
« Reply #10 on: June 02, 2020, 01:19:29 pm »
For your own code that does not matter too much if at all, but when you try to add or change code or data to a signed program it will not execute at all in modern Windows.. (Ahhh the days of XP have long gone....) and since resources are data, same effect...

It will still run by default (it's probably possible to disable this using Local Security Policies). If it requires administrative access however you'll get a yellow dialog with an unknown signature giver instead of the blue dialog with a named signature giver.

Thaddy

  • Hero Member
  • *****
  • Posts: 10213
Re: Best approach to create a *.exe
« Reply #11 on: June 02, 2020, 01:47:47 pm »
It will still run by default
But only on your development machine .... :o
I am more like donkey than shrek

PascalDragon

  • Hero Member
  • *****
  • Posts: 1768
  • Compiler Developer
Re: Best approach to create a *.exe
« Reply #12 on: June 03, 2020, 09:21:57 am »
It will still run by default
But only on your development machine .... :o

As our executables at work are signed I just tested it for you. I created a signed executable (that requires administrative access) on my Windows 7 development machine. It runs with the blue UAC dialog. Now I modified an unrelated byte inside the executable. It still works, but now shows the yellow UAC dialog. Now I copy the modified executable to a Windows 10 test computer. And would you look at that... it still runs. Who would have thought? Right... Me. 8)

To prevent modified or unsigned executables from running one has to configure that explicitly.

Thaddy

  • Hero Member
  • *****
  • Posts: 10213
Re: Best approach to create a *.exe
« Reply #13 on: June 03, 2020, 10:11:25 am »
Yes. That is probably why I wrote that. I did not know that there are more scenario's.
Seems we have (in my case had, I do not have to work) well schooled systems guys (banking).
One note: we used an in-house certificate issuer during development and only signed the final binaries *after* development was in release state.
I am more like donkey than shrek

PascalDragon

  • Hero Member
  • *****
  • Posts: 1768
  • Compiler Developer
Re: Best approach to create a *.exe
« Reply #14 on: June 03, 2020, 10:43:35 am »
Yes. That is probably why I wrote that. I did not know that there are more scenario's.
Seems we have (in my case had, I do not have to work) well schooled systems guys (banking).

Well, the default settings of Windows definitely allow it. Microsoft would have another Vista if they'd block unsigned binaries or binaries with malformed signatures.

The problem with restricting it is that all binaries definitely need to be signed then, so that puts additional strain on the IT department. Though of course depending on the environment it might be a good idea to enable it (like in your case for banking).

One note: we used an in-house certificate issuer during development and only signed the final binaries *after* development was in release state.

We have an in-house signing computer that we also use for signing drivers and operating system images (Secure Boot).

 

TinyPortal © 2005-2018