[SOLVED] Handling any attempts to close dynamically allocated console window

[flowCRANE]

I have a GUI application that can allocate a console using the Windows.AllocConsole function. How to detect any console window closing attempt (using a button, Ctrl+C, Alt+F4, etc.) and either ignore them or handle them in my own way, i.e. safely close the application? I did not try with the WndProc yet.

We are talking about an application that displays an SDL window (does not use LCL).

[Thaddy]

look at this:
https://stackoverflow.com/questions/1000223/how-do-i-handle-ctrlc-in-a-delphi-console-application

[flowCRANE]

SetConsoleCtrlHandler is used only to inform the application that after processing the signal by a handler(s), the console will be terminated anyway and the handler cannot do anything with it. I'm using this function only to block the Ctrl+C shortcut, which normally kills the console and the application:

Code: Pascal  [Select][+][-]

1. SetConsoleCtrlHandler(nil, True); // Ctrl+C will not kill the console

What is needed here is something that will inform Windows that the application is rejecting the request to close the console, so that the system does not close it. This is usually done by handling the WM_CLOSE message, so I will try subclassing the window procedure and handling this message.

[440bx]

so I will try subclassing the window procedure and handling this message.

that won't work because the code that handles the window messages is in another process.  Can't subclass that.

[flowCRANE]

Yeah, I was afraid of this. I've tested this and the console window cannot be subclassed, I can't register custom window procedure callback in any way.

So maybe I'll ask differently — how can I detect an attempt to close the console and handle it so that the system doesn't close the console and kill the application process? This is what I need, to detect an attempt to close the console, block it so the system will do nothing, and inform the game process to decide what to do with this attempt — close the game or not.

Detecting an attempt to close and rejecting it is crucial because the game must correctly interrupt the main game loop and then finalize the resources, i.e. mainly save important data. I can easily do this by simply adding a custom event to the SDL queue, so that in the next game frame, in the loop processing SDL events, I just need to handle this specific event and interrupt the game loop normally, thanks to which finalization will be carried out. So it's not a problem at all.

The only problem is how to detect an attempt to close the console and reject it, so that the console lives on.


What is interesting, Lazarus itself can be build in the way that it will open the system console at startup, to print debug data. Does Lazarus somehow control this console, or is it not handled in any way and the IDE just terminates itself? I recently had Lazarus built this way, it opened the console at startup, and when I closed the console window, the IDE closed as well. But I didn't check what would happen if I had unsaved changes in an open project — Lazarus should ask whether to save the changes or not and not close if necessary.

[440bx]

Just in case... is this for Windows only or do you need the solution to be cross platform ?

[flowCRANE]

For now Windows only, so pure Win32 API is preferred.

[440bx]

Attached to this post you'll find a proof of concept program I wrote a few years ago to control the console from a GUI program.  It does most everything you want _except_ that I didn't put a control-c handler (that's easy and left as an exercise for you.)

Slowly and carefully read the source, it will look "strange" at first but, all the "strange" stuff is genuinely needed in order to control the console, remove anything and the console will get the best of you.  The code is tested on both 32 and 64bits.

I think that if you add a control-C handler to that code, it will be what you were looking for.

HTH.

[flowCRANE]

Thanks, but you are using custom window procedure — I can't, because the main window procedure is handled by the SDL.

[flowCRANE]

I will try to block Ctrl+C and Alt+F4 keystrokes using the low level keyboard hook — it should work.
More information on this topic can be found here — Disabling Shortcut Keys in Games.

[flowCRANE]

I found the solution — with low level keyboard hook it can be done, so the problem solved.

It is not known whether the game was launched from the disk or from the console, so first check whether you can connect the parent process to the console. There are many ways to detect whether a program has been launched from the console, but the simplest is to use the AttachConsole function and check whether the operation was successful:

Code: Pascal  [Select][+][-]

1. uses
2.   Windows;
3.  
4. {...}
5.  
6. if AttachConsole(ATTACH_PARENT_PROCESS) then
7.   // The program was launched from the console and managed to attach to it.

If it failed to attach to the console, it means that the program was either not launched from the console or the system was not able to attach it to it. If there is no console available, you need to create one — this is where the AllocConsole function comes in:

Code: Pascal  [Select][+][-]

1. if AllocConsole() then
2.   // We managed to create a console for the program.

If the console cannot be created, the logs should be sent to a file. Now you need to inform RTL that the program uses the console and it must update the handles, including: to standard output. Two lines are enough:

Code: Pascal  [Select][+][-]

1. IsConsole := True;
2. SysInitStdIO();

From now on, the process has a console at its disposal. To prevent the user from closing it and killing the process of our program, this option must be taken away from him. First, lock the button on the console window bar:

Code: Pascal  [Select][+][-]

1. var
2.   ConsoleHandle: HANDLE
3.   ConsoleMenu:   HMENU;
4.  
5. {...}
6.  
7. // Get the handle to the console window.
8. ConsoleHandle := GetConsoleWindow();
9.  
10. if ConsoleHandle <> 0 then
11. begin
12.   // Get the console window system menu handle.
13.   ConsoleMenu := GetSystemMenu(ConsoleHandle, False);
14.  
15.   if ConsoleMenu <> 0 then
16.     // Disable the close button.
17.     EnableMenuItem(GetSystemMenu(ConsoleHandle, False), SC_CLOSE, MF_BYCOMMAND or MF_DISABLED or MF_GRAYED);
18. end;

You can still close the console with two keyboard shortcuts — Ctrl+C and Alt+F4. So, you should install a low-level keyboard hook — you can listen in on and block all keyboard input, as well as ”eat” anything you want (including single keys and keyboard shortcuts). First, you need to define a function that processes the input (our hook):

Code: Pascal  [Select][+][-]

1. function ConsoleKeyboardHook(nCode: LongInt; wParam: WPARAM; lParam: LPARAM): LRESULT; stdcall;

This callback should be registered after successful allocation or connection to the console:

Code: Pascal  [Select][+][-]

1. const
2.   WH_KEYBOARD_LL = 13;
3. var
4.   ConsoleHook: HHOOK;
5.  
6. {...}
7.  
8. ConsoleHook := SetWindowsHookEx(WH_KEYBOARD_LL, @ConsoleKeyboardHook, GetModuleHandle(nil), 0);

Then you need to program the listening function:

Code: Pascal  [Select][+][-]

1. function ConsoleKeyboardHook(nCode: LongInt; wParam: WPARAM; lParam: LPARAM): LRESULT; stdcall;
2. type
3.   KBDLLHOOKSTRUCT = record
4.     vkCode:      DWORD;
5.     scanCode:    DWORD;
6.     flags:       DWORD;
7.     time:        DWORD;
8.     dwExtraInfo: ULONG_PTR;
9.   end;
10. var
11.   HookStruct: ^KBDLLHOOKSTRUCT absolute lParam;
12. begin
13.   if nCode = HC_ACTION then
14.     // If the console window is active, check the input, otherwise pass the data to the next hook.
15.     if GetForegroundWindow() = ConsoleHandle then
16.     begin
17.       // If "Alt+F4" or "Ctrl+C" was pressed, return "1" — the input will not reach the console window (it will be eaten).
18.       if (HookStruct^.vkCode = VK_F4) and (GetAsyncKeyState(VK_MENU)    and $8000 <> 0) then exit(1);
19.       if (HookStruct^.vkCode = VK_C)  and (GetAsyncKeyState(VK_CONTROL) and $8000 <> 0) then exit(1);
20.     end;
21.  
22.   // Pass data to the next hook (extremely important).
23.   Result := CallNextHookEx(ConsoleHook, nCode, wParam, lParam);
24. end;

From this moment on, the console will no longer be able to be closed, at least not in the normal way known to the user. Of course, the initialization of all this must go hand in hand with finalization. At the end of the program session, you need to clean up after yourself — unregister the keyboard hook, enable the system button for closing the console (if our program was attached to an existing one) or destroy the console (if it allocated the console itself). Manually destroying the console is not required, because the system will destroy it itself if our program was the last one attached to it.

Code: Pascal  [Select][+][-]

1. // Unregister keyboard hook.
2. UnhookWindowsHookEx(ConsoleHook);
3.  
4. // Enable close button on the console title bar.
5. ConsoleMenu := GetSystemMenu(ConsoleHandle, False);
6.  
7. if ConsoleMenu <> 0 then
8.   EnableMenuItem(GetSystemMenu(ConsoleHandle, False), SC_CLOSE, MF_BYCOMMAND or MF_ENABLED);
9.  
10. // Destroy the console (or detach it).
11. FreeConsole();

Well, that's pretty much it. In the case of my engine, after detecting an attempt to close the console, the hook function adds the event SDL_QUITEVENT to the SDL queue. In the next frame of the game, the event processing function will encounter it and, if necessary, terminate the main game loop, thanks to which the game will be able to finalize its operation (save data and clean up resources).

If we use the above in a standard window application (created in LCL), the hook's callback can in any way inform the application about an attempt to close the console window — fire it Application.Terminate, send it a WM_CLOSE message or anything else. Of course, you can also eat these shortcuts and not inform the application at all — this way, the console will be able to close only after closing the main window of the application (the system will close the console or disconnect the application from it).

PS: I declared the constant WH_KEYBOARD_LL and type KBDLLHOOKSTRUCT manually because they are not declared in the Windows module.

[440bx]

Just a thought here...

would you be amenable to adding the keyboard hook and ctrl-c handler to the code I posted ?... that would add some value to the example.

[flowCRANE]

Sure, no problem. I've implemented in your project all features mentioned — disabling close button in the console window and blocking the Ctrl+C and Alt+F4 shortcuts. If you don't need any of the features, remove their code from the project.

Some data declarations:

Code: Pascal  [Select][+][-]

1. const
2.   WH_KEYBOARD_LL = 13;
3.  
4. type
5.   PKBDLLHOOKSTRUCT = ^TKBDLLHOOKSTRUCT;
6.   TKBDLLHOOKSTRUCT  = record
7.     vkCode:      DWORD;
8.     scanCode:    DWORD;
9.     flags:       DWORD;
10.     time:        DWORD;
11.     dwExtraInfo: ULONG_PTR;
12.   end;
13.  
14. var
15.   ConsoleHandle : HANDLE;
16.   ConsoleMenu   : HMENU;
17.   ConsoleHook   : HHOOK;

The ConsoleCreate function now is the following:

Code: Pascal  [Select][+][-]

1. function ConsoleCreate: boolean;
2. begin
3.   result := AllocConsole();
4.  
5.   if not result then exit;
6.  
7.   { let the FPC RTL know there is a console                                   }
8.  
9.   {$ifdef FPC}
10.     StdInputHandle  := 0;
11.     StdOutputHandle := 0;
12.     StdErrorHandle  := 0;
13.     IsConsole       := True;
14.  
15.     SysInitStdIO;
16.  
17.     IsConsole       := False;
18.   {$endif}
19.  
20.   // BEGIN OF ADDED CODE.
21.   ConsoleHandle := GetConsoleWindow();
22.  
23.   if ConsoleHandle <> 0 then
24.   begin
25.     ConsoleHook := SetWindowsHookEx(WH_KEYBOARD_LL, @ConsoleKeyboardHook, GetModuleHandle(nil), 0);
26.     ConsoleMenu := GetSystemMenu(ConsoleHandle, False);
27.  
28.     if ConsoleMenu <> 0 then
29.       EnableMenuItem(GetSystemMenu(ConsoleHandle, False), SC_CLOSE, MF_BYCOMMAND or MF_DISABLED or MF_GRAYED);
30.   end;
31.   // END OF ADDED CODE.
32. end;

The ConsoleDestroy function not looks like this:

Code: Pascal  [Select][+][-]

1. procedure ConsoleDestroy;
2. begin
3.   // BEGIN OF ADDED CODE.
4.   if ConsoleHandle <> 0 then
5.   begin
6.     UnhookWindowsHookEx(ConsoleHook);
7.  
8.     if ConsoleMenu <> 0 then
9.       EnableMenuItem(GetSystemMenu(ConsoleHandle, False), SC_CLOSE, MF_BYCOMMAND or MF_ENABLED);
10.   end;
11.   // END OF ADDED CODE.
12.  
13.   { let the FPC RTL know there is no longer a console                         }
14.  
15.   {$ifdef FPC}
16.     SysFlushStdIO;
17.   {$endif}
18.  
19.   FreeConsole;
20.  
21.   {$ifdef FPC}
22.     SysInitStdIO;
23.   {$endif}
24. end;

And the hook callback function:

Code: Pascal  [Select][+][-]

1. function ConsoleKeyboardHook(nCode: LongInt; wParam: WPARAM; lParam: LPARAM): LRESULT; stdcall;
2. var
3.   HookStruct: PKBDLLHOOKSTRUCT absolute lParam;
4. begin
5.   if (nCode = HC_ACTION) and (GetForegroundWindow() = ConsoleHandle) then
6.   begin
7.     if (HookStruct^.vkCode = VK_F4) and (GetAsyncKeyState(VK_MENU)    and $8000 <> 0) then exit(1);
8.     if (HookStruct^.vkCode = VK_C)  and (GetAsyncKeyState(VK_CONTROL) and $8000 <> 0) then exit(1);
9.   end;
10.  
11.   Result := CallNextHookEx(ConsoleHook, nCode, wParam, lParam);
12. end;

Project is in the attachment.

[440bx]

Excellent... thank you!

[flowCRANE]

Your welcome. Note that very important in the case of hooking the console window is to register a hook callback using the WH_KEYBOARD_LL flag (value 13) instead of WH_KEYBOARD (value 2). Using the second one, the hook will not work — input will not be blocked.