I followed the instructions with 2 exceptions. 1st is that I start my GDB server on port 2345, and 2nd is that I incidentally installed into other dir, so I had to adjust fpc.cfg accordingly:
#ifdef embedded
# Extended searchpath for units and other system dependent things on Embedded Targets
-FuC:\PRG\Lazarus\stm32x64\fpc/units/$fpctarget/$fpcsubarch
-FuC:\PRG\Lazarus\stm32x64\fpc/units/$fpctarget/$fpcsubarch/*
-FuC:\PRG\Lazarus\stm32x64\fpc/units/$fpctarget/$fpcsubarch/rtl
#else
# searchpath for units and other system dependent things
-FuC:\PRG\Lazarus\stm32x64\fpc/units/$fpctarget
-FuC:\PRG\Lazarus\stm32x64\fpc/units/$fpctarget/*
-FuC:\PRG\Lazarus\stm32x64\fpc/units/$fpctarget/rtl
#endif
I have then unpacked zip file to overwrite existing files, set IDE debugger settings as per screenshot, set project debugger backend to this new debugger settings, start debug server which is waiting on port 2345, put break point into my previous simplest example (your blupill sample project link is not working), and run it. I still get the same error.

If you think I should use 5432 port and use your exact dir then please say so and I will. I have also unplugged and reset my blue pill many times, and immediately after IDE debug error started STM32 CubeProgrammer to check if I can connect, and I can. I can also debug from PlatformIO, as in screenshot.
I am still experiencing issues with debugging on Windows, I have never used the cheap stlink clones on windows, so some more work is needed.
I am really sorry for the trouble. If you think that it could help I might try to burn Black Magic Probe firmware into my ST-Link v2 clone.
After reading your comments I suspect there is something not good between STM32Cube debug server and my ST-Link. I can debug with Platformio IDE and Mikroelektronika Pascal IDE, but with STM32Cube it seams like I am able to program but not debug. Screenshot attached. That may be the problem. It is late now so I will continue testing tomorrow.