I figured out what's causing this. It happens when lazarus is built with qt5. I could fix the behaviour in OP from Settings > Environment > Form Editor > uncheck Show designer hints.
This must be a bug because what I noticed is that it happens with any hints (tooltips) only using qt5. For example: hovering over any text, component or whatever that shows a hint, makes lazarus "start" a new window in the taskbar with the hint text, while still showing it in a tooltip. Weird.