Okay, so a played a bit with it. When using a different style, like fusion, the problem is reduced but not entirely gone. Breeze uses a 1px wide margin around each control, which ideally should be transparent. Fusion does not, so you don't have that problem as much. But it's not gone completely, for example TButton has rounded edges in Fusion, and it still shows the background color on the four corners.
But in any case, I would like to use Breeze for now. So I had a look at the Qt Designer. Here, on some controls it actually behaves the same when I simply change the background color (using style sheet). But I can make it look the way I want when I add "background-clip: padding;" in the style sheet.
So, is there any way to set this style sheet in Lazarus as well?