I downloaded and tested your program. I confirm it happens on my Ubuntu Mate 64-bit.
Without looking into your code, nothing much we can say. Can you share your code? A stripped-down version or a short demo is okay.
I ever have mouse event issue which I think is similar to yours. If I'm not wrong, I fixed (?) it by adding some more events to detect the mouse behavior.