You have couple of options here, use an overlayed transparent window that encompasses the panels and that way this window will receive the events for anything under it.
Or you can use a Tag flag of sorts to mark the panel in the moving state and then perform the mouse moves while monitor the mouse actions in the OnUserInput event of the TApplication instance.
With that, you can determine which object is moving the mouse etc. and then check the flags within the panel to see if one or more need moving.
Most people tag the controls to be moved and then activate another control on top that could be like transparent as the moving tool.