Rather than using the TListView.Selected property, try instead the TListView.GetItemAt() method, using the TPopupMenu.PopupPoint as the coordinates to search at.
However, I would not abort the PopupMenu in its own OnPopup event. Instead, I would set TPopupMenu.AutoPopup to false, and then call TPopupMenu.Popup() in the TListView.OnMouseUp or TListView.OnContextPopup event if TListView.GetItemAt() finds an item under the mouse.
Do be aware that the popup menu can be invoked by keyboard instead of mouse, in which case using TListView.Selected will make sense in that case.