The button click should just call Close (for the form) which, if it's the main form, will close the application. That will in turn fire the OnClose event: if you re-Click the button there you are caught in an infinite loop, so don't do that.
When the application is closed externally (e.g. by the methods you cite), the OnClose event will be fired, just as if you had called Close, so there is no need (or shouldn't be) to call the button's OnClick handler.