I used to feel this way, until someone pointed out that toggling "Show Component Tree" collapses it down to a drop-down list control. This reclaims the space on the Object Inspector, and can be restored by checked "Show Component Tree" on the context menu when you really need it.