I uploaded Patches to BugTracker for some of the components on the "Misc" tab to add BiDiMode and ParentBiDiMode properties. When BiDiMode:= bdLeftToRight, the button is on the Right as before. When BiDiMode:= bdRightToLeft, the button is on the Left.
The components are:
TCalcEdit
TDateEdit
TDirectoryEdit
TEditButton
TFileNameEdit
I also added BiDiMode and ParentBiDiMode properties to TFileListBox.