I think r8315 should be fairly complete now. If there's still a bug somewhere, let me know.
The reason why the dialog could not be close by clicking the OK or Cancel button is not a fault of the buttons, but the fact that you changed the stringgrid cells in the OnPrepareCanvas event. This event belongs to the grid's drawing code, it is called immediately before the specified cell is painted. If you change the grid cell text here this triggers another painting cycle, and so on. I don't see that a stringgrid must be used here because it just causes double storage of the dot matrix which is already stored in FTmpDotRows. Therefore, I returned to the DrawGrid of my previous code which has less overhead. It reads the bits from FTmpDotRows and paints a cell black when the correspondig bit is set. I also added a MouseDown handler to toggle the bits. (I also replaced the integer helper methods for bit access by methods of the form; I think using the helpers is an unnecessary limitation on the usable FPC version).
I am rather sure that this also fixed the issue that the changes were not applied to the LCD's chardefs. Nevertheless, I had to debug the code for restoring the original chardefs when Cancel was pressed - this did not seem to work. Finally I found that there was code which did not really "copy" a DotRows array, but just the pointer to it, and I had to introduce a function CopyDotRows to create a real, independent copy.
The edAddReplace control was removed because it is needed only for the "Add" case (in the "Restore" case the character can be retrieved from the charselector combobox). When you now select "Add" (which I renamed to "Add as...") an input box opens first to query the name of the new character.
And I used control anchoring to avoid the position calculations in FormActivate.