Input Chinese using Microsoft Bopomofo (注音輸入法) in TEdit

[momigo]

Problem:
On Windows 11 using Lazarus 2.2.6, the TEdit control overwrites previous characters when inputting text using Microsoft Bopomofo (Zhuyin) input method and selecting from the candidate list.
(describe in Chinese: TEdit 中用注音輸入法, 只要有選字, 之前輸入的文字就會被所選的字覆蓋)

Steps to reproduce:

Create a form with a standard TEdit.

Input characters like "AAA".

Use Microsoft Bopomofo to select the next character (e.g., "有") from the candidate list manually using number keys or arrow keys.

After confirming, the previously entered text ("AAA") is unexpectedly overwritten by the selected character ("有").

Expected behavior:
The newly selected character should be inserted at the caret position without deleting or replacing previous text.

Additional observation:

If the user does not select from the candidate list and accepts the first suggestion directly, the behavior is correct.

This issue does not occur in TMemo. Only TEdit is affected.

Environment:

Lazarus 2.2.6

Windows 11 (64-bit)

System input: Microsoft Bopomofo

[momigo]

Is there any Taiwanese using Lazarus in Mandarin ?
這裡有台灣人使用Lazarus開發中文版系統嗎?

[Martin_fr]

I don't think I will be of much help... I only use Latin based languages (German/English).

I am not aware of any deliberate changes, yet I would recommend you try Lazarus 4.0 instead of 2.2.6.


I have tried some IME to see if I can get the error (but did not). I have Windows 10. I tried
- the smiley IME
- Japanese Hiragana (no I don't understand it / just got it installed)
- Arabic (no I don't understand it / just got it installed)

I have those installed to test SynEdit. All I can do though is blindly type in random chars...

Anyway none of them caused the Issue.
I used the edit for "Title" in the "Project Options" (I did not put an TEdit on a new form).


Does it matter if you enable/disable "use manifest" in the Project Options?
Or (with manifest on) change "ansi codepage is utf8"?

Does it make a different how you select the candidate?
- cursor up down
- number key 1 - 9
- write till only one entry is offered
?

You mentioned TEdit vs TMemo. Have you tried in a combobox (any combobox in the IDE options)?


You could search for
WM_IME
in  lcl\interfaces\win32\win32callback.inc

I couldn't find anything that looked suspicious. But I did not look in depth.
Note, that those messages must be offered to the control that receives them in the LCL, so for examlpe SynEdit can do its own handling.

If they are not handled then I would hope (but have not tested) that they go to default windowproc... (see "WinProcess")


There is the following. Which for some reason does not test if the control is a combobox

  {for ComboBox IME sends WM_IME_NOTIFY with WParam=WM_IME_ENDCOMPOSITION}
  if (Msg = WM_IME_NOTIFY) and (WPARAM=WM_IME_ENDCOMPOSITION) then
    WindowInfo^.IMEComposed:=True;

It affect the keyup. Maybe it needs something else for TEdit.


You can debug the IDE by
- Tools > Configure build Lazarus => add to "Custom options":  -gw3 -O-1
- Build and restart

Then
- open project: ide/lazarus.lpi
- run

If you edit any code, you must rebuild from the  "Tools" menu.

You can set breakpoints, and edit their properties
- break: uncheck the checkbox
- take snapshot: check the checkbox
Add watches for whatever you may need to inspect.

Those breakpoints will not interrupt, so when you test the IME the focus will not go to the debugger.

When you done typing in the IME, in the debugger, you can open menu: View > Debug Windows > History
- Switch to the first camera
And you can select any entry, and see what any watch was at that breakpoint.

That way you can see what happened while you were using the IME.

[momigo]

Thanks to Martin_fr for the suggestions and directions. This updated report includes findings based on Martin_fr's advice.

Problem:

On Windows 11 (Traditional Chinese environment), when using the Microsoft Bopomofo (Zhuyin) input method, pressing the arrow keys to navigate IME candidate lists and then pressing Enter causes previously typed text to be overwritten.

This issue occurs in multiple Lazarus controls, including:

TEdit

TComboBox

TDBGrid (cell editor)

The Object Inspector in the Lazarus IDE itself

The problem seems to be caused by incomplete handling of IME composition messages, possibly WM_IME_COMPOSITION or WM_IME_NOTIFY, leading to incorrect text selection or caret behavior after committing a character via Enter.

🟨 Steps to Reproduce:
On Windows 11, set your system locale to Traditional Chinese.

Use the built-in Microsoft Bopomofo IME.

Run a Lazarus application using any of these controls: TEdit, TComboBox, or TDBGrid.

Type a string like AAA in the control.

Start composing the next character using Zhuyin, then:

Use the Up/Down arrow keys to navigate the candidate list.

Press Enter to confirm the selected candidate.

Observe: the earlier input (AAA) is selected and overwritten by the newly committed character.

🟩 Expected Behavior:
The selected IME candidate should be inserted at the current caret position without affecting previously typed characters.

🟨 Additional Notes:
If the candidate is selected using number keys (1–9) instead of arrow keys, the problem does not occur.

If the IME auto-commits the first suggestion without explicit selection, the behavior is also correct.

TMemo is not affected, only single-line and editable controls are impacted.

🟪 Environment:
Lazarus versions tested:
✅ 2.2.6
✅ 4.0 — same issue persists in latest release

Platform: Windows 11 Pro (Traditional Chinese)

IME: Microsoft Bopomofo (注音輸入法)

DPI setting: 125%

Manifest / UTF-8 setting: Tried both enabled and disabled — no difference.

[Martin_fr]

Can you report this on the bug tracker (with the info from your last post), please.

The problem seems to be caused by incomplete handling of IME composition messages, possibly WM_IME_COMPOSITION or WM_IME_NOTIFY,

Do you know which ones?

Needs to be double checked, but normally TEdit forwards most messages to the DefaultWindowProc. Then it would all be handled by Windows itself. But maybe its the reverse. Something isn't filtered out and reaches the LCL KeyDown/Up.
But I have not checked that. (Too busy right now)


And, does the Source-editor in the IDE (SynEdit) handle them? (so maybe this can be used as reference.
There is a setting in the option (affecting the source editor only), would be good to know for the setting on/off.
Menu: Tools > Options
Page: Editor > General > Miscellaneous
Checkbox in the top section: "IME handled by System"

[Martin_fr]

And for someone like me, who doesn't speak/write/read Chinese....

So I installed this in a VM. I also installed the keyboard (the default "simple" seems to work).

Then in the statusbar, I switch from English to "Chinese Macao SAR", and after that I also switch from "English mode" to "Chinese mode" (the extra symbol that appears in the status bar).

Now if I type at random (TEdit, or SynEdit, or Microsoft Notepad, or the search edit in Microsoft explorer.exe) then I see Chinese chars.
But I do not get a drop down. I can not select from any list.

The chars are dotted underlined, so an IME is active. And continue typing does change the existing chars, rather than adding new. In fact even if I press enter, I can not add more chars...


However, I can actually get the bug.

Even though I have no dropdown, when I press enter, in Lazarus the entire edit gets selected (this does not happen in the search edit of the explorer).


So probably (not tested) for some reason the enter keystroke reaches the TEdit. Even though while the IME is active the OS shouldn't send them...

[chenyuchih]

I am a traditional Chinese user from Taiwan and use the same IME.

After my test, by default setup, the issue that momigo mentioned happens. However, if I set the TEdit property "AutoSelect" to "False", everything would be fine.

把 TEdit 裡面「AutoSelect」設為 False 即可正常操作。

Tested under Windows 10 with Lazarus 2.0.10.

David

[momigo]

I am a traditional Chinese user from Taiwan and use the same IME.

After my test, by default setup, the issue that momigo mentioned happens. However, if I set the TEdit property "AutoSelect" to "False", everything would be fine.

把 TEdit 裡面「AutoSelect」設為 False 即可正常操作。

Tested under Windows 10 with Lazarus 2.0.10.

David

Thank you David, that's a very helpful observation.
I tested it as well, and yes — setting AutoSelect := False in TEdit does prevent the overwrite issue when using Zhuyin and selecting candidates via arrow keys + Enter.

This confirms that the problem is likely due to a combination of IME commit behavior and TEdit's default auto-selection mechanism.

However, since many controls (like TComboBox, TDBGrid, and the Object Inspector) do not expose AutoSelect or rely on default behavior, I believe this still deserves a core-level review or workaround in LCL for IME compatibility.
Dear David ,謝謝您提供的線索,
AutoSelect := False 確實有效
只是TDBGrid 之類的其他元件無法設定 AutoSelect, 這個問題可能還是需要再LCL中解決

[momigo]

Can you report this on the bug tracker (with the info from your last post), please.

The problem seems to be caused by incomplete handling of IME composition messages, possibly WM_IME_COMPOSITION or WM_IME_NOTIFY,

Do you know which ones?

Needs to be double checked, but normally TEdit forwards most messages to the DefaultWindowProc. Then it would all be handled by Windows itself. But maybe its the reverse. Something isn't filtered out and reaches the LCL KeyDown/Up.
But I have not checked that. (Too busy right now)

And, does the Source-editor in the IDE (SynEdit) handle them? (so maybe this can be used as reference.
There is a setting in the option (affecting the source editor only), would be good to know for the setting on/off.
Menu: Tools > Options
Page: Editor > General > Miscellaneous
Checkbox in the top section: "IME handled by System"

Hi Martin, thank you for your helpful guidance earlier — especially the suggestions about message handling and testing with SynEdit.

I've done additional testing and would like to summarize the updated results:

The issue (text being overwritten after IME candidate selection via arrow keys + Enter) occurs in:

TEdit

TComboBox

TDBGrid (when editing a cell)

The Object Inspector inside the Lazarus IDE

The issue does not occur in:

TMemo

SynEdit (tested via the IDE’s Source Editor)

I also tested the SynEdit option “IME handled by System” (under Tools > Options > Editor > General > Miscellaneous), and it makes no difference — the behavior is consistent either way.

Additionally, setting AutoSelect := False on TEdit does prevent the overwrite issue.

I tested both with and without manifest and UTF-8 codepage enabled. These settings do not affect the outcome.

I have now confirmed that the issue still exists in Lazarus 4.0, so it's not limited to version 2.2.6.

At the moment, I don’t have time to study the Lazarus source code, so I’m only reporting what I’ve observed during testing. I’ll report this to the bug tracker as you suggested. Thanks again for pointing me in the right direction.

[n7800]

Thank you for creating the report: https://gitlab.com/freepascal.org/lazarus/lazarus/-/issues/41755

Please add a link to it next time, as well as in the report on this topic. There is a lot of information here that will also be useful to developers.

[momigo]

Thank you for creating the report: https://gitlab.com/freepascal.org/lazarus/lazarus/-/issues/41755

Please add a link to it next time, as well as in the report on this topic. There is a lot of information here that will also be useful to developers.

Thank you n7800