Recent

Author Topic: Mouse and key input bug  (Read 1806 times)

xinyiman

  • Hero Member
  • *****
  • Posts: 1995
    • Lazarus and Free Pascal italian community
Mouse and key input bug
« on: June 14, 2019, 06:33:09 pm »
Hi guys, I have a problem with the package to manage the mouse and keyboard. I created a small example. If you try it you will understand what I mean. You need to mousedown in memo1 and then press the button. If the check is flagged it should write 'abcd' in the memo, if it is not flagged it should write '1234'. Both on Ubuntu and on Win10 it only works when I write numbers. Does anyone know what I'm wrong or if there's a bug ?! Thank you
Win10, Ubuntu and Mac
Lazarus: 2.1.0
FPC: 3.3.1

JimD

  • Jr. Member
  • **
  • Posts: 62
Re: Mouse and key input bug
« Reply #1 on: June 14, 2019, 10:40:00 pm »
MouseAndKeyInput uses the VK codes defined in LCLType.
Your example works if you use caps that correlate to VK_A etc.

Code: Pascal  [Select]
  1. KeyInput.Press('ABCD')

You can send lowercase as follows;

Code: Pascal  [Select]
  1. KeyInput.Apply([ssShift]);
  2. KeyInput.Press('ABCD');
  3. KeyInput.UnApply([ssShift]);

xinyiman

  • Hero Member
  • *****
  • Posts: 1995
    • Lazarus and Free Pascal italian community
Re: Mouse and key input bug
« Reply #2 on: June 15, 2019, 01:36:49 pm »
What you told me to do doesn't work. It's not a case of small or capital letters, but of the fact that it just doesn't work. Have you tried to compile the example with lazarus 2.1.0? You'll see that it doesn't work.

My need is to read a string from a database and type it from the keyboard.
Win10, Ubuntu and Mac
Lazarus: 2.1.0
FPC: 3.3.1

Handoko

  • Hero Member
  • *****
  • Posts: 3237
  • My goal: build my own game engine using Lazarus
Re: Mouse and key input bug
« Reply #3 on: June 15, 2019, 03:49:48 pm »
I tested the code, I can reproduce the issue and I believe it is a bug. I really want to help to solve this issue but unfortunately I am busy at this weekend. I'll be back some days later if it hasn't been solved.

What I found, providing alphabets will give the results of numeric keys:
- 'A' ---> '1'
- 'B' ---> '2'
- 'C' ---> '3'
- 'D' ---> '4'
....

Hope someone will come to help.

xinyiman

  • Hero Member
  • *****
  • Posts: 1995
    • Lazarus and Free Pascal italian community
Re: Mouse and key input bug
« Reply #4 on: June 15, 2019, 10:21:16 pm »
I tested the code, I can reproduce the issue and I believe it is a bug. I really want to help to solve this issue but unfortunately I am busy at this weekend. I'll be back some days later if it hasn't been solved.

What I found, providing alphabets will give the results of numeric keys:
- 'A' ---> '1'
- 'B' ---> '2'
- 'C' ---> '3'
- 'D' ---> '4'
....

Hope someone will come to help.

In fact I don't understand why there is a function that says it allows you to pass the string to it and then it doesn't work. The only way to correctly parameterize the press is through the virtual key codes. I started writing a string to virtual key translator, but I can't find many characters. What follows is the point I arrived at. Who helps me complete it?

Code: Pascal  [Select]
  1. unit uExtendVK;
  2.  
  3. {$mode objfpc}{$H+}
  4.  
  5. interface
  6.  
  7. uses
  8.   Classes, SysUtils, Forms;
  9.  
  10. type
  11.  
  12.     { TExtendVK }
  13.  
  14.     TExtendVK = class
  15.       constructor Create;
  16.       destructor Free;
  17.       function StringToVKControl(valore: string; var errore: string): boolean;
  18.       procedure StringToVKSend(valore: string;
  19.         sleep_millisecond_between_characters: integer);
  20.     private
  21.            procedure MyPress(key: word; maiuscolo: boolean;
  22.              sleep_millisecond_between_characters: integer);
  23.     end;
  24.  
  25. implementation
  26. uses
  27.   MouseAndKeyInput, LCLType;
  28.  
  29. { TExtendVK }
  30.  
  31. constructor TExtendVK.Create;
  32. begin
  33.  
  34. end;
  35.  
  36. destructor TExtendVK.Free;
  37. begin
  38.  
  39. end;
  40.  
  41. function TExtendVK.StringToVKControl(valore: string; var errore: string): boolean;
  42. var
  43.   ret: boolean;
  44.   i: integer;
  45. begin
  46.      ret:=false;
  47.      errore:='';
  48.      for i:=1 to Length(valore) do
  49.      begin
  50.  
  51.        Case valore[i] of
  52.         'q' : ;
  53.         'w' : ;
  54.         'e' : ;
  55.         'r' : ;
  56.         't' : ;
  57.         'y' : ;
  58.         'u' : ;
  59.         'i' : ;
  60.         'o' : ;
  61.         'p' : ;
  62.         'a' : ;
  63.         's' : ;
  64.         'd' : ;
  65.         'f' : ;
  66.         'g' : ;
  67.         'h' : ;
  68.         'j' : ;
  69.         'k' : ;
  70.         'l' : ;
  71.         'z' : ;
  72.         'x' : ;
  73.         'c' : ;
  74.         'v' : ;
  75.         'b' : ;
  76.         'n' : ;
  77.         'm' : ;
  78.  
  79.         'Q' : ;
  80.         'W' : ;
  81.         'E' : ;
  82.         'R' : ;
  83.         'T' : ;
  84.         'Y' : ;
  85.         'U' : ;
  86.         'I' : ;
  87.         'O' : ;
  88.         'P' : ;
  89.         'A' : ;
  90.         'S' : ;
  91.         'D' : ;
  92.         'F' : ;
  93.         'G' : ;
  94.         'H' : ;
  95.         'J' : ;
  96.         'K' : ;
  97.         'L' : ;
  98.         'Z' : ;
  99.         'X' : ;
  100.         'C' : ;
  101.         'V' : ;
  102.         'B' : ;
  103.         'N' : ;
  104.         'M' : ;
  105.  
  106.         '0' : ;
  107.         '1' : ;
  108.         '2' : ;
  109.         '3' : ;
  110.         '4' : ;
  111.         '5' : ;
  112.         '6' : ;
  113.         '7' : ;
  114.         '8' : ;
  115.         '9' : ;
  116.  
  117.         '(' : ;
  118.         ')' : ;
  119.         '=' : ;
  120.         '?' : ;
  121.         '^' : ;
  122.         '\' : ;
  123.         '|' : ;
  124.         '!' : ;
  125.         '"' : ;
  126.         //'£' : ;
  127.  
  128.         '$' : ;
  129.         '%' : ;
  130.         '&' : ;
  131.         '/' : ;
  132.         //'è' : ;
  133.         //'é' : ;
  134.         '*' : ;
  135.         '+' : ;
  136.         //'ç' : ;
  137.         //'ò' : ;
  138.  
  139.         '@' : ;
  140.         //'°' : ;
  141.         //'à' : ;
  142.         '#' : ;
  143.         //'ù' : ;
  144.         '-' : ;
  145.         '_' : ;
  146.         '<' : ;
  147.         '>' : ;
  148.         '[' : ;
  149.         ']' : ;
  150.         '{' : ;
  151.         '}' : ;
  152.  
  153.        else
  154.  
  155.          Case ORD(valore[i]) of
  156.           8 : ; //spazio
  157.           9 : ; //tab
  158.           10: ; //new line
  159.           13: ; //carriage return
  160.          else
  161.            ret:=true; //c'è un carattere che provoca errore
  162.          end;
  163.  
  164.        end;
  165.  
  166.      end;
  167.      result:=ret;
  168. end;
  169.  
  170. procedure TExtendVK.StringToVKSend(valore: string; sleep_millisecond_between_characters: integer);
  171. var
  172.   ret: boolean;
  173.   i: integer;
  174. begin
  175.      ret:=false;
  176.      for i:=1 to Length(valore) do
  177.      begin
  178.  
  179.        Case valore[i] of
  180.         'q' : MyPress(VK_Q,false, sleep_millisecond_between_characters);
  181.         'w' : MyPress(VK_W,false, sleep_millisecond_between_characters);
  182.         'e' : MyPress(VK_E,false, sleep_millisecond_between_characters);
  183.         'r' : MyPress(VK_R,false, sleep_millisecond_between_characters);
  184.         't' : MyPress(VK_T,false, sleep_millisecond_between_characters);
  185.         'y' : MyPress(VK_Y,false, sleep_millisecond_between_characters);
  186.         'u' : MyPress(VK_U,false, sleep_millisecond_between_characters);
  187.         'i' : MyPress(VK_I,false, sleep_millisecond_between_characters);
  188.         'o' : MyPress(VK_O,false, sleep_millisecond_between_characters);
  189.         'p' : MyPress(VK_P,false, sleep_millisecond_between_characters);
  190.         'a' : MyPress(VK_A,false, sleep_millisecond_between_characters);
  191.         's' : MyPress(VK_S,false, sleep_millisecond_between_characters);
  192.         'd' : MyPress(VK_D,false, sleep_millisecond_between_characters);
  193.         'f' : MyPress(VK_F,false, sleep_millisecond_between_characters);
  194.         'g' : MyPress(VK_G,false, sleep_millisecond_between_characters);
  195.         'h' : MyPress(VK_H,false, sleep_millisecond_between_characters);
  196.         'j' : MyPress(VK_J,false, sleep_millisecond_between_characters);
  197.         'k' : MyPress(VK_K,false, sleep_millisecond_between_characters);
  198.         'l' : MyPress(VK_L,false, sleep_millisecond_between_characters);
  199.         'z' : MyPress(VK_Z,false, sleep_millisecond_between_characters);
  200.         'x' : MyPress(VK_X,false, sleep_millisecond_between_characters);
  201.         'c' : MyPress(VK_C,false, sleep_millisecond_between_characters);
  202.         'v' : MyPress(VK_V,false, sleep_millisecond_between_characters);
  203.         'b' : MyPress(VK_B,false, sleep_millisecond_between_characters);
  204.         'n' : MyPress(VK_N,false, sleep_millisecond_between_characters);
  205.         'm' : MyPress(VK_M,false, sleep_millisecond_between_characters);
  206.  
  207.         'Q' : MyPress(VK_Q,true, sleep_millisecond_between_characters);
  208.         'W' : MyPress(VK_W,true, sleep_millisecond_between_characters);
  209.         'E' : MyPress(VK_E,true, sleep_millisecond_between_characters);
  210.         'R' : MyPress(VK_R,true, sleep_millisecond_between_characters);
  211.         'T' : MyPress(VK_T,true, sleep_millisecond_between_characters);
  212.         'Y' : MyPress(VK_Y,true, sleep_millisecond_between_characters);
  213.         'U' : MyPress(VK_U,true, sleep_millisecond_between_characters);
  214.         'I' : MyPress(VK_I,true, sleep_millisecond_between_characters);
  215.         'O' : MyPress(VK_O,true, sleep_millisecond_between_characters);
  216.         'P' : MyPress(VK_P,true, sleep_millisecond_between_characters);
  217.         'A' : MyPress(VK_A,true, sleep_millisecond_between_characters);
  218.         'S' : MyPress(VK_S,true, sleep_millisecond_between_characters);
  219.         'D' : MyPress(VK_D,true, sleep_millisecond_between_characters);
  220.         'F' : MyPress(VK_F,true, sleep_millisecond_between_characters);
  221.         'G' : MyPress(VK_G,true, sleep_millisecond_between_characters);
  222.         'H' : MyPress(VK_H,true, sleep_millisecond_between_characters);
  223.         'J' : MyPress(VK_J,true, sleep_millisecond_between_characters);
  224.         'K' : MyPress(VK_K,true, sleep_millisecond_between_characters);
  225.         'L' : MyPress(VK_L,true, sleep_millisecond_between_characters);
  226.         'Z' : MyPress(VK_Z,true, sleep_millisecond_between_characters);
  227.         'X' : MyPress(VK_X,true, sleep_millisecond_between_characters);
  228.         'C' : MyPress(VK_C,true, sleep_millisecond_between_characters);
  229.         'V' : MyPress(VK_V,true, sleep_millisecond_between_characters);
  230.         'B' : MyPress(VK_B,true, sleep_millisecond_between_characters);
  231.         'N' : MyPress(VK_N,true, sleep_millisecond_between_characters);
  232.         'M' : MyPress(VK_M,true, sleep_millisecond_between_characters);
  233.  
  234.         '0' : MyPress(VK_0,false, sleep_millisecond_between_characters);
  235.         '1' : MyPress(VK_1,false, sleep_millisecond_between_characters);
  236.         '2' : MyPress(VK_2,false, sleep_millisecond_between_characters);
  237.         '3' : MyPress(VK_3,false, sleep_millisecond_between_characters);
  238.         '4' : MyPress(VK_4,false, sleep_millisecond_between_characters);
  239.         '5' : MyPress(VK_5,false, sleep_millisecond_between_characters);
  240.         '6' : MyPress(VK_6,false, sleep_millisecond_between_characters);
  241.         '7' : MyPress(VK_7,false, sleep_millisecond_between_characters);
  242.         '8' : MyPress(VK_8,false, sleep_millisecond_between_characters);
  243.         '9' : MyPress(VK_9,false, sleep_millisecond_between_characters);
  244.  
  245.         ' ' : MyPress(VK_SPACE,true, sleep_millisecond_between_characters); //spazio ;
  246.  
  247.         '|' : MyPress(VK_LCL_BACKSLASH,true, sleep_millisecond_between_characters); //spazio ;
  248.         '\' : MyPress(VK_LCL_BACKSLASH,false, sleep_millisecond_between_characters); //spazio ;
  249.  
  250.         '=' : MyPress(VK_0,true, sleep_millisecond_between_characters);
  251.         '!' : MyPress(VK_1,true, sleep_millisecond_between_characters);
  252.         '"' : MyPress(VK_2,true, sleep_millisecond_between_characters);
  253.         //'£' : MyPress(VK_3,true, sleep_millisecond_between_characters);
  254.         '$' : MyPress(VK_4,true, sleep_millisecond_between_characters);
  255.         '%' : MyPress(VK_5,true, sleep_millisecond_between_characters);
  256.         '&' : MyPress(VK_6,true, sleep_millisecond_between_characters);
  257.         '/' : MyPress(VK_7,true, sleep_millisecond_between_characters);
  258.         '(' : MyPress(VK_8,true, sleep_millisecond_between_characters);
  259.         ')' : MyPress(VK_9,true, sleep_millisecond_between_characters);
  260.         '''' : MyPress(VK_LCL_QUOTE,false, sleep_millisecond_between_characters);
  261.  
  262.         ',' : MyPress(VK_LCL_COMMA,false, sleep_millisecond_between_characters);
  263.         ';' : MyPress(VK_LCL_COMMA,true, sleep_millisecond_between_characters);
  264.  
  265.        else
  266.  
  267.          Case ORD(valore[i]) of
  268.           8 : MyPress(VK_SPACE,true, sleep_millisecond_between_characters); //spazio
  269.           9 : MyPress(VK_TAB,true, sleep_millisecond_between_characters); //tab
  270.           10: ; //new line
  271.           13: MyPress(VK_RETURN,true, sleep_millisecond_between_characters); //carriage return
  272.          else
  273.            //non faccio nulla
  274.          end;
  275.  
  276.        end;
  277.  
  278.      end;
  279. end;
  280.  
  281. procedure TExtendVK.MyPress(key: word; maiuscolo: boolean;
  282.   sleep_millisecond_between_characters: integer);
  283. begin
  284.  
  285.     if maiuscolo then
  286.        KeyInput.Apply([ssShift]);
  287.  
  288.     KeyInput.Press(key);
  289.  
  290.     if maiuscolo then
  291.        KeyInput.UnApply([ssShift]);
  292.  
  293.     if sleep_millisecond_between_characters>0 then
  294.     begin
  295.       Sleep(sleep_millisecond_between_characters);
  296.       Application.ProcessMessages;
  297.     end;
  298.  
  299. end;
  300.  
  301. end.
  302.  
Win10, Ubuntu and Mac
Lazarus: 2.1.0
FPC: 3.3.1

engkin

  • Hero Member
  • *****
  • Posts: 2513
Re: Mouse and key input bug
« Reply #5 on: June 15, 2019, 10:59:20 pm »
You slightly misunderstood Virtual Keys. They represent the physical keys on a keyboard. To get characters you need to "translate" these keys. On Windows, use something like MapVirtualKeyEx or VkKeyScanEx, where it takes the layout of the keyboard into consideration to produce the correct character. If you need more than one language, you'll have to use more than one layout to find the correct one. I don't know if there is an easier way.

Remember, the same virtual key -like VK_A- maps to many different characters, even in the same layout -A or a-. It is one-to-many relation.

You don't mention details, but you might be able to use messages instead.

Edit:
I just saw your signature (Ubuntu and Mac). My post is mostly irrelevant, sorry.
« Last Edit: June 15, 2019, 11:03:38 pm by engkin »

xinyiman

  • Hero Member
  • *****
  • Posts: 1995
    • Lazarus and Free Pascal italian community
Re: Mouse and key input bug
« Reply #6 on: June 16, 2019, 10:13:55 am »
Thanks anyway for the thought. If I wish I can also use a windows machine for the current project. So a small example of how you would deal with the problem might help me.
Win10, Ubuntu and Mac
Lazarus: 2.1.0
FPC: 3.3.1

engkin

  • Hero Member
  • *****
  • Posts: 2513
Re: Mouse and key input bug
« Reply #7 on: June 16, 2019, 08:00:46 pm »
Initially I started as I mentioned in my previous post, but then I remembered Alt Codes. Using numpad you can press and hold Alt key and type + then a number on your numpad, when you release the Alt key you get a character. You practically can use KeyInput to do the same. Probably other OSes should have something similar, you need to research that.

Anyway, as I moved to using Alt Codes, I discovered that Windows SendInput can accept Unicode if used with KEYEVENTF_UNICODE flag, here is the code:
Code: Pascal  [Select]
  1. uses
  2.   JwaWinUser, ... ;
  3.  
  4. ..
  5.  
  6. procedure SendStringInput(AStr: String);
  7. var
  8.   Input: TInput;
  9.   uStr: UnicodeString;
  10.   i: Integer;
  11. begin
  12.   FillChar(Input, SizeOf(Input), 0);
  13.   Input.type_ := INPUT_KEYBOARD;
  14.  
  15.   { Covert to Unicode UTF16 }
  16.   uStr := UnicodeString(AStr);
  17.   for i := 1 to Length(uStr) do
  18.   begin
  19.     { Keydown }
  20.     Input.ki.dwFlags := KEYEVENTF_UNICODE;
  21.     Input.ki.wScan := Word(uStr[i]);
  22.     SendInput(1, @Input, SizeOf(Input));
  23.  
  24.     { I don't think Keyup is needed
  25.     Input.ki.dwFlags := KEYEVENTF_UNICODE or KEYEVENTF_KEYUP;
  26.     SendInput(1, @Input, SizeOf(Input));//}
  27.   end;
  28. end;

Just be aware that the user can interfere with MouseAndKeyInput or SendInput by moving the mouse or pressing Shift ..etc

1st Edit:
Forgot to mention, since wScan is of type Word, you can only use the BMP of Unicode, no emojis for you.  :( sorry.

1nd Edit:
Yes you can.  without any change to the code. :D
« Last Edit: June 16, 2019, 08:21:17 pm by engkin »

Handoko

  • Hero Member
  • *****
  • Posts: 3237
  • My goal: build my own game engine using Lazarus
Re: Mouse and key input bug
« Reply #8 on: June 20, 2019, 07:37:50 pm »
@xinyiman

Have you solved the problem?

xinyiman

  • Hero Member
  • *****
  • Posts: 1995
    • Lazarus and Free Pascal italian community
Re: Mouse and key input bug
« Reply #9 on: June 22, 2019, 08:53:47 am »
On linux? No!
Win10, Ubuntu and Mac
Lazarus: 2.1.0
FPC: 3.3.1

xinyiman

  • Hero Member
  • *****
  • Posts: 1995
    • Lazarus and Free Pascal italian community
Re: Mouse and key input bug
« Reply #10 on: June 22, 2019, 08:59:49 am »
Initially I started as I mentioned in my previous post, but then I remembered Alt Codes. Using numpad you can press and hold Alt key and type + then a number on your numpad, when you release the Alt key you get a character. You practically can use KeyInput to do the same. Probably other OSes should have something similar, you need to research that.

Anyway, as I moved to using Alt Codes, I discovered that Windows SendInput can accept Unicode if used with KEYEVENTF_UNICODE flag, here is the code:
Code: Pascal  [Select]
  1. uses
  2.   JwaWinUser, ... ;
  3.  
  4. ..
  5.  
  6. procedure SendStringInput(AStr: String);
  7. var
  8.   Input: TInput;
  9.   uStr: UnicodeString;
  10.   i: Integer;
  11. begin
  12.   FillChar(Input, SizeOf(Input), 0);
  13.   Input.type_ := INPUT_KEYBOARD;
  14.  
  15.   { Covert to Unicode UTF16 }
  16.   uStr := UnicodeString(AStr);
  17.   for i := 1 to Length(uStr) do
  18.   begin
  19.     { Keydown }
  20.     Input.ki.dwFlags := KEYEVENTF_UNICODE;
  21.     Input.ki.wScan := Word(uStr[i]);
  22.     SendInput(1, @Input, SizeOf(Input));
  23.  
  24.     { I don't think Keyup is needed
  25.     Input.ki.dwFlags := KEYEVENTF_UNICODE or KEYEVENTF_KEYUP;
  26.     SendInput(1, @Input, SizeOf(Input));//}
  27.   end;
  28. end;

Just be aware that the user can interfere with MouseAndKeyInput or SendInput by moving the mouse or pressing Shift ..etc

1st Edit:
Forgot to mention, since wScan is of type Word, you can only use the BMP of Unicode, no emojis for you.  :( sorry.

1nd Edit:
Yes you can.  without any change to the code. :D
I test this code, but not press any key
Win10, Ubuntu and Mac
Lazarus: 2.1.0
FPC: 3.3.1

Handoko

  • Hero Member
  • *****
  • Posts: 3237
  • My goal: build my own game engine using Lazarus
Re: Mouse and key input bug
« Reply #11 on: June 22, 2019, 01:16:54 pm »
After some inspection, I found that  KeyInput.Press only works with upper case alphabets does not work with lower case alphabets. So this below works on my test:

Code: Pascal  [Select]
  1.      KeyInput.Press(UpperCase('abcd'));
« Last Edit: June 22, 2019, 01:21:27 pm by Handoko »

engkin

  • Hero Member
  • *****
  • Posts: 2513
Re: Mouse and key input bug
« Reply #12 on: June 22, 2019, 01:38:00 pm »
After some inspection, I found that  KeyInput.Press only works with upper case alphabets. So this below works on my test:

Code: Pascal  [Select]
  1.      KeyInput.Press(UpperCase('abcd'));
It works with Virtual Keys. The numerical values for Virtual Keys VK_A to VK_Z map to the the ASCII values for upper case letters. For instance VK_A = 65 and Ord('A') is also 65

Virtual Keys map to keyboard keys, like VK_F1 = 112 is F1 button
As you noticed, this value is *not* going to give small letter p, Ord('p') = 112.

Handoko

  • Hero Member
  • *****
  • Posts: 3237
  • My goal: build my own game engine using Lazarus
Re: Mouse and key input bug
« Reply #13 on: June 22, 2019, 02:47:13 pm »
Yes, you're correct. I have modified my previous post:

"KeyInput.Press does not work with lower case alphabets."

SymbolicFrank

  • Hero Member
  • *****
  • Posts: 635
Re: Mouse and key input bug
« Reply #14 on: June 22, 2019, 05:30:01 pm »
The only thing the keyboard does is send a keycode to the computer when a key is pressed or released. That's it. Even the state of the LEDs on the keyboard for the shift-states are directly controlled by the PC and not the keyboard. Everything else is the PC comparing the current state of the keys to some mapping, like the type and language of the keyboard, and spitting out what it thinks that current keypress should represent. And there's lots of different mappings.