Mouse button status outside the form (fullscreen) on Rasbian/arm

[xardomain]

Hi All, any pointer to find info about mouse buttons status on the screen?

I need a Linux/arm solution, it easy to get the mouse position but not the status of the buttons.
I need to acquire mouse position and button status and send to another raspberry, thanks.

I don't need to emulate it (not here), I would just to have a device(a pi for example) that will tell me where the mouse is on its screen and when the user presses the one of the three button on the mouse. The data could be passed through network or even serial.

At worst I could use a form that is the size of the screen and borderless (it should work I guess) but maybe there is a better solution.

The screen is not really used, so even a full blown empty form will do but...

TIA

Giuseppe

[KodeZwerg]

I am unsure about what you actual try to do but I suggest that you "uses mouse" and look into this unit. It offer a lot, I bet also things that you searching for.
It is unknown to me how far it is compatible to your target.

[xardomain]

>I am unsure about what you actual try to do but I suggest that you "uses mouse" and look into this unit. It offer a lot, I bet >also things that you searching for.
>It is unknown to me how far it is compatible to your target.

I can use mouse for the mouse position, but I am unable to detect mouse clicks. This is the reason of the request. I am unable to find this information/examples. I can find the solution for Windows, but I would  like it to be solved for Raspbian on PI, or even better cross platform.


I need to create a dual headed computer(like a PI4), one monitor, one primary keyboard and a primary mouse are the real hw, normal stuff.

On the same computer, but on the second monitor, I will have the output of a frame grabber "showing" the monitor of a target(another) computer.

I would like to setup a second keyboard and a second mouse on the dual headed computer, completely independent from the target but also independent from the primary keyboard and the mouse, they need not to interfere each other.

Connecting two keyboards and two mouses won't work because:
1) I cannot detect events from one particular keyboard or mouse connected
2) They are free to go from one monitor to another, I want them to be restricted to their monitors.

On the dual headed computer I will record macro events from the keyboard and mouse, and will generate the clicks and the keystrokes using a couple of devices (like 2 PI zero W) that will regenerate the events for the target computer using USB gadget keyboard and mouse emulation.

I could use a Teensy and or Arduino Nano but the PI Zero W would allow easier control. This is for the "regeneration" of the events captured but I would like to capture keyboard and mouse on the dual headed computer.

Apparently, It would be like you are using the target computer from the dual headed because the monitor will be replicated and the secondary keyboard and secondary mouse events will be captured on the dual headed computer and regenerated using the keyboard and mouse emulation provided by USB gadget.

I have one PI2 that could run just a Lazarus program that will transfer the mouse coordinates and button clicks to the dual headed computer but I am unable to extract mouse click information.

Hope it is more clear (not easy at all).

Giuseppe

[xardomain]

Here it is using fullscreen form:
procedure TForm1.FormCreate(Sender: TObject);
begin
      Form1.Top := 0;
      Form1.Left := 0;
      //Form1.Width:= 1920;
      // Form1.Height:= 1080;
      Form1.WindowState := wsFullScreen
end;

procedure TForm1.FormMouseDown(Sender: TObject; Button: TMouseButton;
  Shift: TShiftState; X, Y: Integer);
begin
  if Button = mbleft then
  begin
    CheckBox1.Checked := TRUE;
  end;
  if Button = mbmiddle then
  begin
    CheckBox2.Checked := TRUE;
  end;
  if Button = mbright then
  begin
    CheckBox3.Checked := TRUE;
  end;
end;

procedure TForm1.FormMouseUp(Sender: TObject; Button: TMouseButton;
  Shift: TShiftState; X, Y: Integer);
begin
       if Button = mbleft then
  begin
    CheckBox1.Checked := FALSE;
  end;
  if Button = mbmiddle then
  begin
    CheckBox2.Checked := FALSE;
  end;
  if Button = mbright then
  begin
    CheckBox3.Checked := FALSE;
  end;
end;


procedure TForm1.Timer1Timer(Sender: TObject);
begin
  // butn := mbleft;
  Label1.caption := 'X=' + IntToStr(Mouse.CursorPos.X);
  Label2.caption := 'Y=' + IntToStr(Mouse.CursorPos.Y);
end;

Not exactly an elegant solution but it works.

[Joanna from IRC]

Hi xardomain, just want to let you know that you should place your code inside the code viewer. It can be found when you modify your post...select code pascal and paste your code inside

Code: Pascal  [Select][+][-]

1. code goes here

Also you went to wrong place earlier which is why nobody answered.

[Lulu]

Hi, may be this can help you.

[xardomain]

Hi Lulu,
that unit does not work for me, if I understood correctly it is for console application (text?) not gui.

KodeZwerg,
Since that raspberry will be dedicated for mouse and keyboard events I just created an app with a form maximized, I can get mouse movement events by using mouse.cursorpos.x and y. This is not available if I specify uses mouse clause btw.

About keypress and mouse buttons event they trigger the form events, having the form maximized they always work.

>Hi xardomain, just want to let you know that you should place your code inside the code viewer.
Hi Joanna, thanks.

>Also you went to wrong place earlier which is why nobody answered.
Do you mean the IRC?

Currently starting to add a Synapse client that will send mouse and keyboard events to the server, another raspberry.

Code: Pascal  [Select][+][-]

1. procedure TForm1.Timer1Timer(Sender: TObject);
2. var test : longint;
3. begin
4.  // Label1.caption := IntToStr(Mouse.CursorPos.x);
5.   // Label2.caption := IntToStr(Mouse.CursorPos.y);
6.   Label1.caption := IntToStr(Mouse.CursorPos.X);
7.   Label2.caption := IntToStr(Mouse.CursorPos.y);
8.   Label3.caption := IntToStr(Form1.Width);
9.   Label4.caption := IntToStr(Form1.height)
10. end;
11.  
12. procedure TForm1.FormCreate(Sender: TObject);
13. begin
14.       Form1.Top := 0;
15.       Form1.Left := 0;
16.       // Form1.Width:= 1920;
17.       // Form1.Height:= 1080;
18.       Form1.KeyPreview := True;
19.       Form1.WindowState := wsFullScreen
20. end;
21.  
22. procedure TForm1.FormMouseDown(Sender: TObject; Button: TMouseButton;
23.   Shift: TShiftState; X, Y: Integer);
24. begin
25.     if Button = mbleft then
26.   begin
27.     CheckBox1.Checked := TRUE
28.   end;
29.   if Button = mbmiddle then
30.   begin
31.     CheckBox2.Checked := TRUE
32.   end;
33.   if Button = mbright then
34.   begin
35.     CheckBox3.Checked := TRUE
36.   end
37. end;
38.  
39. procedure TForm1.FormMouseUp(Sender: TObject; Button: TMouseButton;
40.   Shift: TShiftState; X, Y: Integer);
41. begin
42.   if Button = mbleft then
43.   begin
44.     CheckBox1.Checked := FALSE;
45.   end;
46.   if Button = mbmiddle then
47.   begin
48.     CheckBox2.Checked := FALSE;
49.   end;
50.   if Button = mbright then
51.   begin
52.     CheckBox3.Checked := FALSE;
53.   end;
54. end;

[VisualLab]

I would like to setup a second keyboard and a second mouse on the dual headed computer, completely independent from the target but also independent from the primary keyboard and the mouse, they need not to interfere each other.

Connecting two keyboards and two mouses won't work because:
1) I cannot detect events from one particular keyboard or mouse connected
2) They are free to go from one monitor to another, I want them to be restricted to their monitors.

On the dual headed computer I will record macro events from the keyboard and mouse, and will generate the clicks and the keystrokes using a couple of devices (like 2 PI zero W) that will regenerate the events for the target computer using USB gadget keyboard and mouse emulation.

The success of this venture probably does not depend on Lazarus or FPC, but on the operating system. The system would have to allow for unambiguous identification of mice and keyboards (i.e. provide their identifiers). In the mouse handling events implemented in LCL controls, there is no information from which mouse (something like ID or similar) the data (coordinates, button IDs) comes from. So the mouse information (device ID or something) would have to be obtained from the system in some other way.

Unfortunately, there is no standard GUI support in Linux - there is the old (and overly complicated) X Window and the still undercooked Wayland, which do not support all GUI functionality, but shift some of the functionality to desktop environment managers (GNOME, KDE, etc.).

So the solution to your problem may exist, but it will require a lot of your own work and custom workarounds.

---

EDIT:

And could you describe (more generally) what kind of software you are developing and what goal you want to achieve? Maybe some alternative ideas for your application would emerge from such a discussion? Such brainstorming of many people can sometimes bring interesting results.

[xardomain]

hi VisualLab,
I have a couple of PI2 laying around so the solution is already found.

One Pi2 will be a "dedicated" device, the keyboard and mouse will be connected to it. Its only purpose is to capture mouse and keyboard events.
The form is maximixed (tested with a PI4 at the moment) and correctly detect mouses movements and mouse clicks in headless mode using VNC. Just in case I ordered a HDMI "dummy" plug.

I will have to add some glue for handling keyboard events and send the data through a telnet connection to the dual headed pi.

The dual headed pi will use a "native" keyboard and mouse to use the GUI (let's say left monitor) and keep a VLC maximixed on the right monitor.
On the right monitor VLC will show maximixed the frame grabber with the output of the target computer.

Reading the events from the PI2 will generate mouse and keyboard using a PI zero (hopefully one will be enough for mouse/keyboard, else one for the keyboard and another for the mouse).

Not the easiest solution but doable. The "native" mouse will be free to roam from one monitor to another (maybe is possible to limit its movement to the left monitor) but will not influence the target machine (you need to generate events using the pi zero in gadget mode to transmit commands/events).
Same goes for the "native" keyboard, it could not communicate with the target, unless the program running on the dual headed pi will decide to.

I only need a pi wireless ready adapter, I had one N150 small dongle that I am unable to locate. That was nice not requiring driver setup and such.

Giuseppe

[VisualLab]

Reading the events from the PI2 will generate mouse and keyboard using a PI zero (hopefully one will be enough for mouse/keyboard, else one for the keyboard and another for the mouse).

Well, operating systems currently in use were designed for workstations with the assumption that one keyboard, one mouse and one monitor would suffice for one user (except that support for more monitors in the most popular systems has somehow been patched over time). Anyway, from your description it seems that you solved the problem optimally.

[KodeZwerg]

For your fullscreen demo, I would do it like this:

Code: Pascal  [Select][+][-]

1. procedure TForm1.FormMouseMove(Sender: TObject; Shift: TShiftState; X,
2.   Y: Integer);
3. begin
4.   // unit mouse must not be included!
5.   Panel2.Caption := Format('x:%d - y:%d',[Mouse.CursorPos.X, Mouse.CursorPos.Y]);
6. end;
7.  
8. procedure TForm1.FormMouseDown(Sender: TObject; Button: TMouseButton;
9.   Shift: TShiftState; X, Y: Integer);
10. begin
11.   // set all boxes according to the pressed button
12.   CheckBox1.Checked := Button = mbLeft;
13.   CheckBox2.Checked := Button = mbMiddle;
14.   CheckBox3.Checked := Button = mbRight;
15.   CheckBox4.Checked := Button = mbExtra1;
16.   CheckBox5.Checked := Button = mbExtra2;
17. end;
18.  
19. procedure TForm1.FormMouseUp(Sender: TObject; Button: TMouseButton;
20.   Shift: TShiftState; X, Y: Integer);
21. begin
22.   // reset all boxes
23.   CheckBox1.Checked := False;
24.   CheckBox2.Checked := False;
25.   CheckBox3.Checked := False;
26.   // problem, XButton does not fire event
27.   CheckBox4.Checked := False;
28.   CheckBox5.Checked := False;
29. end;

Attach those handler to all your window controls and you have a more accurate variant

[KodeZwerg]

If you want to keep track of all mouse buttons that are pressed at same time:

Code: Pascal  [Select][+][-]

1. procedure TForm1.FormMouseDown(Sender: TObject; Button: TMouseButton;
2.   Shift: TShiftState; X, Y: Integer);
3. begin
4.   CheckBox1.Checked := ssLeft in Shift;
5.   CheckBox2.Checked := ssMiddle in Shift;
6.   CheckBox3.Checked := ssRight in Shift;
7.   CheckBox4.Checked := ssExtra1 in Shift;
8.   CheckBox5.Checked := ssExtra2 in Shift;
9. end;

[xardomain]

Many thanks for the suggestions.

>attach those handler to all your window controls and you have a more accurate variant
Controls may simply disappear once the code is good enough, after all it is a dedicated pi, no need to look at the screen, usually it will be headless.

Code: Pascal  [Select][+][-]

1. CheckBox1.Checked := ssLeft in Shift;
2.   CheckBox2.Checked := ssMiddle in Shift;
3.   CheckBox3.Checked := ssRight in Shift;
4.   CheckBox4.Checked := ssExtra1 in Shift;
5.   CheckBox5.Checked := ssExtra2 in Shift;
6.  

Can't decide between the two...

Code: Pascal  [Select][+][-]

1.   CheckBox1.Checked := Button = mbLeft;
2.   CheckBox2.Checked := Button = mbMiddle;
3.   CheckBox3.Checked := Button = mbRight;
4.   CheckBox4.Checked := Button = mbExtra1;
5.   CheckBox5.Checked := Button = mbExtra2;
6.  

Given you both reminded me that I may have two extra buttons (Yeah, I have a Razer fetish), I could detect wheelmouse as well.

I tried this:

Code: Pascal  [Select][+][-]

1. procedure TForm1.FormMouseWheel(Sender: TObject; Shift: TShiftState;
2.   WheelDelta: Integer; MousePos: TPoint; var Handled: Boolean);
3. begin
4.   if (WheelDelta > 0) then Label5.Caption := 'Up!';
5.   end;
6.   if (WheelDelta < 0) then Label5.Caption := 'Down!';
7.   Handled := true
8. end;  
9.  

And it works but how to detect when the wheel stops?
TIA

[KodeZwerg]

Feel free to try this.

[Joanna from IRC]

Xardomain, yes , some places there are inactive. Feel free to message me if you need more information.