Attitude animation with BGRAbitmap

[ccrause]

So following on from this suggestion I reworked (or rather hacked) the 3D pyramid example, moved the RotateX/Y/Z bit into a timer that pulls attitude data from an MPU6050 over a serial connection.

The rendering rate is in the order of 40 - 70 ms per frame, any pointers on how to improve the animation speed?

Code below (for inspection, obviously it requires the serialobject unit and a serial input data stream to work properly).

Code: Pascal  [Select][+][-]

1. unit Unit1;
2.  
3. {$mode objfpc}{$H+}
4.  
5. interface
6.  
7. uses
8.   Classes, SysUtils, Forms, Controls, Graphics, Dialogs, ExtCtrls, ComCtrls;
9.  
10. type
11.  
12.   { TForm1 }
13.  
14.   TForm1 = class(TForm)
15.     StatusBar1: TStatusBar;
16.     Timer1: TTimer;
17.     procedure FormClose(Sender: TObject; var CloseAction: TCloseAction);
18.     procedure FormPaint(Sender: TObject);
19.     procedure Timer1Timer(Sender: TObject);
20.   private
21.     procedure CreateShape;
22.   public
23.  
24.   end;
25.  
26. var
27.   Form1: TForm1;
28.  
29. implementation
30.  
31. uses
32.   BGRAScene3D, BGRABitmap, BGRABitmapTypes,
33.   epiktimer, serialobject;
34.  
35. {$R *.lfm}
36.  
37. { TForm1 }
38. var
39.   scene: TBGRAScene3D;
40.   obj3D: IBGRAObject3D;
41.   bmp: TBGRABitmap;
42.   base: array of IBGRAVertex3D;
43.   topV: IBGRAVertex3D;
44.   serial: TSerialObj;
45.   eptimer: TEpikTimer;
46.  
47. procedure TForm1.FormPaint(Sender: TObject);
48. begin
49.   CreateShape;
50.   if not Assigned(serial) then
51.   begin
52.     serial := TSerialObj.Create;
53.     if serial.OpenPort('/dev/ttyACM0', 115200) then
54.     begin
55.       Timer1.Enabled := true;
56.       serial.FlushInput;
57.       eptimer := TEpikTimer.Create(nil);
58.       eptimer.Start;
59.     end
60.     else
61.     begin
62.       FreeAndNil(serial);
63.       ShowMessage('Error opening serial port.');
64.     end;
65.   end;
66. end;
67.  
68. procedure TForm1.FormClose(Sender: TObject; var CloseAction: TCloseAction);
69. begin
70.   Timer1.Enabled := false;
71.   if Assigned(serial) then
72.     FreeAndNil(serial);
73.   if Assigned(scene) then
74.     FreeAndNil(scene);
75.   if Assigned(obj3D) then
76.     FreeAndNil(obj3D);
77.   if Assigned(bmp) then
78.     FreeAndNil(bmp);
79. end;
80.  
81. var
82.   x: integer = 0;
83.   y: integer = 0;
84.   z: Integer = 0;
85.  
86. procedure TForm1.Timer1Timer(Sender: TObject);
87. var
88.   data: packed array[0..2] of int16;
89.   databytes: packed array[0..5] of byte absolute data;
90.   r: integer;
91. begin
92.   Timer1.Enabled := false;
93.  
94. // Request new data
95.   if Assigned(serial) then
96.   begin
97.     serial.FlushInput;
98.     serial.Write(ord('.'));
99.   end;
100.  
101.   // Update display with previous values
102.   StatusBar1.Panels[3].Text := IntToStr(x) + 'º';
103.   StatusBar1.Panels[5].Text := IntToStr(y) + 'º';
104.   StatusBar1.Panels[7].Text := IntToStr(z) + 'º';
105.  
106.   eptimer.Clear;
107.   eptimer.Start;
108.   bmp.Canvas.Brush.Color := clBlack;
109.   bmp.Canvas.FillRect(0, 0, ClientWidth, ClientHeight);
110.   CreateShape;
111.   obj3D.MainPart.RotateYDeg(y);  // Rotate around vertical axis, + = clockwise as viewed from top
112.   obj3D.MainPart.RotateXDeg(x);  // vertical, - tilt top away from viewer
113.   obj3D.MainPart.RotateZDeg(z);  // vertical, - tilt top clockwise in front of viewer
114.   scene.Render;
115.   bmp.Draw(Canvas,0,0);
116.   eptimer.Stop;
117.   StatusBar1.Panels[1].Text := FloatToStrF(epTimer.Elapsed*1000, ffGeneral, 3, 2);
118.  
119.   // Read new data
120.   if Assigned(serial) then
121.   begin
122.     r := serial.ReadTimeout(databytes, 6, 50);  // 3x 2 byte values
123.     if r <> 6 then
124.       ShowMessage('data error')
125.     else
126.     begin
127.       y := data[0];
128.       z := data[1];
129.       x := -data[2];
130.     end;
131.   end;
132.   Timer1.Enabled := true;
133. end;
134.  
135. procedure TForm1.CreateShape;
136. begin
137.   if not Assigned(bmp) and not Assigned(scene) then
138.   begin
139.     bmp := TBGRABitmap.Create(ClientWidth,ClientHeight,BGRABlack);
140.     scene := TBGRAScene3D.Create(bmp);
141.   end
142.   else
143.     scene.Clear;
144.  
145.   //create a pyramid
146.   obj3D := scene.CreateObject(BGRA(255,240,128));
147.   with obj3D do
148.   begin
149.     //create vertices
150.     topV := MainPart.Add(0,-50,0);
151.     //pyramid base is in a clockwise order if we look at the pyramid from below
152.     base := MainPart.Add([-20,25,-20,  20,25,-20,  20,25,20,  -20,25,20]);
153.  
154.     AddFace(base, BGRA(127,127,127));
155.     //add four faces, the three vertices are in a clockwise order
156.     AddFace([base[0],topV,base[1]], BGRA(255, 0, 0));
157.     AddFace([base[1],topV,base[2]], BGRA(0, 255, 0));
158.     AddFace([base[2],topV,base[3]], BGRA(0, 0, 255));
159.     AddFace([base[3],topV,base[0]]);
160.     topV := nil;
161.     base := nil;
162.   end;
163. end;
164.  
165. end.
166.  

[Handoko]

As I do not have the hardware so I cannot test your code. And please do not take what I going to say seriously, I'm not an expert.

Why do you call Timer event (by enabling it) inside Form.OnPaint event? Isn't it better the Timer event do some non-drawing calculations then call Form.Invalidate when necessary? Timer event shouldn't be called inside OnPaint, I think.

Your OnPaint calls CreateShape then it calls Timer event, which also calls CreateShape. It seems overlapped. Also in CreateShape obj3D is created but I saw it's only freed in FormClose. obj3D is created several times but only be freed once in FormClose, it doesn't seem good to me.

Usually, to improve graphics performance, we should keep the OnPaint event as short/fast as possible. Your OnPaint is not tightly optimized. At its beginning it does some drawings by calling CreateShape but later it does hardware checking. That is not good. You should separate them.

I believe BGRAScene3D is hardware accelerated but for painting to the screen, we have to use TBGRABitmap.Draw, which is extremely slow. TBGRABitmap.Draw is a deal-breaker if you need high performance drawing. Use others instead, maybe TOpenGLControl.

[ccrause]

@Handoko, thank you very much for your observations.  I've cleaned up the code a little to address some of your comments, and replaced the serial data with simple increments of x, y, z rotations so that anyone can test the code (if you have the BGRA components installed).  The new test project is attached.

You are probably correct that TBGRABitmap.Draw is the bottleneck, I simply adapted a demo which wasn't geared towards animation.

I guess I need to figure out how to use OpenGL...

[winni]

Hi!

No. BGRABitmap.draw is NOT the bottlenek.

I took your code, deleted the epiktimer and the serialstuff.

Took a timer at 50 ms.
And replaced the serial input with random(360).

I measured the painting with  getTickCount64.
It shows always  between 15 and 21 ms for one pyramid.
Every 50 ms.
Don't look on it too long . You get mad.

Source in the attachment.

Winni

[ccrause]

No. BGRABitmap.draw is NOT the bottlenek.

Thanks winni. On my modest laptop your code runs between 40 - 60ms, within the same range as my demo. I've tested using OpenGLControl, but then the frame draw time slows down to approximately 75 ms.  You are correct in saying that BGRABitmap.draw is not the bottleneck.

Attached the project I use to compare TOpenGLControl/TBGLBitmap rendering vs. normal Canvas/TBGRABitmap. A define at the top of unit1 controls which case gets compiled.

[Handoko]

I downloaded your code but I can test it, I only got a blank project. I believe that happened because you're using higher version of Lazarus than mine, I'm using Lazarus 2.0.10.

I eye-inspected your code, you're using BGLCanvas.PutImage then OpenGLCtrl.SwapBuffers. I believe that is not correct. PutImage is as slow as Draw. I look deep into the code of PutImage, it calls FillRect, which is not hardware optimized. It manually fills the lines from top to bottom as you can see in the code below:

Code: Pascal  [Select][+][-]

1. procedure TCustomUniversalBitmap.FillRect(ALeft, ATop, ARight, ABottom: integer;
2.   const ABrush: TUniversalBrush; AAlpha: Word);
3. var
4.   yb, sx: integer;
5.   pScan: PByte;
6.   delta: PtrInt;
7.   ctx: TUniBrushContext;
8. begin
9.   if ABrush.Colorspace <> Colorspace then RaiseInvalidBrushColorspace;
10.   if not CheckClippedRectBounds({%H-}ALeft,{%H-}ATop,{%H-}ARight,{%H-}ABottom) or
11.     (AAlpha = 0) or ABrush.DoesNothing then exit;
12.  
13.   LoadFromBitmapIfNeeded;
14.   pScan := GetPixelAddress(ALeft, ATop);
15.   if LineOrder = riloBottomToTop then
16.     delta := -RowSize
17.   else
18.     delta := RowSize;
19.   sx := ARight - ALeft;
20.  
21.   for yb := ATop to ABottom-1 do
22.   begin
23.     ABrush.MoveTo(@ctx, pScan,ALeft,yb);
24.     ABrush.PutNextPixels(@ctx, AAlpha,sx);
25.     inc(pScan, delta);
26.   end;
27.   InvalidateBitmap;
28. end;

I'm not familiar with BGRAthings. But I saw there is a component called TBGLVirtualScreen. I think that is what you need. Or instead you should totally write it in OpenGL. You were combining BGLCanvas and TOpenGLControl that harms performance even more because they need to convert/move the data in the memory to the other.

And yeah, maybe winni is correct. The bottleneck is somewhere else not the graphics performance. You need to find out where the problem is before optimize the graphics.

[ccrause]

I downloaded your code but I can test it, I only got a blank project. I believe that happened because you're using higher version of Lazarus than mine, I'm using Lazarus 2.0.10.

My apologies, I was using an old trunk version, have now build the latest release.

I eye-inspected your code, you're using BGLCanvas.PutImage then OpenGLCtrl.SwapBuffers. I believe that is not correct. PutImage is as slow as Draw. I look deep into the code of PutImage, it calls FillRect, which is not hardware optimized.
...
I'm not familiar with BGRAthings. But I saw there is a component called TBGLVirtualScreen. I think that is what you need. Or instead you should totally write it in OpenGL. You were combining BGLCanvas and TOpenGLControl that harms performance even more because they need to convert/move the data in the memory to the other.

Spot on Handoko! I spliced together a project from various tests using TBGRAVirtualScreen (not sure how different it is from TBGLVirtualScreen) and it is significantly faster (~50 ms down to ~ 25 ms). I will attach this test, it should be readable in Lazarus 2.0.10.

And yeah, maybe winni is correct. The bottleneck is somewhere else not the graphics performance. You need to find out where the problem is before optimize the graphics.

Since I am completely new to BGRA graphics I'm sure there is plenty that can be improved in my code - as you have demonstrated.

[winni]

Hi!

It is indeed the next way to speed up your code:

Put a BGRAvirtualScreen on your form.

Then you dont need the var bmp (BGRAbitmap) in your code.
The  BGRAvirtualScreen has the properety

bitmap: TBGRABitmap;


Just use that instead of var bmp.
Either the Vscreen draws itself or you have to force a refresh -
I dont know.

Winni

[ccrause]

Hi!

It is indeed the next way to speed up your code:

Put a BGRAvirtualScreen on your form.

Then you dont need the var bmp (BGRAbitmap) in your code.
The  BGRAvirtualScreen has the properety

bitmap: TBGRABitmap;


Just use that instead of var bmp.
Either the Vscreen draws itself or you have to force a refresh -
I dont know.

Winni

Winni, yes I've made the changes you mentioned.  To update the virtual screen I call BGRAVirtualScreen1.RedrawBitmap after constructing the pyramid in the timer event.  The rendering now happens in BGRAVirtualScreen1Redraw.

[circular]

Hi there

If you would like to optimize with OpenGL, then you need to draw the scene using OpenGL rendering, not create a bitmap that you then put on the OpenGL surface.

Here you have a test program that can either run with or without OpenGL:
https://github.com/bgrabitmap/bgrabitmap/tree/master/test/bgratutorial3d

The unit BGRAOpenGL3D provides TBGLScene3D which is like TBGRAScene3D but with RenderGL method as well.

Regards

[ccrause]

The unit BGRAOpenGL3D provides TBGLScene3D which is like TBGRAScene3D but with RenderGL method as well.

Thank you for the advice circular.  I've switched to TBGLScene3D as suggested and the frame display time dropped from previous time of ~ 25 ms to just below 10 ms.

The code for the spinning pyramid test is attached for reference.