LinePen Width decreases performance when chart has many data

[Muso]

This is surely a beginners question but googling around did not yet help me.

In my application, every 2 seconds a new datapoint is added to a TLineSeries. The measurement producing the data should run 2 weeks. Now, after just 12 hours, my program becomes really slow. I get the full program speed back when I only display the last 1000 values by changing the chart extent so that the user see the last 1000 seconds in a scrolling view.

However, from time to time the whole dataset should be shown and then the user can turn off the scrolling view. And when doing so the program becomes immediately slow.

So what is the best strategy do display 100k datapoints ore more? I could e.g. remove 9 out of every10 datapoints but maybe there is an other solution.

[wp]

Are you using my code for scrolling the visible viewport that I posted recently? This certainly is not optimized for many data points. Since the data are coming in one by one it is not required to calculate the series extent from the start with every new data value. Rather than that, store the previously used extent (FViewPort) and expand it when a new data value is added, but falls outside the previous extent. This way thousands of comparisons can be avoided.

Code: Pascal  [Select][+][-]

1. procedure TForm1.AdjustViewport(x: Double = NaN; y: Double = NaN);
2. begin
3.   if not IsNaN(x) and not IsNaN(y) then
4.   begin
5.     ExpandRect(FViewport, DoublePoint(x,y));
6.     FViewport.a.x := FViewport.b.x - 10;
7.   end;
8. end;

FViewport should be initialized to EmptyRect before the first data point arrives.

[Muso]

Are you using my code for scrolling the visible viewport that I posted recently?

When scrolling is off, the only chart action that is performed on every new data is

Code: Pascal  [Select][+][-]

1. LineSeries.AddXY(x, y)

The extent of the chart is however still the same because when scrolling was on, the extent was limited using the function

Code: Pascal  [Select][+][-]

1. Chart.Extent.FixTo(Extent)

I don't update the extent because when watching the full range of hours, some seconds more or less cannot be seen anyway. Therefore I am wondering why it gets so slow.

[Muso]

Hmm, attached is a small example program of my solution and even with 250 k datapoints it works.

The only issue I encounter is that when scrolling is off, you go to the first tab and then press the button to show the chart, it takes a while until the chart is displayed.

So I have to find now out why in my real-life program the same solution leads to a slow program just with 25 k datapoints.

[Muso]

I found the bug:
- take the example program from my previous post
- set for at least one LineSeries the LinePen Width to 2

- run the program

result: it is very slow.

So really only the pen width makes the difference. Is there any workaround? I set in my real-life program the width to 2 to make the lines better visible.

[wp]

Is there any workaround?

No, a Windows-internal optimization (https://docs.microsoft.com/en-us/windows/win32/gdi/cosmetic-pens) - see also: https://forum.lazarus.freepascal.org/index.php/topic,47074.msg339537.html#msg339537.

What you can do is
* stick to pen width 1
* or: reduce your data density. For example you could store your data in arrays and use a user-defined chartsource to get them into the series by means of the OnGetDataItem event - and here you could skip an amount of data in case of the unzoomed display (say: plot only every 10th data point when there are more than 10,000 points, plot every 100th data point when there are more than 100,000 points etc.)

In the attachment there is an example how the second point could be done

[winni]

Hi!

Workaround:

Draw your TAChart on a BGRAbitmap.
There is no M$ "optimization."

https://wiki.freepascal.org/BGRABitmap_tutorial_TAChart

Winni

[wp]

The attached demo is  the same speed test of the previous post, but now using BGRABitmap. The lineseries is drawn considerably faster. But if the user would decide to display data point symbols he will be punished by an unacceptably long drawing time.

[Muso]

The attached demo is  the same speed test of the previous post, but now using BGRABitmap. The lineseries is drawn considerably faster. But if the user would decide to display data point symbols he will be punished by an unacceptably long drawing time.

Many thanks.

My real-life program has 9 line series and 9 different data sources. I expect 600 k values over 14 days. Therefore I modified your program to this and also added to switch between BGRA or not, see attached.

The result for 300k values is:

* no BGRA
** no symbol
*** pen Width 1 0.13
*** pen Width 2: 0.255

** with symbol
*** pen Width 1 1.20
*** pen Width 2: 1.34

* with BGRA
** no symbol
*** pen Width 1 0.19
*** pen Width 2: 0.245

** with symbol
*** pen Width 1 15.8
*** pen Width 2: 15.8

So as soon as symboly come into play BGRA has problems.

I decided now to stay without BGRA and pen width 1 when displaying all values. I mean the difference is simply invisible. When in scrolling mode only 1000k datapoints are displayed and then he pen Width makes no real difference. Therefore I use the a pen width of 2.

[wp]

Something must be wrong in your speed test because pen.Width 2 must be considerably slower than pen.Width 1, in particular in the no symbol case.

[Muso]

Something must be wrong in your speed test because pen.Width 2 must be considerably slower than pen.Width 1, in particular in the no symbol case.

Indeed, now the values are correct.

However, my example project also visualizes a bug:
1. make the chart title visible in the IDE
2. in the running program toggle on/off BGRA

result: with BGRA, the title backgorund is different than without, see the attached screenshot. In the IDE I can set different background colors, but when BGRA is used the background is always black.
Should I open a bugreport?

[wp]

Try the new revision in trunk.

Indeed, now the values are correct.

Your program starts with hard-coded linewidth 2, but the PenWidth spinedit is at 1. Put the numbers in sync.

[Muso]

Try the new revision in trunk.

Many thanks for the quick fix!

Your program starts with hard-coded linewidth 2, but the PenWidth spinedit is at 1. Put the numbers in sync.

Thanks, I updated my post accordingly. However, the measured times remain the same.

[wp]

I do not want todebug what's wrong when the LinePen.Width = 2 and SpinEdit.value = 2 at startup. But when I reset them to 1, I get

- linewidth 1, no symbols, no BGRA --> 0.35 s
- linewidth 2, no symbols, no BGRA --> 0.67 s
- linewidth 2, symbols, no BGRA --> 1.36 s
- linewidth 1, symbols, no BGRA --> 1.02 s

i.e. the time penalty for linewidth 2 is 0.3 s (this is for one series only).