[SOLVED] Program runs smoothly on Linux, but not on Windows

[munair]

I have been working on a program on Linux but without a specific OS in mind. Theoretically it should run more or less the same on Windows, but it doesn't.

The program is supposed to load csv files with up to about 110.000 lines (a month worth of recording values) and display their values on a TAChart. The csv file included in the project contains close to 71.000 lines. On Linux TAChart has no problem copying and processing these vast amounts of data, even using pan and zoom, but on Windows 10 it seems to take forever and the program becomes non-responsive when panning and zooming.

I know there is a way to optimize adding data using TUserDefinedChartSource, but for some reason formatting the x values (date/time) did not work well. But even once loaded, panning and zooming performs really bad on Windows.

The program also has a screenshot function that automatically takes a picture of the chart area. On Linux this works flawlessly but on Windows I get a black image.

EDIT: Corrected project: https://ditrianum.org/downloads/aeview-1.0.0.45.zip

[winni]

Confirmed. On Linux everything runs fluently.

Windows 7, Virtualbox: Loading on start takes ~33 seconds. Zoom: first time it take ~ 11 seconds, 2nd time it takes 4 seconds. And the screenshot is black.

I had the same issue with 65.000 lines csv to be read into a stringgrid. It ran realy speedy with linux and boring slow with windows.

I did the folowing:

Loop 1: count csv lines
Loop 2: assign the needed space in your data and read in the data.

In the case of the stringgrid that speeded up windows significant.

Winni

[munair]

Yes, loading (counting lines first) could be optimized. But there is still the issue of panning and zooming. Windows seems to be having a hard time plotting large amounts of points.

[winni]

Just played around with AirView on Windows.

Only to refresh the windows after moving another window over it took around half a minute. Meanwhile the taskmanager shows 25% CPU used - that means that 100% of one of the 4 CPUs is used. Windows tells that the app is not answering, but you only have to wait!

I don't know how to solve that. Buy a magnifier glass for windows?

Winni

[wp]

Stepping through your FormCreate procedure it can be seen that LoadFile (it is lacking a "CloseFile", BTW) is executed immediately, as well as "LoadChart". File reading and chart creation, therefore, can be excluded to be the reason for the slow execution. Further investigation shows that painting of the chart seems to be the bottleneck.

I left your program alone for the moment and wrote a my own charting program for some dummy data with similar data range and even more data points (100,000). The chart of this programs is visible within fractions of a second, Zooming and panning occur instantly.

So, what is the difference between my program and yours?

[munair]

Thanks wp. Added CloseFile. 

The only difference I can think of regarding the painting of so many points is the processing of DateTime elements on the x axis.

[wp]

No. My program has added TDateTime values as well (as floats, though, but there is a TDateTimeIntervalChartSource). To convince you I loaded your file in the attached new version of the demo. The plot appears almost immediately.

There is something which you must be doing differently related to the chart, but I can't see it... (Your file reading is different, but this is not the issue because the debugger steps over your "LoadFile" instantly).

[munair]

Yes, I see that now. I removed the chart tools component and constant lines from my project but it makes no difference.

EDIT: It is zooming in/out large parts (lots of points) that chokes the program.

[wp]

EDIT: It is zooming in/out large parts (lots of points) that chokes the program.

I doubt that this is the problem because no matter how I zoom into my demo the program always runs fast.

Just to make sure: When the zoomed rectangle is large and contains many data points (magnification near 1) then your program is slow? And when the zoomed rect is small and contains few points (large magnification factor) then your program is fast? In this case sorting the ChartSource could help (Chart1LineSeries1.ListSource.Sorted := true, before adding data per AddXY). There was a recent change in TAChart regarding sorted sources. What is your Laz version?

[munair]

Just to make sure: When the zoomed rectangle is large and contains many data points (magnification near 1) then your program is slow? And when the zoomed rect is small and contains few points (large magnification factor) then your program is fast?

Yes, that is indeed the case.

In this case sorting the ChartSource could help (Chart1LineSeries1.ListSource.Sorted := true, before adding data per AddXY). There was a recent change in TAChart regarding sorted sources. What is your Laz version?

Chart1LineSeries1.ListSource.Sorted := true (I suppose before the iteration loop) seems to make no difference. Prior to your latest post I used version 1.8.4. I just installed 2.0.4, but with the same result.

[wp]

Can you confirm explicitly that my demo ("..._v2") is running fast? If yes, I'd ask you to add chart-related features from your program to my program step by step and find out which feature makes it slow (instead of removing features from your program until it becomes fast) .

[munair]

Yes your program (v2) runs OK. I'll be adding features from my project one by one.

[munair]

It is in Chart1LineSeries1. I figured it had to do something with data calculation. So I removed Chart1LineSeries1 from the project and added a new one. Now the program loads and runs FAST. Could it be a Chart1LineSeries1 setting ?

[wp]

Looking again at the LineSeries properties I noticed now that you had Width = 2. Setting it to 1 made your program run as fast as mine. This probably also explains the difference between Linux: Maybe the gtk2 or qt canvas is better optimized for non-default linewidths than the Windows canvas.

I never noticed this issue, thus it is good to know the reason. But, in fact, there is already a short note in https://wiki.lazarus.freepascal.org/TAChart_documentation#Fast_lines.

[munair]

Looking again at the LineSeries properties I noticed now that you had Width = 2. Setting it to 1 made your program run as fast as mine. This probably also explains the difference between Linux: Maybe the gtk2 or qt canvas is better optimized for non-default linewidths than the Windows canvas.

I never noticed this issue, thus it is good to know the reason. But, in fact, there is already a short note in https://wiki.lazarus.freepascal.org/TAChart_documentation#Fast_lines.

Yes, I noticed the same thing with the LinePen. I removed the series from the Linux project, added a new one and noticed the thin line, so I figured, what if I set the LinePen width in the Windows project to 2. And indeed, it takes about 30 seconds for Windows to (re)draw large parts of the canvas. Needle in a hay stack, but good to know.

Thanks a lot for helping me out here.