Recent

Author Topic: [SOLVED] Populating a Chart from StringGrid. AddXY problem  (Read 1667 times)

barsoom

  • New Member
  • *
  • Posts: 13
Re: [SOLVED] Populating a Chart from StringGrid. AddXY problem
« Reply #15 on: November 15, 2019, 01:42:54 pm »
OMG! :o
I just discovered the cause of the slow behavior! It is the navigation panel! If you hide it (it is not required to disconnect it from the chart), just only to hide it.
Now the program is pretty fast when zooming and pannning!

However, i need to see the navigation pannel to know where i am when zooming in the plot!  >:D  So, i have a problem... may be it could be a different topic for this forum.

wp

  • Hero Member
  • *****
  • Posts: 6502
Re: [SOLVED] Populating a Chart from StringGrid. AddXY problem
« Reply #16 on: November 15, 2019, 05:49:15 pm »
Testing your demo I do not see a speed improvement when the NavPanel is gone.

The issue is something very simple, and we had this topic several times recently in the forum: when the Canvas.Pen.Width is >1 a severe speed loss is observed in Windows (due to "cosmetic" and "geometric" pen style optimizations in the API). While not noticable with "normal-sized" plots it becomes very dramatic with large datasets as in your case. So, just setting "ls.LinePen.Width := 1" in AcPlotFromGridExecute does the job.

You can also speed up populating the chart series when you put a "ls.ListSource.BeginUpdate" and "ls.ListSource.EndUpdate" around the "i" loop in AcPlotFromGridExecute. With this change the file reads in 0.46 s instead of 1.21 s.

To make the data file readable also in other countries (I could not read your data file at first) you should also take care of the DateSeparator and the year-month-day order in ShortDateFormat. Instead of touching DefaultFormatSettings I usually introduce a local TFormatSettings variable which defaults to DefaultFormatSettings and changes the elements needed, and is passed to the string conversion function as last parameter. This way the format settings of the OS are retained and the user can work with his standard format settings.

In order to visualize the zoomed range within the entire data range you can also use a TChartNavScrollBar, instead of the TChartNavPanel.

Code: Pascal  [Select]
  1. procedure TForm1.AcPlotfromGridExecute(Sender: TObject);
  2. var
  3.   colnumber: Integer;
  4.   rownumber: integer;
  5.   ls: Tlineseries;
  6.   x: TDateTime;
  7.   y: double;
  8.   i, n: integer;
  9.   fs: TFormatSettings;    // <------ added
  10. begin
  11.   // <---- added
  12.   fs := DefaultFormatSettings;
  13.   fs.DecimalSeparator := '.';
  14.   fs.DateSeparator := '/';
  15.   fs.ShortDateFormat := 'dd/mm/yyyy';
  16.   // ------- end addition ------>
  17.  
  18.   colnumber := StringGrid1.ColCount;
  19.   rownumber := StringGrid1.RowCount;
  20.  
  21.   for i := 1 to colnumber-1 do
  22.     begin
  23.       ls := Tlineseries.Create(Chart1);
  24.       chart1.addSeries(ls);
  25.       ls.Title := StringGrid1.Cells[i, 0];
  26.       ls.SeriesColor:= RGBToColor(Random(256), Random(256), Random(256));
  27.       ls.LinePen.Width := 1;   // <--- changed from 2 to 1.
  28.       ls.Pointer.Style := psCircle;
  29.       ls.ShowPoints:= true;
  30.       ls.Pointer.Visible := false;            // <----------- This reverts the previous line. ShowPoints and Pointer.Visible are the same!
  31.       //ls.Marks.Style:= smsLabel;
  32.       ls.Marks.Visible := False;
  33.       ls.ZPosition := colnumber-1-i;
  34.       ls.ListSource.BeginUpdate;      // <-------- added
  35.       for n := 1 to Rownumber-1 do
  36.         begin
  37.           if TryStrToDateTime(StringGrid1.Cells[0, n], x, fs) then
  38.           begin
  39.             if not TryStrToFloat(StringGrid1.Cells[i, n], y, fs) then
  40.               y := NaN;
  41.           end;
  42.           ls.AddXY(x, y);
  43.         end;
  44.       ls.ListSource.EndUpdate;    // <-------- added
  45.     end;
  46. end;

To avoid double storage of values as floating point numbers in the chart's source and as strings in the grid you could introduce an array or list of records (or objects) resembling the native data and use a TUserDefinedChartSource to pass these data to the series. And you could use a TDrawGrid instead of a TStringGrid to display the data values; this will also speed up populating the grid dramatically.

I am attaching a modified version of your demo in which a UserDefinedChartSource and a DrawGrid are used.
« Last Edit: November 18, 2019, 01:16:46 pm by wp »
Lazarus trunk / fpc 3.0.4 / all 32-bit on Win-10

barsoom

  • New Member
  • *
  • Posts: 13
Re: [SOLVED] Populating a Chart from StringGrid. AddXY problem
« Reply #17 on: November 18, 2019, 12:09:11 pm »
Thanks so much @wp!

yes, the linePen.Width is also an issue... Value 1 is quite small and not in all cases (may be colors), the plots looks nice. However, i have on mind to allow to change it on run time, so, not a big problem.

About the Navigation section, i will explore th chartnacscrollbar as you propose.

Great idea the TFormatSettings variable. I think i will create a form to allow to do changes on this settings

Thank you so much for your code. It works nicely. There is only an small issue. It does not change the entire data... It miss the last column of data. In the example file there are 8 columns (datetime  and 7 temperatures), but only 7 are loaded inthe drawgrid, and in the chart.

I think the problem is in LoadfromCSVFile procedure, but i do not see where is the problem.

Thanks!



wp

  • Hero Member
  • *****
  • Posts: 6502
Re: [SOLVED] Populating a Chart from StringGrid. AddXY problem
« Reply #18 on: November 18, 2019, 01:28:44 pm »
I think the problem is in LoadfromCSVFile procedure, but i do not see where is the problem.
This is because stringhelper "Split()" in FPC 3.04 has the issue that it does not add an empty string as last entry when the split string ends with a separator. I think this was fixed recently in fpc trunk.

You can take care of it when you set the length of the Temperatures array of each data point by using the  FTitles array where every temperature column has an entry:
Code: Pascal  [Select]
  1.         SetLength(FData[n].Temperatures, Length(FTitles));  // instead of ... Length(sa)-1)
Lazarus trunk / fpc 3.0.4 / all 32-bit on Win-10

barsoom

  • New Member
  • *
  • Posts: 13
Re: [SOLVED] Populating a Chart from StringGrid. AddXY problem
« Reply #19 on: November 18, 2019, 03:30:36 pm »
Much better! Thanks. The issue now is that it fills this last column with "0,000" when there is not data, not keeping it empty like in the other columns. It results on cero values also in the plot and not "NaN"
Thank you!

Edited:
There is another issue: It does not assign the symbol to the first creater series in the chart:
Code: Pascal  [Select]
  1. procedure TForm1.PlotData;
  2. var
  3.   i: integer;
  4.   //ls: TLineseries;
  5.   ucs: TUserDefinedChartSource;
  6. begin
  7.   Chart1.ClearSeries;
  8.   if Length(FData) = 0 then
  9.     exit;
  10.  
  11.   for i := 0 to Length(FData[0].Temperatures) - 1 do // <-- It starts here to crease new line series for each temperature column.
  12.   begin
  13.     ucs := TUserDefinedChartSource.Create(self);
  14.     ucs.PointsNumber := Length(FData);
  15.     ucs.OnGetChartDataItem := @GetSeriesData;
  16.  
  17.     ls := TLineSeries.Create(Chart1);
  18.     ls.Title := FTitles[i];
  19.     ls.SeriesColor := RGBToColor(Random(256), Random(256), Random(256));
  20.     ls.LinePen.Width := 1;
  21.     ls.Pointer.Style := psCircle;  //  <-- It seems this does not affect to the first created serie.
  22.     ls.ShowPoints := true;
  23.     ls.Marks.Visible := false;
  24.     ls.Source := ucs;
  25.     ls.Tag := i;
  26.     ls.ZPosition := i;
  27.     ucs.Tag := PtrInt(ls);
  28.     Chart1.AddSeries(ls);
  29.   end;
  30. end;      

After to load the data, all the series shows points, except the first one... But it is created, so the loop is working properly for the data reading...
« Last Edit: November 18, 2019, 04:24:31 pm by barsoom »

wp

  • Hero Member
  • *****
  • Posts: 6502
Re: [SOLVED] Populating a Chart from StringGrid. AddXY problem
« Reply #20 on: November 18, 2019, 06:29:41 pm »
The issue now is that it fills this last column with "0,000" when there is not data, not keeping it empty like in the other columns. It results on cero values also in the plot and not "NaN"
This was a quickly written example without much testing. You just must get the indexes right. It is only a bit complicated because the Length of the string array sa[] may be shorter than that of the FTitles array.
Code: Pascal  [Select]
  1. procedure TForm1.LoadFromCSVFile(AFileName: String);
  2. var
  3.   L: TStrings;
  4.   sa: TStringArray;
  5.   n: Integer;
  6.   fs: TFormatSettings;
  7.   y: Double;
  8.   i, j, k: Integer;
  9.   t: TDateTime;
  10. begin
  11.   t := Now;
  12.  
  13.   fs := DefaultFormatSettings;
  14.   fs.DecimalSeparator := '.';
  15.   fs.DateSeparator := '/';
  16.   fs.ShortDateFormat := 'dd/mm/yyyy';
  17.  
  18.   L := TStringList.Create;
  19.   try
  20.     L.LoadFromFile(AFileName);
  21.     SetLength(FData, L.Count-1);
  22.  
  23.     sa := L[0].Split(';');
  24.     SetLength(FTitles, Length(sa) - 1);
  25.     for i := 1 to High(sa) do
  26.       FTitles[i-1] := sa[i];
  27.  
  28.     n := 0;
  29.     for i := 1 to L.Count - 1 do
  30.     begin
  31.       sa := L[i].Split(';');
  32.       if sa[0] = '' then
  33.         Continue;
  34.       if TryStrToDateTime(sa[0], FData[n].DateTime, fs) then
  35.       begin
  36.         SetLength(FData[n].Temperatures, Length(FTitles));
  37.         for j := 0 to High(FTitles) do
  38.         begin
  39.           k := j+1;  
  40.           if (k < Length(sa)) and TryStrToFloat(sa[k], y, fs) then
  41.             FData[n].Temperatures[j] := y
  42.           else
  43.             FData[n].Temperatures[j] := NaN;
  44.         end;
  45.         inc(n);      // count valid lines; adjust array length at end
  46.       end;
  47.     end;
  48.     SetLength(FData, n);  // Set array length to count of lines with date
  49.   finally
  50.     L.Free;
  51.   end;
  52. end;

There is another issue: It does not assign the symbol to the first creater series in the chart
[...]
After to load the data, all the series shows points, except the first one... But it is created, so the loop is working properly for the data reading...
Maybe just a consequence of the index error in LoadFromCSVFile.
Lazarus trunk / fpc 3.0.4 / all 32-bit on Win-10

barsoom

  • New Member
  • *
  • Posts: 13
Re: [SOLVED] Populating a Chart from StringGrid. AddXY problem
« Reply #21 on: November 18, 2019, 08:27:59 pm »
Perfect!!!

The problem with values on empty cells is solved with your new code. Thanks a lot!!!

About the problem with visible points... forget it. It was caused by my fault. I accidentally changed a number in another piece of code in other part of the program.

So, definetively SOLVED.

Thanks a lot @wp!!
« Last Edit: November 18, 2019, 08:40:30 pm by barsoom »