Recent

Author Topic: TAChart Fit Series  (Read 7765 times)

wp

  • Hero Member
  • *****
  • Posts: 6444
TAChart Fit Series
« on: October 05, 2011, 12:55:17 am »
I would like to donate the enclosed TFitSeries component to TAChart in order to acknowledge the great work that has been put into this library and into Lazarus.

TFitSeries allows to fit an equation to a set of x,y values and displays the fitted curve in a TChart. Internally it uses the least squares fitting routine provided by the numlib package which is distributed along with FPC.

Unfortunately this library implements only fitting to a polynomial, so TFitSeries is not good for general fitting purposes, but at least, it is at the same level as Excel:

(1) fit to a polynomial: y = b0 + b1*x + ... + bn*x^n
(2) linear fit: y = a + b*x
(3) exponential fit: y = a * exp(b*x)
(4) power law fit: y = a*x^b

(2) is a simple special case of (1), I added it nevertheless since line fits are needed quite often. (3) and (4) are special cases as well, but not so straightforward, since they need some transformations.

The attached demo project shows the main features. It uses several test data sets generated for each of these function types and allows to play around with the fit settings. The project creates the FitSeries at runtime, so no need to install anything.

Unfortunately, there are some open issues:

- The drawing procedure of TFitSeries ignores axis transformations. You can see this when you check one of the logarithmic check boxes. Alexander, can you help?

- I'm still struggling to get good logarithmic axis marks. Finally, I used a dedicated chart source with full decade values. Some data sets have only a restricted range, and the log axis has too few tick marks. The procedure TForm1.CbLogClick, therefore, detects when the data range covers less than 2 decades, deactivates the chart source and tries to apply standard interval generation. However, in this case, the program crashes in TChartAxis.Draw. See my comments in CbLogClick to reproduce this issue.

- The demo contains two checkboxes and edits to restrict the fit range. My original intention was to recalculate the fit whenever one of these controls changes (TFitSeries.ExecFit). These controls directly change properties of TFitSeries.FitRange which is of type TChartRange. Unfortunately, these changes do not trigger recalculation of the fit. What do I have to do to make this work? (In order to get a smooth demo, I added an "Apply" button which calls ExecFit directly).
Lazarus trunk / fpc 3.0.4 / all 32-bit on Win-10

Ask

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 687
Re: TAChart Fit Series
« Reply #1 on: October 05, 2011, 10:15:24 am »
Quote
I would like to donate the enclosed TFitSeries component to TAChart
Thanks.

Quote
The drawing procedure of TFitSeries ignores axis transformations

You need to call AxisToGraph/GraphToAxis in addition to ImageToGraph/GraphToImage.
See http://wiki.lazarus.freepascal.org/TAChart_documentation#Coordinates_and_axises
After that, publish AxisIndexX and AxisIndexY properties to let user assign axises at design-time.

However, you should not write this code yourself. Instead, use TDrawFuncHelper
from the TAFuncSeries unit (it is currently private to this unit, so you need to
move it to the interface first).
See TCubicSplineSeries for an example.

Quote
I'm still struggling to get good logarithmic axis marks.
I see. Will investigate later.
Edit: This was because TAChart tried to draw minor marks below zero.
Fixed by r32700.
As for good marks, setting NiceSteps to either just "1.0" or
"0.30102999566398|1.0", and UseGraphsCoords=true gives decent results.

Quote
hese controls directly change properties of TFitSeries.FitRange which is of type TChartRange. Unfortunately, these changes do not trigger recalculation of the fit. What do I have to do to make this work?

Yes, this is slightly tricky. When you create series at run-time, it's parent chart is not defined until you call AddSeries. The following code makes it work:

Code: [Select]
procedure TFitSeries.AfterAdd;
begin
  inherited AfterAdd;
  FFitRange.SetOwner(ParentChart);
end;
« Last Edit: October 05, 2011, 12:26:25 pm by Ask »

wp

  • Hero Member
  • *****
  • Posts: 6444
Re: TAChart Fit Series
« Reply #2 on: October 05, 2011, 04:23:02 pm »
Thank you for your explanations.

Two more ideas:

- Declare the methods "Calculate" and "ExecFit" as virtual. This would make it easier to replace the fitting engine while keeping the series infrastructure in place.

- In "ExecFit", ignore NaNs when collecting the data for the ipfpol procedure.
Lazarus trunk / fpc 3.0.4 / all 32-bit on Win-10

Ask

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 687
Re: TAChart Fit Series
« Reply #3 on: October 05, 2011, 05:30:10 pm »
Reasonable.

Will you make v2?

wp

  • Hero Member
  • *****
  • Posts: 6444
Re: TAChart Fit Series
« Reply #4 on: October 06, 2011, 12:44:18 am »
I packed the entire FitSeries into the TAFuncSeries unit, this avoids exposing the DrawFuncHelper. Some issues are fixed, others are new:

- Using the DrawFuncHelper, the logarithmic fit curves are drawn correctly.

- There's an issue with log axes: run the program and select the third test function (exponential). When switching to log y the program crashes. This seems to be due to the negative values of the fitted curve between x=70 and 80. The AxisToGraph method of the LogarithmAxisTransform returns -INF for negative numbers. Some procedures in the library go crazy with that input - see my other posting. Ideally, the rest of the data should be drawn correctly, the illegal values should be ignored, at least optionally.

- I implemented the overridden AddSeries method and removed the Apply button. When clicking on "FitRange / Use Minimum" now the shorter fit curve is drawn immediately, but the ExecFit procedure is still not executed. This is my understanding of the notification sequence: change of FitRange --> FitRange.StyleChanged --> Chart.StyleChanged--> Broadcast --> Series Listener calls SourceChanged of Series. TFitSeries overrides the SourceChanged method and should run the fit, but it is not called. What is wrong?

- Old story: logarithmic labels. After playing around quite a lot, I found settings which avoid the chartsource and show the full decade labels (Options = [aiGraphCoords, aipUseMinLength, aipUseMaxLength, aipUseNiceSteps], MinLength=50, MaxLength=200, NiceSteps='1'), as well as settings for the constant intervals on log axis (Options = [aipUseMinLength, aipUseNiceSteps], MinLength=10, NiceSteps=log2|log5|log10). In the latter case, however, the labels are too far apart - run the program with the first test function and the first fit function, and activate log y. How can I reduce the interval length from 100 to, say, 20?

Lazarus trunk / fpc 3.0.4 / all 32-bit on Win-10

Ask

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 687
Re: TAChart Fit Series
« Reply #5 on: October 10, 2011, 05:39:51 pm »
Thanks, committed your code with some changes and fixes in r32802, please review.

Some notes:
  • you forgot to register the series and publish Source property
  • ParamCount had default value 2, but was set to 3 by constructor
  • I have rearranged arguments of equation string procedures to allow for default values
  • perhaps the title should be calculated automatically?
  • although numlib uses manual memory allocation, I consider this practice outdated, and prefer to use dynamic arrays
  • you call OnFitComplete before FValidFitParams := true; in ExecFit. Is this intentional?
  • Some parameter combinations caused FP exceptions in Calculate. I have added checks to prevent few of them, but probably not all. Please report if you notice more.

Quote
TFitSeries overrides the SourceChanged method and should run the fit, but it is not called. What is wrong?
Series Listener should be called 'SourceListener' -- it subscribes to the series source, not the chart. Generally, TAChart notification system needs a redesign.
For now, I have fixed the problem in r32083.

Quote
Old story: logarithmic labels
I will get to that later.

wp

  • Hero Member
  • *****
  • Posts: 6444
Re: TAChart Fit Series
« Reply #6 on: October 10, 2011, 11:24:03 pm »
Thank you. Perfect, as usual.

Quote
perhaps the title should be calculated automatically?
You mean the series title? That's a good idea.

Quote
you call OnFitComplete before FValidFitParams := true; in ExecFit. Is this intentional?
Yes. Maybe I am a bit too careful there. But if an exception occurs in the OnFitComplete event handler, I want this variable to be false.

Quote
Series Listener ... subscribes to the series source, not the chart
Where do I see that in the TAChart code?
Lazarus trunk / fpc 3.0.4 / all 32-bit on Win-10

Ask

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 687
Re: TAChart Fit Series
« Reply #7 on: October 11, 2011, 02:43:14 am »
Quote
if an exception occurs in the OnFitComplete event handler, I want this variable to be false.

I see. However, this means you can not call Calculate from OnFitComplete handler.

Quote
Quote
Series Listener ... subscribes to the series source, not the chart
Where do I see that in the TAChart code?
https://github.com/graemeg/lazarus/blob/upstream/components/tachart/tacustomseries.pas#L573
https://github.com/graemeg/lazarus/blob/upstream/components/tachart/tacustomseries.pas#L753

wp

  • Hero Member
  • *****
  • Posts: 6444
Re: TAChart Fit Series
« Reply #8 on: October 11, 2011, 09:29:41 am »
Quote
However, this means you can not call Calculate from OnFitComplete handler.
I see - Calculate uses the ValidFitParams which would not be true here. Well, at the time when the OnFitComplete handler is called the fit calculation is finished anyway. So it really is better to assign FValidFitParams to true before the OnFitComplete handler, just as you suggested.
Lazarus trunk / fpc 3.0.4 / all 32-bit on Win-10