Print Page - LazStats - A new statistics package

Announcements => Third party => Topic started by: wgmiller on January 19, 2010, 06:31:14 pm

Title: LazStats - A new statistics package
Post by: wgmiller on January 19, 2010, 06:31:14 pm

I have converted a previous Delphi program to Lazarus. It currently is only compiled for Windows and can be obtained at:
http://statpages.org/miller/LazStats
I do not have Linux or other platforms to attempt to compile it for these other platforms. If any of you are willing, I would appreciate your participation.

Title: Re: LazStats - A new statistics package
Post by: Troodon on January 19, 2010, 07:04:01 pm

Impressive work!

Quote from: wgmiller on January 19, 2010, 06:31:14 pm

I do not have Linux or other platforms to attempt to compile it for these other platforms. If any of you are willing, I would appreciate your participation.

To develop for Linux, you can install VirtualBox for Windows as host and Linux Ubuntu as client. They are both available free of charge.

Title: Re: LazStats - A new statistics package
Post by: kamischi on January 19, 2010, 08:03:39 pm

I had a quick try on Mc OS X: The windows unit is not available. According to this part of the lazarus wiki conversion page:

http://wiki.freepascal.org/Code_Conversion_Guide#Converting_Delphi_projects.2Fforms.2Funits_to_Lazarus

I replaced the windows unit by LCLIntf.

Also, the case sensitivity of the filesystem gives problems. BLKANOVAUNIT.lrs and many more .lrs files. After correcting them i could get the program to start. As usual, the positions of texts and boxes need adjustments. Still some more warnings and stuff, but good enough for a screen shot.

Mischi.

Title: Re: LazStats - A new statistics package
Post by: crorden on January 20, 2010, 02:44:32 am

Fantastic. OpenStat is a terrific tool. LazStats looks great. This looks like a very mature conversion - you already have the graphs and charts looking great.

Title: Re: LazStats - A new statistics package
Post by: Troodon on January 20, 2010, 04:46:32 am

Quote from: wgmiller on January 19, 2010, 06:31:14 pm

It currently is only compiled for Windows

You may wish to follow the suggestions in this thread http://www.lazarus.freepascal.org/index.php/topic,6244.0.html to minimize the size of your executable (I got around 4.2 MB on win32 and 5.4 MB on lnx32).

Please consider adding LazStats to the Lazarus application gallery http://wiki.freepascal.org/Lazarus_Application_Gallery (BTW, perhaps the Lazarus and FPC galleries should be merged).

Title: Re: LazStats - A new statistics package
Post by: wgmiller on January 23, 2010, 01:18:49 am

Chris Rorden has compiled versions for Linux (32 bit and 64 bit) as well as Mac OSX. He also "cleaned up" the code and made valuable suggestions. You can download these at
http://statpages.org/miller/LazStats/

Title: Re: LazStats - A new statistics package
Post by: tatamata on January 29, 2020, 10:25:00 am

Hello, is LazStats still actively maintained by mr. Miller? Are there any recent forks or similar developments?
Is there any Lazarus software dealing specifically with DOE (Design Of Experiments)?
Are there any books or other materials covering specifically how to use LazStats for DOE?

Title: Re: LazStats - A new statistics package
Post by: marcov on January 29, 2020, 11:03:33 am

Quote from: tatamata on January 29, 2020, 10:25:00 am

Hello, is LazStats still actively maintained by mr. Miller? Are there any recent forks or similar developments?
Is there any Lazarus software dealing specifically with DOE (Design Of Experiments)?
Are there any books or other materials covering specifically how to use LazStats for DOE?

A newer url seems to be https://openstat.info/LazStatsMain.htm

Title: Re: LazStats - A new statistics package
Post by: wp on January 29, 2020, 11:14:40 am

Quote from: tatamata on January 29, 2020, 10:25:00 am

Hello, is LazStats still actively maintained by mr. Miller?

Does not look like that, even the link in the previous post is not working any more. I finally could download the Windows (*) sources from https://openstat.info/LazStatsMain.htm. Compilation did work after some changes (replaced GraphFrm.AutoScale by .AutoScaled due to a naming conflict), but at runtime I get a grid error that "FixedRows can't be > RowCount." which can be resolved by setting "DictionaryFrm.DictGrid.RowCount := DictionaryFrm.DictGrid.FixedRows;" in TOS3MainFrm.FormShow, unit MainUnit.

Quote from: tatamata on January 29, 2020, 10:25:00 am

Is there any Lazarus software dealing specifically with DOE (Design Of Experiments)?

I don't think so. Isn't it true that DOE in the end comes down to linear regression? In this case you could try the fitting routins built into TAChart. Or try DMath available via Online Package Manager, or JediMath (https://sourceforge.net/projects/jedimath/). Some forms require some redesign.

(*) The windows unit is used by some units. In all of them it can be removed. There is a variable named RichEdit in the code but this already has been replaced by a TMemo.

Title: Re: LazStats - A new statistics package
Post by: tatamata on March 29, 2020, 11:20:11 am

I think there is a possible bug in LazStats BlkANOVAUnit or ANOVATestUnit, namely it seems that Post-Hoc comparisons are not working with One-Way ANOVA in LazStats.
I couldn't figure out what is the reason, can somebody please take a look?

Test dataset is here:
https://drive.google.com/file/d/1qsJE1sOHFYYkizcK7W0nbiTgpAQK2dUn/view?usp=sharing
"Etch Rate (A/min)" is a dependent response variable, while "RF Power (W)" is independent input variable.

My LazStats source code (CodeTyphon version) is here:
https://drive.google.com/file/d/1FnAZqaZrhwBYgr9luYSytCucRhi_nmmt/view?usp=sharing

Could somebody please test it?

Title: Re: LazStats - A new statistics package
Post by: jwdietrich on March 29, 2020, 11:35:15 am

Quote from: wgmiller on January 23, 2010, 01:18:49 am

Chris Rorden has compiled versions for Linux (32 bit and 64 bit) as well as Mac OSX. He also "cleaned up" the code and made valuable suggestions. You can download these at
http://statpages.org/miller/LazStats/

Unfortunately, this URL doesn't seem to be correct. Under the URL posted by @marcov I can't find the source code for macOS. Is there a source code repository (VCS, SVN, git etc.) available?

Title: Re: LazStats - A new statistics package
Post by: Bart on March 29, 2020, 12:13:13 pm

The code (found at https://openstat.info/LazStatsMain.htm) is GPL (at least that is what license.lfm says), so we could host it on lazarus-ccr?

Bart

Title: Re: LazStats - A new statistics package
Post by: wp on March 29, 2020, 12:13:50 pm

Quote from: jwdietrich on March 29, 2020, 11:35:15 am

Quote from: wgmiller on January 23, 2010, 01:18:49 am
Chris Rorden has compiled versions for Linux (32 bit and 64 bit) as well as Mac OSX. He also "cleaned up" the code and made valuable suggestions. You can download these at
http://statpages.org/miller/LazStats/
Unfortunately, this URL doesn't seem to be correct. Under the URL posted by @marcov I can't find the source code for macOS. Is there a source code repository (VCS, SVN, git etc.) available?

I have begun porting my own version, see https://github.com/wp-xyz/LazStats. All forms have consistent layout now, independent of widgetset. I have begun refactoring the code, adding validation of input parameters to avoid the frequent crashes of the program, checking the examples, rebuilding help files etc. With 100+ forms this will keep me busy for a while... The program must have been thesis for quite a lot of students.

@Bart: I could move it to ccr/applications...

Title: Re: LazStats - A new statistics package
Post by: tatamata on March 30, 2020, 10:00:27 am

Quote from: wp on March 29, 2020, 12:13:50 pm

Quote from: jwdietrich on March 29, 2020, 11:35:15 am
Quote from: wgmiller on January 23, 2010, 01:18:49 am
Chris Rorden has compiled versions for Linux (32 bit and 64 bit) as well as Mac OSX. He also "cleaned up" the code and made valuable suggestions. You can download these at
http://statpages.org/miller/LazStats/
Unfortunately, this URL doesn't seem to be correct. Under the URL posted by @marcov I can't find the source code for macOS. Is there a source code repository (VCS, SVN, git etc.) available?
I have begun porting my own version, see https://github.com/wp-xyz/LazStats. All forms have consistent layout now, independent of widgetset. I have begun refactoring the code, adding validation of input parameters to avoid the frequent crashes of the program, checking the examples, rebuilding help files etc. With 100+ forms this will keep me busy for a while... The program must have been thesis for quite a lot of students.

@Bart: I could move it to ccr/applications...

Great!
For your info, there is a bug in LSMRUnit.

Code: Pascal [Select][+]

    { get independendent variable columns }
     for i := 0 to IndepVars.Count-1 do
     begin
         {
          cellstring := OS3Mainfrm.DataGrid.Cells[i+1,0];   //This is a bug!
         }
         cellstring := IndepVars.Items.Strings[i];   //Bugfix 
         for j := 1 to NoVariables do
         begin
             if cellstring = OS3MainFrm.DataGrid.Cells[j,0] then
             begin
                IndepVarsCols[i] := j;
                ColEntered[i] := j;
                NEntered := NEntered + 1;
                IndRowLabels[NEntered-1] := cellstring;
                IndColLabels[NEntered-1] := cellstring;
             end;
         end;
     end;                                                    
 

Title: Re: LazStats - A new statistics package
Post by: wp on March 30, 2020, 11:16:41 am

Thanks. I just refactored the unit to my current standard. But I cannot find the position in the code, there is no comment "get independent variable columns" in my copy, only a "get dependent variaible columns". Except for this difference, a few lines later there are lines which seem to match. Just to make sure: What is the name of the procedure? TLSMregForm.ComputeBtnClick? Near the beginning after the bunch of SetLength instructions?

Did you happen to find the help file related to this form? The pdf usually contains a tested example of OpenStat which allows me to check the routine.

If there is not help file and no demo can you submit a sample file with the expected results?

Title: Re: LazStats - A new statistics package
Post by: tatamata on March 30, 2020, 11:31:10 am

Quote from: tatamata on March 29, 2020, 11:20:11 am

I think there is a possible bug in LazStats BlkANOVAUnit or ANOVATestUnit, namely it seems that Post-Hoc comparisons are not working with One-Way ANOVA in LazStats.
I couldn't figure out what is the reason, can somebody please take a look?

Test dataset is here:
https://drive.google.com/file/d/1qsJE1sOHFYYkizcK7W0nbiTgpAQK2dUn/view?usp=sharing
"Etch Rate (A/min)" is a dependent response variable, while "RF Power (W)" is independent input variable.

My LazStats source code (CodeTyphon version) is here:
https://drive.google.com/file/d/1FnAZqaZrhwBYgr9luYSytCucRhi_nmmt/view?usp=sharing

Could somebody please test it?

Attached screenshot of similar output from JASP software, with Post-Hoc comparisons - these are expected results...

Title: Re: LazStats - A new statistics package
Post by: tatamata on March 30, 2020, 11:45:21 am

Quote from: wp on March 30, 2020, 11:16:41 am

Thanks. I just refactored the unit to my current standard. But I cannot find the position in the code, there is no comment "get independent variable columns" in my copy, only a "get dependent variaible columns". Except for this difference, a few lines later there are lines which seem to match. Just to make sure: What is the name of the procedure? TLSMregForm.ComputeBtnClick? Near the beginning after the bunch of SetLength instructions?

Did you happen to find the help file related to this form? The pdf usually contains a tested example of OpenStat which allows me to check the routine.

If there is not help file and no demo can you submit a sample file with the expected results?

it's in procedure TLSMregForm.ComputeBtnClick(Sender: TObject);
yeah, comment was added by me...
you can use the file (9_23_results.tab), link here: https://drive.google.com/file/d/1qsJE1sOHFYYkizcK7W0nbiTgpAQK2dUn/view?usp=sharing.
Result of least squares multiple regression should be:
"
Least Squares Multiple Regression by Bill Miller

Product-Moment Correlations Matrix with 20 cases.

Variables
RF Power (W)Etch Rate (A/min)
RF Power (W) 1,000 0,940
Etch Rate (A/min) 0,940 1,000

Means with 20 valid cases.

Variables RF Power (W)Etch Rate (A/min)
190,000 617,750

Standard Deviations with 20 valid cases.

Variables RF Power (W)Etch Rate (A/min)
22,942 61,648

Variance Y = 3800,513 SSY = 7704511,000 SDY = 61,648

Least Squares Multiple Regression by Bill Miller

Product-Moment Correlations Matrix with 20 cases.

Variables
RF Power (W)Etch Rate (A/min)
RF Power (W) 1,000 0,940
Etch Rate (A/min) 0,940 1,000

Means with 20 valid cases.

Variables RF Power (W)Etch Rate (A/min)
190,000 617,750

Standard Deviations with 20 valid cases.

Variables RF Power (W)Etch Rate (A/min)
22,942 61,648

Variance Y = 3800,513 SSY = 7704511,000 SDY = 61,648

MEANS with 20 valid cases.

Variables RF Power (W) Intercept
190,000 1,000

VARIANCES with 20 valid cases.

Variables RF Power (W) Intercept
526,316 0,000

STD. DEV.S with 20 valid cases.

Variables RF Power (W) Intercept
22,942 0,000

SSY = 72209,75, SSreg = 63857,29, SSres = 8352,46
R2 = 0,8843, F = 137,62, D.F. = 1 18, Prob>F = 0,0000
Standard Error of Estimate = 21,54

Variable Beta B Std.Err. t prob VIF TOL
RF Power (W) 0,940 2,527 0,215 11,731 0,000 1,000 1,000
Intercept 0,000 137,620 41,211 3,339 0,004 20,000 0,050

SOURCE DF SS MS F Prob.>F
Regression 1 63857,290 63857,290 137,6159
Residual 19 8352,460 464,026
Total 19 72209,750

R2 = 0,8843, F = 137,62, D.F. = 1 18 Prob.>F = 0,0000
Adjusted R2 = 0,8779
Standard Error of Estimate = 21,54

Variable Beta B Std.Err. t prob VIF TOL
RF Power (W) 0,940 2,527 0,215 11,731 0,000 1,000 1,000
Intercept 0,000 137,620 41,211 3,339 0,004 20,000 0,050

SOURCE DF SS MS F Prob.>F
Regression 1 63857,290 63857,290 137,6159
Residual 19 8352,460 464,026
Total 19 72209,750

R2 = 0,8843, F = 137,62, D.F. = 1 18 Prob.>F = 0,0000
Adjusted R2 = 0,8779
Standard Error of Estimate = 21,54
Entry requirements met

Title: Re: LazStats - A new statistics package
Post by: wp on March 30, 2020, 01:27:29 pm

Thank you. Now I understand why your change was necessary. I added your file to the data folder (as LSMR.laz).

Title: Re: LazStats - A new statistics package
Post by: tatamata on March 30, 2020, 02:36:10 pm

Regarding post-hoc comparisons (not working with one-way ANOVA properly),
you can test it with the same input file.
Expected results should be something like these (see Scheffe and Tukey columns) from JASP software (screenshot attached).

Title: Re: LazStats - A new statistics package
Post by: Bart on March 30, 2020, 02:51:30 pm

Quote from: wp on March 29, 2020, 12:13:50 pm

@Bart: I could move it to ccr/applications...

I think that would be nice.
It makes contributing much easier IMO.

Bart

Title: Re: LazStats - A new statistics package
Post by: wp on March 30, 2020, 11:27:41 pm

OK - I moved my LazStats adaption now to CCR. It is in folder applications/lazstats. Download via SVN or create a zipped snapshot: https://sourceforge.net/p/lazarus-ccr/svn/HEAD/tree/applications/lazstats/

Title: Re: LazStats - A new statistics package
Post by: af0815 on March 30, 2020, 11:39:13 pm

I think it would be nice to include the license statement in the ReadMe file. So it is clear to have a GPL'ed project and units

Title: Re: LazStats - A new statistics package
Post by: wp on March 31, 2020, 12:30:31 am

~~I doubt that the application is GPL - this is not mentioned anywhere in the files (as far as I could see). In the documentation file "Introduction.pdf", Bill writes, however:~~

Quote

"While I reserve the copyright protection of these packages, I make no restriction on their distribution or use. It is common courtesy, of course, to give me credit if you use these resources. Because I do not warrant them in any manner, you should insure yourself that the routines you use are adequate for your purposes."

This text is now contained in the README file.

I had contacted Bill by mail already some time ago, but did not get a response.

Sorry for the confusion. Bart had referred to the license.lfm, and this is where GPL2+ is clearly mentioned.