Calculation of Student's t function crashing in LMath

[wp]

Running the following program crashes with the error "Illegal instruction" when calculating the value for x=0.5 (note: package LMath.lpk required).

De-activating the define {$DEFINE USE_LMATH} uses the tDist function from FPC's NumLib, and this works correctly and writes values which can be confirmed by Excel.

Code: Pascal  [Select][+][-]

1. program Project1;
2.  
3. {$DEFINE USE_LMATH}
4.  
5. uses
6.   {$IFDEF USE_LMATH}
7.   uibtdist;
8.   {$ELSE}
9.   spe;
10.   {$ENDIF}
11.  
12. const
13.   xmin = 0.0;
14.   xmax = 3.0;
15.   dx = 0.5;
16.   DF = 10;
17.   s = 2;
18.  
19. var
20.   i: Integer;
21.   x, y: Double;
22. begin
23.   WriteLn('Student''s t distribution with ', DF, ' degrees of freedom (', s, '-sided)');
24.   WriteLn;
25.   WriteLn('x':15, 'y':15);
26.   x := xmin;
27.   while (x <= xmax) do begin
28.     {$IFDEF USE_LMATH}
29.     y := FStudent(DF, x);
30.     {$ELSE}
31.     if x < 0 then
32.       y := 1.0 - tDist(-x, DF, s)
33.     else
34.       y := tDist(x, DF, s);
35.     {$ENDIF}
36.     WriteLn(x:15:5, y:15:5);
37.     x := x + dx;
38.   end;
39.  
40.   ReadLn;
41. end.

[glorfin]

Hi! Strange, but I could not reproduce the bug. The program with {$DEFINE USE_LMATH} worked successfully, here is the output:

Student's t distribution with 10 degrees of freedom (2-sided)

              x              y
        0.00000        0.50000
        0.50000        0.68605
        1.00000        0.82955
        1.50000        0.91775
        2.00000        0.96331
        2.50000        0.98428
        3.00000        0.99333

Could you please give additional information. Platform, compile options of the package LMath.
Could be that you compiled it for instruction set which is not present in your computer?   

[wp]

I used Laz-trunk/FPC-trunk, Laz-trunk/FPC3.2.0, Laz2.10/FPC3.2.0, each 32 bit on Win 10/64bit, Laz-trunlk/FPC3.20 also for 64 bit. Always the same problem. Compilation settings are more or less default:

C:\lazarus-trunk_fpc320\fpc\3.2.0\bin\i386-win32\fpc.exe
-MObjFPC
-Scghi
-O1
-gw2
-godwarfsets
-gl
-l
-vewnhibq
-Filib\i386-win32
-Fu..\..\..\svn\lmath\LMath\Trunk\UMathStat\lib\i386-win32
-Fu..\..\..\svn\lmath\LMath\Trunk\ULineAlgebra\lib\i386-win32
-Fu..\..\..\svn\lmath\LMath\Trunk\uMathUtils\lib\i386-win32
-FuC:\lazarus-trunk_fpc320\lcl\units\i386-win32\win32
-FuC:\lazarus-trunk_fpc320\lcl\units\i386-win32
-FuC:\lazarus-trunk_fpc320\components\freetype\lib\i386-win32
-Fu..\..\..\svn\lmath\LMath\Trunk\UGenMath\lib\i386-win32
-FuC:\lazarus-trunk_fpc320\components\lazutils\lib\i386-win32
-FuC:\lazarus-trunk_fpc320\packager\units\i386-win32
-Fu.
-FUlib\i386-win32
-FE.
-oProject1.exe
-dLCL
-dLCLwin32

Trying to compile the test program of the 1st post in Laz 2.08./FPC3.0.4 (older FPC version) fails with

ucholesk.pas(39,26) Error: Asm: [vsqrtsd xmmreg,xmmreg,mem128] invalid combination of opcode and operands

[glorfin]

Trying to compile the test program of the 1st post in Laz 2.08./FPC3.0.4 (older FPC version) fails with

ucholesk.pas(39,26) Error: Asm: [vsqrtsd xmmreg,xmmreg,mem128] invalid combination of opcode and operands

Hi!
Judging by this message, with [vsqrtsd xmmreg,xmmreg,mem128], package lmLineAlgebra had compile options including use of SSE registers, which could be unsupported in older FPC versions and maybe not present in your system. Compliation options of a package are defined separately from options of a project.

If you are using Lazarus, you can open package file (menu Package -> open Package file -> find LineAlgebra.lpk and open it), open Options in the window of the package, find there "Compilation and Linking", set Optimization level 1, then go to Custom Options, and remove there everything.

Or just update LMath with SVN from trunk and recompile all packages. I have removed these high optimization levels everywhere. Note that in SVN names of package directories changed from uXXXX to more logical lmXXXX, and package lmLineAlgebra renamed to lmLineraAlgebra.

Also, creating new topic in this forum is not a best way to get immediate response from me, I do not visit it every day. Rather, you could use tickets system in LMath.

Best regards,

Viatcheslav

[wp]

No solution.

Now I did a new download of LMath-trunk on a new VM with Win7 (which I installed yesterday for another reason) and a new Laz trunk. Again, the same error.

[marcov]

Long, long ago I had many such problems with DMath (most notably the levenhart-marquardt stuff), and I switched to TPMath.

[glorfin]

Wp: new Lazarus trunk or new LMath trunk? Did you try change compile options of LMath packages?

[wp]

New Lmath-trunk, new Laz trunk, new VM with Windows 7 (32 bit). No changes in compilation settings of LMath. The error message of my t-distribution program is now

Project Project1 raised exception class 'External: SIGILL'.

 At address 41076C

Checked also some of the demos and get the same kind of runtime error in several cases:

console/complex/testcomplex, unit ucomplex:

Code: [Select]

Project testcomp raised exception class 'External: SIGILL'.

 In file 'ucomplex.pas' at line 397:
Result := AbsY * Sqrt(1.0 + Sqr(AbsX / AbsY));

console/curfit/pcatest, unit utrigo

Code: [Select]

Project pcatest raised exception class 'External: SIGILL'.

 In file 'utrigo.pas' at line 41:
Pythag := AbsX * Sqrt(1.0 + Sqr(AbsY / AbsX))
console/proba/binom, unit uibeta

Code: [Select]

Project binom raised exception class 'External: SIGILL'.

 In file 'uibeta.pas' at line 302:
Y := X1 * (A1 + B1 - 2.0) - (A1 - 1.0);
Most (all?) in console/stat 'External: SIGILL'.


Demos in console/integral (convtrap, gauss, test_rkf, trap): contain the old package names (uintegrals and uGenmath)


Package lmath.lpk is lacking the requirement for lmLinearAlgebra,

[VTwin]

Long, long ago I had many such problems with DMath (most notably the levenhart-marquardt stuff), and I switched to TPMath.

Both were maintained by Jean Debord. DMath was updated more recently, 2016-02-06 versus 2013-03-16 for TPMath. I have not compared the two, but have been using DMath because of the update. Were bugs introduced in the update?

[marcov]

Long, long ago I had many such problems with DMath (most notably the levenhart-marquardt stuff), and I switched to TPMath.

Both were maintained by Jean Debord. DMath was updated more recently, 2016-02-06 versus 2013-03-16 for TPMath. I have not compared the two, but have been using DMath because of the update. Were bugs introduced in the update?

I used the older version. Dmath hadn't seen updates in a really long time, and TPMath did.

I only know that I constantly found problems with dmath in the errorchecking department (not all places checked for zero before dividing etc, no guards against overflows, converging less often etc), and TPMath was better there, and much faster, since handling exceptions is slow.

This seems to match this behaviour

But TPmath didn't have levenhard-marquardt, so I used basic linear regression, but in a loop and some code conditionally removing outliers.

[glorfin]

Yes, Demo programs are not yet updated to new package naming.
Problem is that I cannot reproduce this error. May I ask you make one more experiment?
Just copy needed units from LMath to your project and compile it, without packages system?
So we will know at least what causes the problem: pascal code or something with packages?

[wp]

ok. I am attaching a version of my demo program in which the packages are removed and all lmath units required are copied to the project folder. This program works correctly.

I downloaded fresh sources from your svn site once more and noticed suddenly that there are custom options in the package lmGenMath - didn't you say some days ago that you removed them? I wonder why I did not see this yesterday... Anyway, after removing them everything is working.

Nevertheless, I came across another latent issue: Package lmGenMath has LCL in its requirements - is this necessary? Somebody wanting to write a slim console program does not want to pull in the huge LCL. Or somebody writing a program for FreeVision will probably have issues because of duplicate unit names. I found out that LCL is only needed by unit lmMathUtils/uwinstr which has "StdCtrl" in its "uses" clause because of function "ReadNumFromEdit()" having a TEdit as parameter. I think the price for the convenience of using "Edit" as parameter instead of "Edit.Text" is immense. Although changing the parameter type will break existing code the solution for the compilation error is straight-forward.

[glorfin]

Hi!
Yes, remaining custom compilation options in lmGenMath was my oversight, sorry, and thank you for finding it out.

As for LCL dependencies, in lmGenMath it is indeed unnecessary and I removed it. However, such dependency does not pull anything to a program. Basically, adding a dependency on a package into a project or another package, one merely informs the compiler where to look for units which are not found in the project location. Since mechanism of packages exists only in Lazarus, and LCL in Lazarus is always present, this dependency is pretty innocent. Hence, I keep it for lmMathUtil. Again: it does not influence size of console programs.

Best regards!

Viatcheslav