Lazarus

Free Pascal => General => Topic started by: MathMan on May 18, 2020, 08:53:03 am

Title: How to programmatically test for exceptions
Post by: MathMan on May 18, 2020, 08:53:03 am
Hi all,

I am working on a programm where I intentionally throw exceptions under certain circumstances - i.e. range checks, floating point operations, etc., transporting FPC behaviour on standard types to my own non-standard types.

Now I want to test that these exceptions are actually working in the intended way without requesting any user intervention when running the test cycle. From what I have read so far this seems not possible.

The only way to solve this that I found, is defining a special test-mode compile - e.g. {$define TEST_EXCPT} - and at the place where I throw the exception do a manipulation that can be checked by the testing routine. This solves it, but makes code a bit less readable due to added {$ifdef TEST_EXCPT} semantics.

Are there better alternatives to the approach above?

Cheers,
MathMan
Title: Re: How to programmatically test for exceptions
Post by: jamie on May 18, 2020, 01:04:53 pm
And what is wrong with testing your work in production?  :o
Seems to be the norm these days.

Yes of course you create a test app to generate bad data so you can test your code.
Title: Re: How to programmatically test for exceptions
Post by: eljo on May 18, 2020, 01:35:15 pm
which test suit are you using? Most of the test suits have a way to test for exceptions as well as proper results.
Title: Re: How to programmatically test for exceptions
Post by: MathMan on May 18, 2020, 01:46:02 pm
And what is wrong with testing your work in production?  :o
Seems to be the norm these days.

Yes of course you create a test app to generate bad data so you can test your code.

Regarding the first point (if it wasn't meant ironically, which is sometimes hard to detect in written) - maybe just my ambition to develop tested & verified programms, especially if it concerns mathematics.

Regarding the second point - I am not sure I get what you are heading at? It's not that I do not create a testing app - from my perspective it is about how to do that effectively.

Cheers,
MathMan
Title: Re: How to programmatically test for exceptions
Post by: MathMan on May 18, 2020, 01:49:55 pm
which test suit are you using? Most of the test suits have a way to test for exceptions as well as proper results.

Thanks for the hint eljo. At the moment I plan to develope my own test suite. But if existing test suites can handle exceptions then I am more than willing to give it a try.

Can you recommend something? Must be FOSS though, as I am doing all this for my personal pleasure without any commercial intention.

Cheers,
MathMan
Title: Re: How to programmatically test for exceptions
Post by: Thaddy on May 18, 2020, 01:54:20 pm
I am working on a programm where I intentionally throw exceptions under certain circumstances - i.e. range checks, floating point operations, etc., transporting FPC behaviour on standard types to my own non-standard types.
The compiler does that already for you. This is not clear to me. What do you want to achieve? Be more precise.
I could give an example if you want, but you are not exactly a noob. The example would be brief and short.

(btw: these kind of exceptions are indeed  caught, but are expensive if they happen too often)

For all the mentioned Exceptions there are pre-programmed Exxx in sysutils.pas and of course you have the compiler options.
Hence my question? Is there anything lacking? Can it be a case of blinded by knowledge? (without offence)
Title: Re: How to programmatically test for exceptions
Post by: MathMan on May 18, 2020, 02:06:18 pm
I am working on a programm where I intentionally throw exceptions under certain circumstances - i.e. range checks, floating point operations, etc., transporting FPC behaviour on standard types to my own non-standard types.
The compiler does that already for you. This is not clear to me. What do you want to achieve? Be more precise.
I could give an example if you want, but you are not exactly a noob. The example would be brief and short.

(btw: these kind of exceptions are indeed  caught, but are expensive if they happen too often)

The compiler will catch things like the following at compile-time

Code: Pascal  [Select][+][-]
  1. var
  2.   a: byte;
  3.   b: shortint;
  4.  
  5.   a := 256;
  6.   b := -128; a := b;
  7.   a := b div 0;
  8.  

With {$R+} also run-time misbehaviour is excepted <= this is what I want to implement and test for derived types like int128, int256 (first) and fp128, fp256 (later).
Title: Re: How to programmatically test for exceptions
Post by: Thaddy on May 18, 2020, 02:09:47 pm
In that case,this is more clear to me, maybe ranges would help? The compiler does a great job in guarding ranges.

(btw where has all your website code gone since 2020... :P )
Title: Re: How to programmatically test for exceptions
Post by: PascalDragon on May 18, 2020, 04:18:18 pm
which test suit are you using? Most of the test suits have a way to test for exceptions as well as proper results.

Thanks for the hint eljo. At the moment I plan to develope my own test suite. But if existing test suites can handle exceptions then I am more than willing to give it a try.

Can you recommend something? Must be FOSS though, as I am doing all this for my personal pleasure without any commercial intention.

FPC ships with fpcunit (https://wiki.lazarus.freepascal.org/fpcunit) and there is also a Lazarus package to create a testsuite that can display the results in a GUI frontend.
Title: Re: How to programmatically test for exceptions
Post by: eljo on May 18, 2020, 04:25:38 pm
which test suit are you using? Most of the test suits have a way to test for exceptions as well as proper results.

Thanks for the hint eljo. At the moment I plan to develope my own test suite. But if existing test suites can handle exceptions then I am more than willing to give it a try.

Can you recommend something? Must be FOSS though, as I am doing all this for my personal pleasure without any commercial intention.

Cheers,
MathMan
FPCUnit already mentioned and my preferred suit is https://github.com/graemeg/fptest (https://github.com/graemeg/fptest) both add extra application types in lazarus you can use them by selecting file\New.. and the dialog that pops FPCUnit and FPTest Applications.
Title: Re: How to programmatically test for exceptions
Post by: avra on May 18, 2020, 06:10:05 pm
types like int128, int256 (first) and fp128, fp256 (later).
+1  ;)
Title: Re: How to programmatically test for exceptions
Post by: MathMan on May 19, 2020, 07:45:51 am
Maybe my first initial question wasn't formulated clear enough (thanks to Thaddy for making me aware). So I created a very limited project to clarify my point.

The project (console app) can be compiled without range checks, with range checks enabled but commented out {$define TEST_EXCPT} or active {$define TEST_EXCPT}.

If run, then in the first case it simply executes. In the second case it generates a range check error on the first fault test, but terminates thereafter. In the third case it executes both faulty tests and prints that it has a range check error detected.

I would like to test everything in a single run without having to resort to the use of {$define TEST_EXCPT}.

Hope that clarifies my requirements.

Cheers,
MathMan
Title: Re: How to programmatically test for exceptions
Post by: MathMan on May 19, 2020, 07:49:10 am
which test suit are you using? Most of the test suits have a way to test for exceptions as well as proper results.

Thanks for the hint eljo. At the moment I plan to develope my own test suite. But if existing test suites can handle exceptions then I am more than willing to give it a try.

Can you recommend something? Must be FOSS though, as I am doing all this for my personal pleasure without any commercial intention.

Cheers,
MathMan
FPCUnit already mentioned and my preferred suit is https://github.com/graemeg/fptest (https://github.com/graemeg/fptest) both add extra application types in lazarus you can use them by selecting file\New.. and the dialog that pops FPCUnit and FPTest Applications.

Thanks - I am willing to give both a try. If you look at my previous post (with the project attached) - do you know if I they can handle my requirement?
Title: Re: How to programmatically test for exceptions
Post by: MathMan on May 19, 2020, 07:53:52 am
which test suit are you using? Most of the test suits have a way to test for exceptions as well as proper results.

Thanks for the hint eljo. At the moment I plan to develope my own test suite. But if existing test suites can handle exceptions then I am more than willing to give it a try.

Can you recommend something? Must be FOSS though, as I am doing all this for my personal pleasure without any commercial intention.

FPC ships with fpcunit (https://wiki.lazarus.freepascal.org/fpcunit) and there is also a Lazarus package to create a testsuite that can display the results in a GUI frontend.

Thanks for the feedback, I will look into this.

While I am pondering with the design of the program - if I do it thoroughly I will have to create a lot (>100 at least) of operator overloads to catch all variants. Must I worry about some compiler internal limits?
Title: Re: How to programmatically test for exceptions
Post by: eljo on May 19, 2020, 02:14:28 pm
Thanks - I am willing to give both a try. If you look at my previous post (with the project attached) - do you know if I they can handle my requirement?
Both can handle almost everything. But in your case I'm attaching two projects in this message. a fpcunit based one with two different methods of testing and an FPTest one with only one of the testing methods just to demonstrate how similar the two suits are.
Title: Re: How to programmatically test for exceptions
Post by: MathMan on May 20, 2020, 08:50:26 am
Thanks - I am willing to give both a try. If you look at my previous post (with the project attached) - do you know if I they can handle my requirement?
Both can handle almost everything. But in your case I'm attaching two projects in this message. a fpcunit based one with two different methods of testing and an FPTest one with only one of the testing methods just to demonstrate how similar the two suits are.

Thanks eljo, much appreciated - especially taking the time to setup two sample test projects.

I already downloaded fptest and will go forward with that I think.
TinyPortal © 2005-2018