How to programmatically test for exceptions

[MathMan]

Hi all,

I am working on a programm where I intentionally throw exceptions under certain circumstances - i.e. range checks, floating point operations, etc., transporting FPC behaviour on standard types to my own non-standard types.

Now I want to test that these exceptions are actually working in the intended way without requesting any user intervention when running the test cycle. From what I have read so far this seems not possible.

The only way to solve this that I found, is defining a special test-mode compile - e.g. {$define TEST_EXCPT} - and at the place where I throw the exception do a manipulation that can be checked by the testing routine. This solves it, but makes code a bit less readable due to added {$ifdef TEST_EXCPT} semantics.

Are there better alternatives to the approach above?

Cheers,
MathMan

[jamie]

And what is wrong with testing your work in production? 
Seems to be the norm these days.

Yes of course you create a test app to generate bad data so you can test your code.

[eljo]

which test suit are you using? Most of the test suits have a way to test for exceptions as well as proper results.

[MathMan]

And what is wrong with testing your work in production? 
Seems to be the norm these days.

Yes of course you create a test app to generate bad data so you can test your code.

Regarding the first point (if it wasn't meant ironically, which is sometimes hard to detect in written) - maybe just my ambition to develop tested & verified programms, especially if it concerns mathematics.

Regarding the second point - I am not sure I get what you are heading at? It's not that I do not create a testing app - from my perspective it is about how to do that effectively.

Cheers,
MathMan

[MathMan]

which test suit are you using? Most of the test suits have a way to test for exceptions as well as proper results.

Thanks for the hint eljo. At the moment I plan to develope my own test suite. But if existing test suites can handle exceptions then I am more than willing to give it a try.

Can you recommend something? Must be FOSS though, as I am doing all this for my personal pleasure without any commercial intention.

Cheers,
MathMan

[Thaddy]

I am working on a programm where I intentionally throw exceptions under certain circumstances - i.e. range checks, floating point operations, etc., transporting FPC behaviour on standard types to my own non-standard types.

The compiler does that already for you. This is not clear to me. What do you want to achieve? Be more precise.
I could give an example if you want, but you are not exactly a noob. The example would be brief and short.

(btw: these kind of exceptions are indeed  caught, but are expensive if they happen too often)

For all the mentioned Exceptions there are pre-programmed Exxx in sysutils.pas and of course you have the compiler options.
Hence my question? Is there anything lacking? Can it be a case of blinded by knowledge? (without offence)

[MathMan]

I am working on a programm where I intentionally throw exceptions under certain circumstances - i.e. range checks, floating point operations, etc., transporting FPC behaviour on standard types to my own non-standard types.

The compiler does that already for you. This is not clear to me. What do you want to achieve? Be more precise.
I could give an example if you want, but you are not exactly a noob. The example would be brief and short.

(btw: these kind of exceptions are indeed  caught, but are expensive if they happen too often)

The compiler will catch things like the following at compile-time

Code: Pascal  [Select][+][-]

1. var
2.   a: byte;
3.   b: shortint;
4.  
5.   a := 256;
6.   b := -128; a := b;
7.   a := b div 0;
8.  

With {$R+} also run-time misbehaviour is excepted <= this is what I want to implement and test for derived types like int128, int256 (first) and fp128, fp256 (later).

[Thaddy]

In that case,this is more clear to me, maybe ranges would help? The compiler does a great job in guarding ranges.

(btw where has all your website code gone since 2020... )

[PascalDragon]

which test suit are you using? Most of the test suits have a way to test for exceptions as well as proper results.

Thanks for the hint eljo. At the moment I plan to develope my own test suite. But if existing test suites can handle exceptions then I am more than willing to give it a try.

Can you recommend something? Must be FOSS though, as I am doing all this for my personal pleasure without any commercial intention.

FPC ships with fpcunit and there is also a Lazarus package to create a testsuite that can display the results in a GUI frontend.

[eljo]

which test suit are you using? Most of the test suits have a way to test for exceptions as well as proper results.

Thanks for the hint eljo. At the moment I plan to develope my own test suite. But if existing test suites can handle exceptions then I am more than willing to give it a try.

Can you recommend something? Must be FOSS though, as I am doing all this for my personal pleasure without any commercial intention.

Cheers,
MathMan

FPCUnit already mentioned and my preferred suit is https://github.com/graemeg/fptest both add extra application types in lazarus you can use them by selecting file\New.. and the dialog that pops FPCUnit and FPTest Applications.

[avra]

types like int128, int256 (first) and fp128, fp256 (later).

+1 

[MathMan]

Maybe my first initial question wasn't formulated clear enough (thanks to Thaddy for making me aware). So I created a very limited project to clarify my point.

The project (console app) can be compiled without range checks, with range checks enabled but commented out {$define TEST_EXCPT} or active {$define TEST_EXCPT}.

If run, then in the first case it simply executes. In the second case it generates a range check error on the first fault test, but terminates thereafter. In the third case it executes both faulty tests and prints that it has a range check error detected.

I would like to test everything in a single run without having to resort to the use of {$define TEST_EXCPT}.

Hope that clarifies my requirements.

Cheers,
MathMan

[MathMan]

which test suit are you using? Most of the test suits have a way to test for exceptions as well as proper results.

Thanks for the hint eljo. At the moment I plan to develope my own test suite. But if existing test suites can handle exceptions then I am more than willing to give it a try.

Can you recommend something? Must be FOSS though, as I am doing all this for my personal pleasure without any commercial intention.

Cheers,
MathMan

FPCUnit already mentioned and my preferred suit is https://github.com/graemeg/fptest both add extra application types in lazarus you can use them by selecting file\New.. and the dialog that pops FPCUnit and FPTest Applications.

Thanks - I am willing to give both a try. If you look at my previous post (with the project attached) - do you know if I they can handle my requirement?

[MathMan]

which test suit are you using? Most of the test suits have a way to test for exceptions as well as proper results.

Thanks for the hint eljo. At the moment I plan to develope my own test suite. But if existing test suites can handle exceptions then I am more than willing to give it a try.

Can you recommend something? Must be FOSS though, as I am doing all this for my personal pleasure without any commercial intention.

FPC ships with fpcunit and there is also a Lazarus package to create a testsuite that can display the results in a GUI frontend.

Thanks for the feedback, I will look into this.

While I am pondering with the design of the program - if I do it thoroughly I will have to create a lot (>100 at least) of operator overloads to catch all variants. Must I worry about some compiler internal limits?

[eljo]

Thanks - I am willing to give both a try. If you look at my previous post (with the project attached) - do you know if I they can handle my requirement?

Both can handle almost everything. But in your case I'm attaching two projects in this message. a fpcunit based one with two different methods of testing and an FPTest one with only one of the testing methods just to demonstrate how similar the two suits are.