Recent

Author Topic: How to programmatically test for exceptions  (Read 1498 times)

MathMan

  • Full Member
  • ***
  • Posts: 205
How to programmatically test for exceptions
« on: May 18, 2020, 08:53:03 am »
Hi all,

I am working on a programm where I intentionally throw exceptions under certain circumstances - i.e. range checks, floating point operations, etc., transporting FPC behaviour on standard types to my own non-standard types.

Now I want to test that these exceptions are actually working in the intended way without requesting any user intervention when running the test cycle. From what I have read so far this seems not possible.

The only way to solve this that I found, is defining a special test-mode compile - e.g. {$define TEST_EXCPT} - and at the place where I throw the exception do a manipulation that can be checked by the testing routine. This solves it, but makes code a bit less readable due to added {$ifdef TEST_EXCPT} semantics.

Are there better alternatives to the approach above?

Cheers,
MathMan

jamie

  • Hero Member
  • *****
  • Posts: 3543
Re: How to programmatically test for exceptions
« Reply #1 on: May 18, 2020, 01:04:53 pm »
And what is wrong with testing your work in production?  :o
Seems to be the norm these days.

Yes of course you create a test app to generate bad data so you can test your code.
The only true wisdom is knowing you know nothing

eljo

  • Sr. Member
  • ****
  • Posts: 408
Re: How to programmatically test for exceptions
« Reply #2 on: May 18, 2020, 01:35:15 pm »
which test suit are you using? Most of the test suits have a way to test for exceptions as well as proper results.

MathMan

  • Full Member
  • ***
  • Posts: 205
Re: How to programmatically test for exceptions
« Reply #3 on: May 18, 2020, 01:46:02 pm »
And what is wrong with testing your work in production?  :o
Seems to be the norm these days.

Yes of course you create a test app to generate bad data so you can test your code.

Regarding the first point (if it wasn't meant ironically, which is sometimes hard to detect in written) - maybe just my ambition to develop tested & verified programms, especially if it concerns mathematics.

Regarding the second point - I am not sure I get what you are heading at? It's not that I do not create a testing app - from my perspective it is about how to do that effectively.

Cheers,
MathMan

MathMan

  • Full Member
  • ***
  • Posts: 205
Re: How to programmatically test for exceptions
« Reply #4 on: May 18, 2020, 01:49:55 pm »
which test suit are you using? Most of the test suits have a way to test for exceptions as well as proper results.

Thanks for the hint eljo. At the moment I plan to develope my own test suite. But if existing test suites can handle exceptions then I am more than willing to give it a try.

Can you recommend something? Must be FOSS though, as I am doing all this for my personal pleasure without any commercial intention.

Cheers,
MathMan

Thaddy

  • Hero Member
  • *****
  • Posts: 10492
Re: How to programmatically test for exceptions
« Reply #5 on: May 18, 2020, 01:54:20 pm »
I am working on a programm where I intentionally throw exceptions under certain circumstances - i.e. range checks, floating point operations, etc., transporting FPC behaviour on standard types to my own non-standard types.
The compiler does that already for you. This is not clear to me. What do you want to achieve? Be more precise.
I could give an example if you want, but you are not exactly a noob. The example would be brief and short.

(btw: these kind of exceptions are indeed  caught, but are expensive if they happen too often)

For all the mentioned Exceptions there are pre-programmed Exxx in sysutils.pas and of course you have the compiler options.
Hence my question? Is there anything lacking? Can it be a case of blinded by knowledge? (without offence)
« Last Edit: May 18, 2020, 02:07:30 pm by Thaddy »
When you ask a question that is actually answered in the documentation, you are either lazy or a moron.

MathMan

  • Full Member
  • ***
  • Posts: 205
Re: How to programmatically test for exceptions
« Reply #6 on: May 18, 2020, 02:06:18 pm »
I am working on a programm where I intentionally throw exceptions under certain circumstances - i.e. range checks, floating point operations, etc., transporting FPC behaviour on standard types to my own non-standard types.
The compiler does that already for you. This is not clear to me. What do you want to achieve? Be more precise.
I could give an example if you want, but you are not exactly a noob. The example would be brief and short.

(btw: these kind of exceptions are indeed  caught, but are expensive if they happen too often)

The compiler will catch things like the following at compile-time

Code: Pascal  [Select][+][-]
  1. var
  2.   a: byte;
  3.   b: shortint;
  4.  
  5.   a := 256;
  6.   b := -128; a := b;
  7.   a := b div 0;
  8.  

With {$R+} also run-time misbehaviour is excepted <= this is what I want to implement and test for derived types like int128, int256 (first) and fp128, fp256 (later).

Thaddy

  • Hero Member
  • *****
  • Posts: 10492
Re: How to programmatically test for exceptions
« Reply #7 on: May 18, 2020, 02:09:47 pm »
In that case,this is more clear to me, maybe ranges would help? The compiler does a great job in guarding ranges.

(btw where has all your website code gone since 2020... :P )
« Last Edit: May 18, 2020, 02:12:01 pm by Thaddy »
When you ask a question that is actually answered in the documentation, you are either lazy or a moron.

PascalDragon

  • Hero Member
  • *****
  • Posts: 2156
  • Compiler Developer
Re: How to programmatically test for exceptions
« Reply #8 on: May 18, 2020, 04:18:18 pm »
which test suit are you using? Most of the test suits have a way to test for exceptions as well as proper results.

Thanks for the hint eljo. At the moment I plan to develope my own test suite. But if existing test suites can handle exceptions then I am more than willing to give it a try.

Can you recommend something? Must be FOSS though, as I am doing all this for my personal pleasure without any commercial intention.

FPC ships with fpcunit and there is also a Lazarus package to create a testsuite that can display the results in a GUI frontend.

eljo

  • Sr. Member
  • ****
  • Posts: 408
Re: How to programmatically test for exceptions
« Reply #9 on: May 18, 2020, 04:25:38 pm »
which test suit are you using? Most of the test suits have a way to test for exceptions as well as proper results.

Thanks for the hint eljo. At the moment I plan to develope my own test suite. But if existing test suites can handle exceptions then I am more than willing to give it a try.

Can you recommend something? Must be FOSS though, as I am doing all this for my personal pleasure without any commercial intention.

Cheers,
MathMan
FPCUnit already mentioned and my preferred suit is https://github.com/graemeg/fptest both add extra application types in lazarus you can use them by selecting file\New.. and the dialog that pops FPCUnit and FPTest Applications.

avra

  • Hero Member
  • *****
  • Posts: 2005
    • Additional info
Re: How to programmatically test for exceptions
« Reply #10 on: May 18, 2020, 06:10:05 pm »
types like int128, int256 (first) and fp128, fp256 (later).
+1  ;)
ct2laz - Conversion between Lazarus and CodeTyphon
bithelpers - Bit manipulation for standard types
pasettimino - Siemens S7 PLC lib

MathMan

  • Full Member
  • ***
  • Posts: 205
Re: How to programmatically test for exceptions
« Reply #11 on: May 19, 2020, 07:45:51 am »
Maybe my first initial question wasn't formulated clear enough (thanks to Thaddy for making me aware). So I created a very limited project to clarify my point.

The project (console app) can be compiled without range checks, with range checks enabled but commented out {$define TEST_EXCPT} or active {$define TEST_EXCPT}.

If run, then in the first case it simply executes. In the second case it generates a range check error on the first fault test, but terminates thereafter. In the third case it executes both faulty tests and prints that it has a range check error detected.

I would like to test everything in a single run without having to resort to the use of {$define TEST_EXCPT}.

Hope that clarifies my requirements.

Cheers,
MathMan

MathMan

  • Full Member
  • ***
  • Posts: 205
Re: How to programmatically test for exceptions
« Reply #12 on: May 19, 2020, 07:49:10 am »
which test suit are you using? Most of the test suits have a way to test for exceptions as well as proper results.

Thanks for the hint eljo. At the moment I plan to develope my own test suite. But if existing test suites can handle exceptions then I am more than willing to give it a try.

Can you recommend something? Must be FOSS though, as I am doing all this for my personal pleasure without any commercial intention.

Cheers,
MathMan
FPCUnit already mentioned and my preferred suit is https://github.com/graemeg/fptest both add extra application types in lazarus you can use them by selecting file\New.. and the dialog that pops FPCUnit and FPTest Applications.

Thanks - I am willing to give both a try. If you look at my previous post (with the project attached) - do you know if I they can handle my requirement?

MathMan

  • Full Member
  • ***
  • Posts: 205
Re: How to programmatically test for exceptions
« Reply #13 on: May 19, 2020, 07:53:52 am »
which test suit are you using? Most of the test suits have a way to test for exceptions as well as proper results.

Thanks for the hint eljo. At the moment I plan to develope my own test suite. But if existing test suites can handle exceptions then I am more than willing to give it a try.

Can you recommend something? Must be FOSS though, as I am doing all this for my personal pleasure without any commercial intention.

FPC ships with fpcunit and there is also a Lazarus package to create a testsuite that can display the results in a GUI frontend.

Thanks for the feedback, I will look into this.

While I am pondering with the design of the program - if I do it thoroughly I will have to create a lot (>100 at least) of operator overloads to catch all variants. Must I worry about some compiler internal limits?

eljo

  • Sr. Member
  • ****
  • Posts: 408
Re: How to programmatically test for exceptions
« Reply #14 on: May 19, 2020, 02:14:28 pm »
Thanks - I am willing to give both a try. If you look at my previous post (with the project attached) - do you know if I they can handle my requirement?
Both can handle almost everything. But in your case I'm attaching two projects in this message. a fpcunit based one with two different methods of testing and an FPTest one with only one of the testing methods just to demonstrate how similar the two suits are.

 

TinyPortal © 2005-2018