Recent

Author Topic: Right way to write tests  (Read 11231 times)

_N_

  • New Member
  • *
  • Posts: 33
Right way to write tests
« on: June 24, 2017, 11:43:56 am »
Hi all!
I need your piece of advice in testing my project.
I have a big class (2000+ lines of code) with few public and many many private methods, sometimes with non-trivial logic. They are bound logically, so, I think, separating is not the case here. Also I don't want to make them non-private. How can I write unit tests for them? I have never written tests in Lazarus, so it would be great to use some testing framework (like JUnit or TestNG in Java). I've googled fpcunit and FPTest, but I do not know which one (or another) is better. Actually I do not need any UI, just want to be sure, that my code works as expected.
Thanks.
 

taazz

  • Hero Member
  • *****
  • Posts: 5368
Re: Right way to write tests
« Reply #1 on: June 24, 2017, 01:46:10 pm »
I do not understand your question.
Are you asking how to write your tests that every single private method is called? If yes then that is up to the logic of the classes you set the appropriate properties and call your public methods in the required order to execute your private tests.
Are you asking how to write test cases? That is simple also, after installing the unit test suit of your choice you select "file\new" in the dialog that open you select from the project tree new fpcUnit Application or FpUnit Application either console or GUI its up to you, Press ok save it and run again file\new and now select from the module tree new FPCUnit Test Case or FPUnit Test Case a new class will be created for you with a failing test that you need to override with an actually test case. After that simple adding new methods to the class will add them to the suit for execution at run time make sure that the methods are published. Build and run the application and the test your code.

As for which suit is better personally I prefer fptest it has more spit polish, but for the most part I haven't found any test case I couldn't write on both suits.
« Last Edit: June 24, 2017, 01:47:44 pm by taazz »
Good judgement is the result of experience … Experience is the result of bad judgement.

OS : Windows 7 64 bit
Laz: Lazarus 1.4.4 FPC 2.6.4 i386-win32-win32/win64

Mr.Madguy

  • Hero Member
  • *****
  • Posts: 844
Re: Right way to write tests
« Reply #2 on: June 24, 2017, 02:20:22 pm »
I don't know. Any unit test packs boil down to just calling some method and checking results manually. So, why not to create your own one, that suits your tasks? Mine looks like this:
Code: Pascal  [Select][+][-]
  1. procedure AddValueRangeUnsortedTests(ATestCase:TObjectContainerTestCase<TAbstractPairList<Integer, TTestObject2>, TArray<TTestValueObjectPair>>;ASorted:Boolean);
  2.   var TestHelper:TAbstractTestHelper;
  3. begin
  4.   ATestCase.TestHelper.AssignInit(TestHelper);
  5.   with TestHelper do begin
  6.     {...}
  7.     ATestCase.AddTestCase(
  8.       'Query test',
  9.       True,
  10.       function(var ASource:TAbstractPairList<Integer, TTestObject2>):TAbstractPairList<Integer, TTestObject2>
  11.         var Values:TAbstractList<TTestObject2>;
  12.       begin
  13.         CreateValueObjectValueRange(ASource).AssignInit(Values);
  14.         Result := ValueObjectPairListFromValueList(
  15.           Values.Query(TDelegatedPredicate<TTestObject2>.Create(
  16.             function(const AValue:TTestObject2):Boolean
  17.             begin
  18.               Result := AValue.Value > 'e';
  19.             end
  20.           ))
  21.         );
  22.         Values.Detach;
  23.       end,
  24.       ArrayOfValueObjectPair(
  25.         [1, 2, 3, 4, 5, 6, 7, 8, 9, 10],
  26.         ['j', 'i', 'h', 'g', 'f', 'e', 'd', 'c', 'b', 'a']
  27.       ),
  28.       ArrayOfValueObjectPair(
  29.         [0, 0, 0, 0],
  30.         ['i', 'h', 'g', 'f']
  31.       )
  32.       {...}
  33.     );
  34.   TestHelper.Detach;
  35. end;
  36.  

And result looks like this:
« Last Edit: June 24, 2017, 02:22:23 pm by Mr.Madguy »
Is it healthy for project not to have regular stable releases?
Just for fun: Code::Blocks, GCC 13 and DOS - is it possible?

Leledumbo

  • Hero Member
  • *****
  • Posts: 8744
  • Programming + Glam Metal + Tae Kwon Do = Me
Re: Right way to write tests
« Reply #3 on: June 24, 2017, 04:39:20 pm »
You don't test private methods directly, but indirectly through public methods that call them.

jc99

  • Hero Member
  • *****
  • Posts: 553
    • My private Site
Re: Right way to write tests
« Reply #4 on: June 24, 2017, 10:02:54 pm »
You don't test private methods directly, but indirectly through public methods that call them.
Why not ? If you have the code why not make them Public/Protected for debugging. And then write tests for them too.
When your class is nearly finished it's hard (but not impossible) to write test afterwards.
You normally test from the ground-up you start with the basic-function and go higher and higher trough your class.
Think of it as a big building, you start /and test the fundament then the cellar, then go floor by floor to the roof.

Start with the following trought:
  • What do You expect from a function ?
  • What do you NOT expect ?
  • where are the boundaries ?
  • what happens if the input-parameters exceed these boundaries?
Then write a test fire as many input-parameter at you function as you can think of, e.G:
 - if your Procedure/Function takes a byte/word a for-loop giong from 0 to 255/65535 is ok modern computers do that in a matter of milliseconds, and you can be shure to have all input values testet.
 - Integer, int64, floatingpoint and strings are another case.
   The Monte-Carlo-Methode comes in handy here, meaning use a Random-Number/String-Generator that generates a sufficient number of tests e.G: >200000
   (that depends on How much time you want to spend and how save it has to be).

To test the function DO NOT just copy the code of the function and test against it.
Think of a slightly different method to check the result.

FPC-Unit-Test is a test-Framework you can use.
It helps you organize your tests.

As I said it's much better to do TDD (Test-Driven-Development).
Whenever you want to create something, start with the test, and falsify it, because the function is not there yet, then create the function and your test succeeds ... and so on. When you think about a function it's just a small amount of brain-power to think about a test. And when you have it, you can test whenever you add a new function, that this doesn't influence the already existing functions.
 
« Last Edit: June 24, 2017, 10:11:36 pm by jc99 »
OS: Win XP x64, Win 7, Win 7 x64, Win 10, Win 10 x64, Suse Linux 13.2
Laz: 1.4 - 1.8.4, 2.0
https://github.com/joecare99/public
'~|    /''
,_|oe \_,are
If you want to do something for the environment: Twitter: #reduceCO2 or
https://www.betterplace.me/klimawandel-stoppen-co-ueber-preis-reduzieren

serbod

  • Full Member
  • ***
  • Posts: 142
Re: Right way to write tests
« Reply #5 on: June 25, 2017, 12:00:23 am »
Pascal have something better, than unit tests.
https://www.freepascal.org/docs-html/rtl/system/assert.html

Put assertions in functions and methods to check parameters and conditions. In case of trouble, it shows custom text and source line number. You can always check you code in real cases, not only in synthetic tests. Also, enable range/overflow/IO/unfase checks in compiler options, its help prevent rare troubles.

In release build, just disable checks and assetions in compiler options.

howardpc

  • Hero Member
  • *****
  • Posts: 4144
Re: Right way to write tests
« Reply #6 on: June 25, 2017, 12:14:57 am »
Pascal have something better, than unit tests.

Assertions (and the other compiler checks you mention) are not better than unit tests. All possible tests have their place in stressing your code to try to eliminate bugs.
An assertion or range check can tell you if your sorting routine contains some glaring bug, but it cannot tell you in a comprehensive way if the routine does not  sort its data correctly. For that you need to design proper tests on as wide a range of data as is practicable.

Leledumbo

  • Hero Member
  • *****
  • Posts: 8744
  • Programming + Glam Metal + Tae Kwon Do = Me
Re: Right way to write tests
« Reply #7 on: June 25, 2017, 03:57:07 am »
Why not ? If you have the code why not make them Public/Protected for debugging. And then write tests for them too.
It breaks encapsulation.
As I said it's much better to do TDD (Test-Driven-Development).
You know TDD, so you should already know why we don't test private methods. TDD talks about public interface, on how a functionality will be used by others, not the implementation details. Private methods are not meant to be used from the outside, they are implementation details. You can freely add/modify/remove private methods but still test against the public methods that call them. If you do something wrong, the test will break anyway. Reopen your TDD books again to confirm my information.

jc99

  • Hero Member
  • *****
  • Posts: 553
    • My private Site
Re: Right way to write tests
« Reply #8 on: June 25, 2017, 08:31:41 am »
Why not ? If you have the code why not make them Public/Protected for debugging. And then write tests for them too.
It breaks encapsulation.
And giving the Test-Environment access, e.G: with Debug-Defines brakes that ? If you could test it, test it !
You know TDD, so you should already know why we don't test private methods. TDD talks about public interface, on how a functionality will be used by others, not the implementation details. Private methods are not meant to be used from the outside, they are implementation details. You can freely add/modify/remove private methods but still test against the public methods that call them. If you do something wrong, the test will break anyway. Reopen your TDD books again to confirm my information.
Yea the books, like this forum, so many people so many opinions. But you agree TDD in general is a good thing ! I agree with you that it's a good thing to start, testing the interface-functions! Normally all you have are the interface-function e.g: in precompiled libraries ... But please don't get mad at me, because I made the experience hat the books in this specific case are wrong. I learned if you could test it, test it, the hard way. As I said that's my experience.
e.G: Once I Had a (private) hashing function
Code: Pascal  [Select][+][-]
  1. function TMyClass.hash(aValue:integer):integer;
  2.  
  3. begin
  4.   result := abs(aValue) mod 23;
  5. end;
  6.  
You see the flaw ?
BTW, Range check don't get this one either !
« Last Edit: June 25, 2017, 05:42:17 pm by jc99 »
OS: Win XP x64, Win 7, Win 7 x64, Win 10, Win 10 x64, Suse Linux 13.2
Laz: 1.4 - 1.8.4, 2.0
https://github.com/joecare99/public
'~|    /''
,_|oe \_,are
If you want to do something for the environment: Twitter: #reduceCO2 or
https://www.betterplace.me/klimawandel-stoppen-co-ueber-preis-reduzieren

jc99

  • Hero Member
  • *****
  • Posts: 553
    • My private Site
Re: Right way to write tests
« Reply #9 on: June 25, 2017, 09:48:34 am »
Don't see the flaw ? Me neither (at first) !
And please don't get me wrong I am not against range-checks ! Also asserts are a good way to define (minimum)-preconditions (among others) ! But the best is to combine all/each of these methods to get a save product.
But as I said if encapsulation keeps you from testing a critical function, kick encapsulation ! (my personal experience/opinion)

[spoiler]
The function gives the result of -6 in rare cases !
OS: Win XP x64, Win 7, Win 7 x64, Win 10, Win 10 x64, Suse Linux 13.2
Laz: 1.4 - 1.8.4, 2.0
https://github.com/joecare99/public
'~|    /''
,_|oe \_,are
If you want to do something for the environment: Twitter: #reduceCO2 or
https://www.betterplace.me/klimawandel-stoppen-co-ueber-preis-reduzieren

_N_

  • New Member
  • *
  • Posts: 33
Re: Right way to write tests
« Reply #10 on: June 25, 2017, 11:43:11 am »
Many thanks all for replays! I appreciate each answer.
I indeed want to test encapsulated private methods (thanks @taazz for describing how to use test frameworks). Maybe not great idea - as mentioned above - and I agree with @Leledumbo that we have to test interfaces, not internal implementation, because last one could be easily changed. But here I have one public method (if don't count getters and settings setters) which runs all complicated logic. If get that way I become scared of possible quantity of incoming data for tests (well, that's files, not function args!). Knowing this I designed my class in such way, that each function is responsible for some small piece of job (well, actually normal code which seems not smells). And now it seems good idea to test each "brick". Thanks @jc99 for explanation. But I still don't want to break encapsulation.
And here just my thoughts, I am not sure will it work or not, but still. What if add in this unit another class, say TTestIjector or TTestInvoker or so, which has access to private items. And then from test class call methods of this class (actually each test like friend function in C++). And here https://stackoverflow.com/questions/4186458/delphi-call-a-function-whose-name-is-stored-in-a-string is some idea, because I'm afraid child classes don't have access from test unit to private fields as the invoker has...
@Mr.Madguy it great, but looks a bit complicated to me...
Be honest, I still haven't decided which way I should use... Anyway, thanks all!

Leledumbo

  • Hero Member
  • *****
  • Posts: 8744
  • Programming + Glam Metal + Tae Kwon Do = Me
Re: Right way to write tests
« Reply #11 on: June 25, 2017, 05:58:43 pm »
As I said that's my experience.
That's something that can't be debated, as it already becomes a habit. My pick is what is considered best practice in general.
e.G: Once I Had a (private) hashing function
Code: Pascal  [Select][+][-]
  1. function TMyClass.hash(aValue:integer):integer;
  2.  
  3. begin
  4.   result := abs(aValue) mod 23;
  5. end;
  6.  
You see the flaw ?
BTW, Range check don't get this one either !
Yes, it's not worth a private method. Why? It doesn't access any of the class properties nor methods, so it should be standalone as in helper class methods or something similar.

From Pragmatic Unit Testing:
Quote
In general, you don't want to break any encapsulation for the sake of testing (or as Mom used to say, "don't expose your privates!"). Most of the time, you should be able to test a class by exercising its public methods. If there is significant functionality that is hidden behind private or protected access, that might be a warning sign that there's another class in there struggling to get out.

Mr.Madguy

  • Hero Member
  • *****
  • Posts: 844
Re: Right way to write tests
« Reply #12 on: June 25, 2017, 10:04:06 pm »
@Mr.Madguy it great, but looks a bit complicated to me...
Nothing complex. All we need - is bunch of callbacks, that call certain methods of our object, compare results with desired ones and of course make sure, that neither exceptions, nor memory leaks happen. What you see - is just add test case method, that takes following parameters:
1) Name of test case.
2) "Create default input list" boolean parameter. Just for convenience - not to create it manually, as majority of test cases need default one. If True - ASource will already have default list, assigned to it.
3) Anonymous test case method. Emm, just callback, that can implicitly store references to variables from context, it's declared in. For example routines like CreateValueObjectValueRange - are in fact methods of TestHelper. Very powerful thing. Unfortunately, still not implemented in FPC/Lazarus, despite of fact, that Delphi has been having it since 2009. Called during execution of ATestCase.RunTests method, i.e. in this case ATestCase - is just container for other test cases. They have tree-like structure. First we fill this tree and then execute all tests from it at once.
4) What ASource should be equal to as result. Needed in some cases, when ASource is being modified, such as Add, Insert, Delete method tests.
5) What Result should be equal to as result. Needed, when new object is being created during test case, like in this case, when we query all elements, where Value > 'e'.
« Last Edit: June 25, 2017, 10:21:47 pm by Mr.Madguy »
Is it healthy for project not to have regular stable releases?
Just for fun: Code::Blocks, GCC 13 and DOS - is it possible?

_N_

  • New Member
  • *
  • Posts: 33
Re: Right way to write tests
« Reply #13 on: June 26, 2017, 08:10:55 pm »
@Mr.Madguy, Thanks.
One more question. How to check expected exception? What is common way in FPC?

taazz

  • Hero Member
  • *****
  • Posts: 5368
Re: Right way to write tests
« Reply #14 on: June 26, 2017, 08:22:08 pm »
@Mr.Madguy, Thanks.
One more question. How to check expected exception? What is common way in FPC?

Code: Pascal  [Select][+][-]
  1. unit TestCase1;
  2.  
  3. {$mode objfpc}{$H+}
  4.  
  5. interface
  6.  
  7. uses
  8.   Classes, SysUtils, fpcunit, testutils, testregistry;
  9.  
  10. type
  11.  
  12.   { TTestCase1 }
  13.  
  14.   TTestCase1= class(TTestCase)
  15.   public
  16.     procedure Raiseexception;
  17.   published
  18.     procedure TestHookUp;
  19.   end;
  20.  
  21. implementation
  22.  
  23. procedure TTestCase1.Raiseexception;
  24. begin
  25.   raise exception.Create('This is a test');
  26. end;
  27.  
  28. procedure TTestCase1.TestHookUp;
  29. begin
  30.   CheckException(@Raiseexception,Exception,'This is a test');
  31. end;
  32.  
  33. Initialization
  34.  
  35.   RegisterTest(TTestCase1);
  36. end.
  37.  
Good judgement is the result of experience … Experience is the result of bad judgement.

OS : Windows 7 64 bit
Laz: Lazarus 1.4.4 FPC 2.6.4 i386-win32-win32/win64

 

TinyPortal © 2005-2018