Monday, September 24, 2012

Unit Tests: automatically generating

I've recently assisted to a course about different ways to automate testing. It includes generating, coverage and generating input for the tests.

CodeCover

This is a tool that analyses the code coverage of tests. When we write tests, the idea is that all the lines of our code will be executed. This tool allows us to check it. The result is going to be shown directly over the code with helpful color highlighting.
An Eclipse plugin is available and also a standalone version for use from Command Line and Ant.
Link to CodeCover: http://codecover.org/index.html

Randoop

This is my favorite one. It randomly generates unit tests! The best thing of this is that is really useful, you don't need to write any single line of test code.

How it works? For each method tries to generate code to call the method in different ways, and using different seeds of code generates random code to achieve unexpected exceptions or problems with the class invariant. To check the class invariant it requires a method usually called repOk(). This method ensures that the object under test is in a valid state. For example:

@CheckRep
public boolean repOk() {
    if (size < 0 ) return false;
    if (size == 0 && last != null) return false;
    return true;
}


Other example a little bit more complex:

@CheckRep
public boolean repOK() {
    int itemsLeft = size;
    Node current = last;
    while (
itemsLeft>0 && current!=null) {
        current = current.next;
        
itemsLeft--;
    }
    return (
itemsLeft==0 && current==null);
}

Randoop http://code.google.com/p/randoop/

Eclipse plugin http://randoop.googlecode.com/hg/plugin/doc/index.html


Jumble

Given a class and a test case, this tools mutate(ie changes) aritmetic operands and logic. The it runs the tests, and if they won't fail, then you've bad test cases.
It's very easy to use and more than helpful.

jumble http://jumble.sourceforge.net/

Korat

Korat helps you to create input data for testing. Given a class invariant and domain restriction about its attributes, Korat generates valid structures and with no redundant results.
This is very usefull if you have a complex data type and you want to test it with lots of valid data sets.




Pex

A powerful tool that uses constraint solving engine to execute all your code and detect possible fails. It's only available for .NET.
There is a fun online app which is called Pex for fun that makes you guess code by modified constraint solving engine.





Tags: test, random, eclipse, junit, code, java, net.