You are here

Unit Testing with Jython

I came across Adam Goucher's blog entry and his discussion of using Jython to execute JUnit tests. I have been experimenting with doing the opposite: Running Jython scripts from JUnit tests. It was a fun exercise, and it was enlightening to see that I could rewrite a 222 line JUnit test as an 87 line Jython script.

Here's what I did:

I want my unit tests to be picked up by a build process (an Ant script and its JUnit test runner), and since I don’t want to add a pre-compile step to build process, I create Java test classes that are subclasses of the JUnit TestCase that look like this:

public class FooTest extends JythonTestCase {
    public void testAll() {
        runJythonTests("fooTest.py");
    }
}

In this example, all of the tests are contained inside the script fooTest.py.

The tests inside the script are functions of the format:

def testFoo():
   ...

eg:

def testfoo():
   mock = MockFoo()
   ...
   assert mock.bar == 1

All the work to execute the Jython test functions is handled by the Java class JythonTestCase. Here’s the code:

import junit.framework.TestCase;
import org.apache.commons.io.IOUtils;
import org.python.core.PyFunction;
import org.python.core.PyList;
import org.python.core.PyStringMap;
import org.python.core.PyTuple;
import org.python.util.PythonInterpreter;
import java.io.IOException;
import java.util.ArrayList;
import java.util.Iterator;
import java.util.List;
import java.util.logging.Logger;

/**
 * Jython test case
 */
public class JythonTestCase extends TestCase {
        private Logger log = Logger.getLogger(getClass().getName());

        /**
         * execute all functions whose name starts with "test"
         *
         * @param scriptLocation location of Jython script on classpath
         */
        public void runJythonTests(String scriptLocation) {
                try {
                        String script = IOUtils.toString(Thread.currentThread().getContextClassLoader().getResourceAsStream(scriptLocation));
                        List failedTests = new ArrayList();
                        PythonInterpreter interpreter = new PythonInterpreter();
                        interpreter.exec(script);
                        PyList items = ((PyStringMap) interpreter.getLocals()).items();
                        while (items.size() > 0) {
                                PyTuple tuple = (PyTuple) items.pop();
                                Iterator iterator = tuple.iterator();
                                String key = (String) iterator.next();
                                Object value = iterator.next();
                                if (key.startsWith("test") && value instanceof PyFunction) {
                                        String functionName = ((PyFunction) value).__name__ + "()";
                                        long start = System.currentTimeMillis();
                                        try {
                                                interpreter.exec(functionName);
                                                long end = System.currentTimeMillis();
                                                log.fine("Test '" + functionName + "' in script '" + scriptLocation + "' passed [" + (end - start) + " s]");
                                        } catch (Throwable e) {
                                                long end = System.currentTimeMillis();
                                                log.severe("Test '" + functionName + "' in script '" + scriptLocation + "' failed [" + (end - start) + " s]");
                                                e.printStackTrace();
                                                failedTests.add(functionName);
                                        }
                                }
                        }
                        if (failedTests.size() > 0) {
                                fail("Test(s) failed in script '" + scriptLocation + "': " + failedTests);
                        }
                } catch (IOException e) {
                        e.printStackTrace();
                }
        }

}

The only complaint I have about this approach is that it appears to the outside world (eg, Ant) that the test consists of only one unit test since the class FooTest has only on test method in it, testAll(). All of the tests inside the script are executed, but if a development team gets excited about the raw number of tests in their test suite, this approach will be disappointing. One needs to consider what is really important: the sheer number of tests reported by your testing tool, or the test quality and the actual coverage reported by your coverage tool. In my case, I reimplemented the same tests as a Jython script and achieved the same test quality and coverage, but Ant reports that my new test class contains one unit test as opposed to the original 222 line Java version that reported 8 since it had eight testXXX() methods.

I experimented with JavaAssist to try to construct JUnit classes on the fly with real testXXX() methods that would each invoke individual functions within the python script, but ran up against classloader issues. That was purely to get Ant to report my test as containing 8 tests instead of one.

In summary, I have found this to be promising. I was able to rewrite a 222 line java test as an 87 line Jython script. If Jython makes writing unit tests less cumbersome, perhaps developers will be less resistant to writing tests for their code using this approach.