picture home | pixelblog | qt_tools

omino code blog

We need code. Lots of code.
David Van Brink // Tue 2006.08.1 23:21 // {broad generalities code}

ST Development

In which dvb considers “named software development methodologies” to each be passing fads… But testing, aah, there’s something to that.

The Claim

A few well-placed tests in your flow can offer benefit far out of proportion to the investment required.

Discovering Testing

There’s lately much chatter about “development methodologies”. At my workplace there was all sorts of endless noise — mostly amongst those not coding — about pair programming and how using XP will guarantee that you can ship your product any day you like, and there’s no need for Alpha or Beta testing, and possibly cure Morgellon’s disease, and such the like. Then the product development cycle began and it was business as usual, fortunately.

One significant change did come from all that pointyhaired murmuring though, and that was the mandate for unit tests. Using unit tests has improved the process immeasurably. No… that’s not right. It’s improved the process measurably.

You’ve probably heard the ritual: “Write the test first.” I don’t know about that… occasionally I do that. But I can explain the concept much more approachably, if perchance you’re not already a believer.

Here’s what I used to do. I’d write some new subroutine (we used to call them that), and then I’d write a little test for it:

int main(int argc,char **argv)
	printf("I get: %d\n",newSubroutine(17));
	// note to self -- dont check in the above!!

I’d build the application, make sure it printed out the right answer, and then delete the test. Delete the test! A “unit test” is just something you keep around, instead.

When I first tried out some Java unit tests with junit, it was profoundly obviously a good thing. (Junit is approximately a tool that runs all the methods named “testSomething” and counts up the ones that programmatically assert a “failTest” method. If you’re coding Java, just use it. End of story.)

Testing QT_Tools

It’s also crept into my personal projects. That’s how right it is.

For my personal projects I don’t use “test-driven development”. I don’t write tests first, or make sure that every case is covered, even really ensure that they’re “unit tests” per se. But I have started to run some tests. I do this as part of my Makefile, and it improves my confidence enormously, even minor tests. And you don’t need any big testing infrastructure, just another target that runs.

I think of this as “ST Development” for “Some Testing.” It turns out to be infinitely better than no testing. (Do the math.) And it’s ever so much more civilized than relying solely on manual testing.

For qt_tools (a freeware Mac OS X product of mine) I run a Perl script to check that everything built correctly and that a few simple cases work. I don’t use any formal testing framework… I just ape junit with simple ad-hoc Perl functions like:

      sub assertEquals($$$)
          my ($what,$want,$got) = (@_);
          die "ERROR $what: $want != $got" if $want != $got;
          print("good: $what == $got\n");

I now keep a link to the test results right on the qt_tools front page. I like to think it lends a professional aura to the affair.

Here’s the Perl qt_tools testing script:

Testing a WordPress Plugin

In developing the WordPress plugin used to show code on this blog I found myself going through the following cycle: Edit the plugin, run “make” to copy it into the blog tree, and then manually edit a draft post to see if it worked.

More often than not, I’d get a PHP error page due to a syntax error, much less a proper result from the plugin. And the cycle time was ridiculous! Hitting reload in the browser and typing a fresh post… come now!

One of the characteristics of a “plugin” is that it lives in a well-defined environment and follows a well-defined contract. This is perfect for unit testing. I created a PHP program that invokes the plugin with a few simple cases, checks the result for sanity, and returns a Unix error code appropriately. In the Makefile, the “test” target is a dependency of the “all” target. This means that the Makefile won’t even copy it if there’s a syntax error. Just running the thing for syntax has saved me countless tedious browser reloads! And checking the plugin for correctness was at first just icing on the cake, but became essential as I loaded more and more features atop. Again, there’s no “testing framework” or any of that, just some handrolled “assert” methods that count up correct and incorrect responses.

Here’s the PHP test:

And here’s the Makefile:


I don’t aim to give an explicit “how to” for testing. The message I’m trying to convey is that no matter how tight your schedule is — my personal projects are squeezed in an hour or three here and there — a few thoughtfully-placed tests in your flow are well worth the effort. Absolutely.

Daniel Jalkut // Sat 2006.09.23 17:245:24 pm

I somehow glossed over this article when you first published it. I am also a member of the “ST Development” club. I have not been able to bite the hook of test-first development, but I rely on and appreciate test suites in some of my Cocoa code – specifically where I expect there to be a lot of vulnerability to failure.

For instance, in FastScripts, I have code that pays attention to the state of the user’s script directory and updates on the fly, reacting to files being moved, deleted, added or modified. It would be way stressful to try to think about this all the time, so I just wrote a test suite that puts the code through “all the usual cases.” It gives me great confidence after tweaking something down deep in the code, that all the tests show up green in Xcode.

I like the idea of testing for syntax in a web plugin situation. That’s a great idea, and something I’ll try to integrate into my WordPress/Xcode environment.

oh, i dont know. what do you think?


(c) 2003-2011 omino.com / contact poly@omino.com