24 Sep 2008
One sticking point a lot of people seem to have is how to unit test a user interface. I’ve read many articles by TDD advocates saying that you absolutely need to have unit tests for your GUI components, and then I’ve also read articles from the RAD tool crowd who say that because of the method of genesis the cost of testing out weighs the benefit. While both groups are certain to remain dead-set against each other, I prefer to take a more pratical approach. I do tend to disagree with the TDD folks that all tests must be written first, as long as there is sufficiant coverage I don’t really care when the test gets written. I believe that it is one of the many jobs of a professional engineer to know when which approach is appropriate for the environment they are operating in. However I strongly disagree with the RAD folks who say that because their UI code was computer generated, it doesn’t need testing. Unit testing is about validating functional behavour, now RAD tools will layout widgets all day long, but they make no guarentee as to what happens when you click a button.
A hybrid approach
Now when looking a the visual appeal of a panel, there is no tool that is better than the human eye. Granted it sometimes does depend on the human, but there are not going to be any tools out there that can ‘test’ for style, continuity etc.. What you can test is the outcome of user interaction. For example:
import java.awt.BorderLayout;
import java.awt.event.ActionEvent;
import java.awt.event.ActionListener;
import javax.swing.DefaultListModel;
import javax.swing.JButton;
import javax.swing.JList;
import javax.swing.JPanel;
public class ExamplePanel extends JPanel {
private JButton clearButton = new JButton("Clear");
private DefaultListModel listModel = new DefaultListModel();
public ExamplePanel() {
super(new BorderLayout());
listModel.addElement("test");
listModel.addElement("test2");
listModel.addElement("test3");
add(new JList(listModel), BorderLayout.CENTER);
add(clearButton, BorderLayout.SOUTH);
clearButton.addActionListener(new ActionListener() {
public void actionPerformed(ActionEvent e) {
listModel.clear();
}
});
}
}
Now when you look at this class you will notice that the only testable function of the panel is the constructor. So you will probably start off with a testcase that looks like:
public class ExamplePanelTest extends TestCase {
public void testExamplePanel() {
assertNotNull(new ExamplePanel());
}
}
And you would see in your coverage report that you have covered 87% of the panel, that’s great right? Well not really, all you’ve tested is that when you new this object an exception isn’t thrown and that really is insufficient for proving that your panel does what you’ve intended. Ideally we would like to know what happens when somebody clicks the “Clear” button.
I’ve attached to the bottom of this post a file that contains some convenience methods for iterating through the java UI containers to locate widgets of interest. Using this class allows us to write more comprehensive test cases. For example, this is how I would test the above class:
public class ExamplePanelTest extends TestCase {
public void testExamplePanel() {
ExamplePanel panel = new ExamplePanel();
assertNotNull(panel);
//This returns a list of all the JLists
//that are contained within the panel
List lists = UiTestUtils.findAllInstances(JList.class, panel);
assertEquals(1, lists.size()); //We should only have one list
//The list should have 3 elements
assertEquals(3, lists.get(0).getModel().getSize());
JButton clearButton = UiTestUtils.findButtonByText("Clear", JButton.class, panel);
assertNotNull(clearButton); //There should be a clear button in the panel
clearButton.doClick(); //this is the method that simulates clicking the button
//The list should have 0 elements
assertEquals(0, lists.get(0).getModel().getSize());
}
}
Now you can see that I am using the iterative property of swing widgets to search through the children and locate the interesting widgets, and then I can simulate interaction with them. As you can see this test case will cover 100% of my GUI class, and it was extremely easy to do. Also my test does not rely on ‘volatile criteria’ such as widget placement to test the functionality. This allows me to make aesthetic changes to the class without breaking my unit tests. Happy Testing.
File Download: UITestUtils.java
23 Sep 2008
I realize that I am not the first to write about unit testing, but there seems to to be an absence of examples dealing with only simple unit testing. Most all documents deal with unit testing as part of an overall strategy like XP or agile, I don’t really care when you do it as long as you do.
Example Class
public class ExampleClass {
private String message;
public String getMessage() {
return this.message;
}
public void setMessage(String message) {
this.message = message;
}
public String operation() {
StringBuffer buf = new StringBuffer();
if(message.length() > 10) {
buf.append(message.hashCode());
} else {
buf.append(doubleMessage());
}
return buf.toString();
}
private String doubleMessage() {
return message + message;
}
}
Now this is a stupid class that doesn’t really do anything but since we’ve written it we need to make sure that it works.
First we start with a new test class:
public class ExampleClassTest extends TestCase {
}
Of course this class doesn’t actually do anything, but it does show us that it is the Test for ExampleClass based on its name, and that it extends TestCase. Now lets add a test method:
public class ExampleClassTest extends TestCase {
public void testMessageStoring() throws Exception {
ExampleClass example = new ExampleClass();
example.setMessage('test');
assertEquals('test', example.getMessage());
}
}
As you can see all we are doing is invoking a method on the class and asserting that it responds in an appropriate way, in this case when I set a string that should be available via the get method. All tests are of the format ‘public void test
() throws Exception { }’, and junit will be able to locate this test method and run it. But this test isn’t very comprehensive because it only executes 10 out ot the possible 45 instructions in our class, so we will need to do more testing.
public class ExampleClassTest extends TestCase {
public void testMessageStoring() throws Exception {
ExampleClass example = new ExampleClass();
example.setMessage('test');
assertEquals('test', example.getMessage());
}
public void testClassOperation() throws Exception {
ExampleClass example = new ExampleClass();
example.setMessage('test');
assertEquals('testtest', example.operation());
}
}
Here we’ve added a method that tests the operation method of our class, but we know there are two paths inside the operation method and we aren’t exercising 100% of the class.
public class ExampleClassTest extends TestCase {
public void testMessageStoring() throws Exception {
ExampleClass example = new ExampleClass();
example.setMessage('test');
assertEquals('test', example.getMessage());
}
public void testClassOperation() throws Exception {
ExampleClass example = new ExampleClass();
example.setMessage('test');
assertEquals('testtest', example.operation());
}
public void testClassOperationLongMessage() throws Exception {
ExampleClass example = new ExampleClass();
example.setMessage('testalongermessage');
assertEquals('-1184182769', example.operation());
}
}
Now we have a test that covers 100% of the instructions in the our class. Now I know that this is a trivial example but unit testing you code shouldn’t be much harder than this, if it is then it is probably a good sign that the complexity of your class has grown beyond what it should be.
11 Sep 2008
OK maybe that’s over stating it a little bit, its not quite as bad as ‘goto’.
I’ve now been using AccuRev for about 6 months now, and I it seems that I am having to adjust my process to suit some of the idiosyncrasies of the tools. Now I’m not going to say that its a bad tool, because as with everything in computer science I’m certain that there was a specific use case that was identified that AccuRev fills perfectly, but I don’t have that use-case.
My primary gripe is the inability to update my local workspace while I have a dirty workspace. Now while I understand the rational for this, it irritates me to no end. When my team is working toward a release we are all working very closely and when somebody else completes a feature that I would find useful I would like to be able to checkout their changes without having to commit my changes. The AccuRev team seems to think that taking a course view of solves many of the SCM problems, and indeed it probably does .. for the tool designers, but what about the tool users?
In any decent software project, the software will naturally fallout into different modules, and I find it extremely useful to sometimes update a piece of the tree without updating other pieces. AccuRev doesn’t allow this at all, it forces you to update everything or nothing. This is the fundamental cornerstone of my problem with this product. With any tool I expect to have some level of choice in how I will use that tool, and I fully understand that I am accepting the responsibility to utilize that tool properly else I only have myself to blame when damage is caused. In AccuRev everything is very one-dimensional and doesn’t allow me to work the way I want to work, and really who is supposed to be in control here, the human or the tool? (hint if your tool is AccuRev then you are not in charge)
Then there is this business with server workspaces, in AccuRev there are two levels for checking in a file, there is a ‘keep’ which stores the change in your private workspace on the server and then there is a ‘promote’ which puts the revision in the public stream to be shared with everybody else. The main problem with the approach is mostly psychological, having these two levels actually lengthens development iterations because people can check into their private workspaces for the duration of the development effort (I’ve seen weeks and months) and then only promote and merge into the public stream when they are completely finished, its very waterfall and not really helping anybody because it lengthens the integration and doesn’t provide the rest of your team with the benefits of your work until its too late. Now I will admit that this is a process problem, but it is something to consider because if you are working with inexperienced developers this might go unnoticed and then everybody will be in for a surprise when a very large merge happens and nothing works anymore.
At the end of the day AccuRev is a inflexible tool that comes with a pretty hefty price tag. I would rather use a free tool like subversion any day. While I’m certain that most of my complaints are petty, when you use a tool everyday its the little things that start to eat away at you. While I’m sure some of my frustrations are my own fault, I’ve tried to get to the bottom of all of them and have found no research to make me think otherwise.
11 Sep 2008
I will go on record saying that I love static analysis tools, especially findbugs. Now I’m not saying that they can work miracles, just like any tool they need to be used in conjunction with a somewhat functioning human brain in order to do more good than harm, but the thing that I like the most about the this tool in particular is that it is so easy. Much like the tag line for the project “Findbugs: because it’s easy” suggests, the software is very simple to run and will quickly provide useful and sometime life saving (I guess this depends on what your software does) results. Each bug that is found is identified with a bug description, a severity, a line number, and on request a detailed description of why what you wrote is less that ideal and suggestions for pattens or more proper methods to employ. It basically does everything short of fixing it for you, how cool is that. Of course there is a catch, findbugs can and will produce some false positives for bugs. However in my view I see this as a half blessing, because it forces you to take a look at something that you probably knew to be dubious while writing and will either shame you into using a less questionable method or at least let you convince yourself that this is exactly what you want.
The other hurdle that a first time findbugs user will find, especially when running it against a large project, is that he/she will be overwhelmed by the number of bugs that they find. Now lets be clear, nobody is suggesting that you sit down in an afternoon and fix all of them, one of the interesting things about static analysis is that it doesn’t actually know the execution path of your code. So it reports on all the possible paths that it figure out, and while it is probably correct that the code is full of bugs the module may be tested well enough for you to have confidence in the way that you are using it anyway. My rule of thumb is that I will run the tool and make myself away of the bugs and then as I go about my normal day adding features and closing trouble tickets I will fix the bugs that are in the classes that I touch, always leaving it better than I found it.
Eclipse Plugin
Now before when I said it was easy I ment it, but since they made an eclipse plugin if you are not using this tool you are coding irresponsibly. Once you install the plugin and restart eclipse all you have to do is right click on the root of your project and say ‘Findbugs -> findbugs’ and it will go out and do its thing, reporting everythign in the console with all other compiler errors and warnings.
Continuous Integration
The other thing that I am a big fan of is letting findbugs (and other tools like PMD, jDepend, etc..) run on your continous integration server of choice. Findbugs comes with a set of ant tasks that integrates nicely into many existing build suites. Most of these tools have findbugs plugins (I know hudson does for sure) and allow you to moniter the number of bugs being reported for each checkin. This will allow you and your team to see the result of their work, if the number goes up then they should know to go back and run the tool and fix their bugs, and if the number goes down they should feel good about themselves so having averted a potential disaster (at least thats how you can sell it to your boss at review time). The beuty of running it on a continous integration server is that you don’t even need to remember to do it, and yet you can still reap the benefits of its insight.
Bravo findbugs, bravo.
10 Sep 2008
I do. Its weird the things that irritate you, but bad style is quickly becoming a top ten pet-peeve. Code style is something that is easily taken for granted and you never fully appreciate it until its gone. I’ve been fortunate to have worked on many projects over the past few years where basic style conventions were enforced, and not by tools such as checkstyle or PMD, but by an actual human.
Using tools to aid in enforcing style can help, but they end up being pretty poor at the end of the day at guaranteeing that you will have nice readable/understandable code. As code bases grow very large, it becomes evermore important that we help our brains navigate quickly and the use of proper style sends subtle signals about the flow of a class. When I see camel-case starting with a lowercase letter I know its a method, uppercase and I know its a constructor.
Strangely enough I am not actually calling for people to follow ‘all’ of the rules of a style guide just rules like the following:
- packages contain lowercase letters only
- classes start with an uppercase letter
- Constructors start with an uppercase letter (this is a freebie if you follow above)
- methods start with lowercase letters
- code blocking such as if/for/while/do always use braces
Following these rules should be pretty easy, actually it should be almost no work at all and yet it will pay huge dividends when your coworkers (or you after a few days) attempt to read through your code. I know that this is merely a small fraction of the rules of any real style guide, but I think if everybody did just these few things we would be well on our way to coding utopia.