In the part Literals and Variables of the chapter 21 the authors advise to use variables/constants in place of meaningless literals:
...test code tends to be more concrete than production code, which means it has more literal values. Literal values without explanation can be difficult to understand because the programmer has to interpret whether a particular value is significant (e.g. just outside the allowed range) or just an arbitrary placeholder to trace behavior (e.g. should be doubled and passed on to a peer).While this rule is rather self-explanatory, I have noticed some interesting pattern. When I do TDD pretty often in the refactoring phase I go through the test code and replace literals with constants (unless it obscures the readability). What happens is 2 kinds of constants emerge:
...
One solution is to allocate literal values to variables and constants with names that describe their function.
- Example Constants (one value out of many possible)
public static final String USER_NAME = "Joe"; //in fact it could be also "John" or "Sue"
public static final int INVALID_ID = 666; //in fact it could be also 667, 668, 669 and so on...
- Significant Constants (concrete value having special meaning)
public static final String ATTR_EVENT_ID = "eventId"; // significant valueSignificant Constants will be very often needed in both test and implementation. So they can be moved to the implementation class and then referred to in the test class.
public static final int AGE_OF_CONSENT = 18; // significant value
On the other hand, Example Constants will usually stay only in the test code.
Here is an example using Spring MVC:
Update: as pointed out in the comments, sharing Significant Constants in both tests and implementations carries the risk of uncaught errors when editing the constant value. The safest way is indeed to have the test class be entirely a specification, thus defining its own constants/variables and not refer to the implementation class.
PS: don't forget to visit awesome Path11 Book Club :)
That's a nice distinction, and one I hadn't thought of before.
ReplyDeleteBut instead of using Example Constants in tests, I usually create helper methods that return RANDOM values of the appropriate kind. This can sometimes catch/prevent silly bugs.
Thanks! Helper methods returning random values are indeed an interesting extension of the idea of Example Constants.
DeleteBy the way, there is a tool called Jester (Java) that will make changes to your source code, replacing the literals with some random values or altering your conditionals, to see if your tests still catch the bugs. Jester will generate a report indicating the changes that didn't break your tests. This might help find out if you are testing the edge conditions properly (promoting a sharper choice of Example Constants).
Also the name of the variable/constant indicates how important its value is (or can it be generated randomly out of some range).
DeleteFor example, if you develop an app for bookkeeping you will want to calculate the income tax based on one's income. If the test case is not focused on the calculation (displaying person details), you might create a Person object with some random income - you could name the variable/constant INCOME. But if you test-drive the tax calculation logic and you want to make sure thresholds (0-10k - 10%, 10k+ - 20%), you could create a variable/constant LOW_INCOME or HIGH_INCOME (narrowed range/ equivalence classes) or even be more precise: MINIMAL_HIGH_INCOME, MAXIMAL_LOW_INCOME (edge conditions).
So I think you could randomly generate your fully random literals (INCOME) and even narrowed range ones (LOW_INCOME). Edge conditions seem the most interesting to me, because you might want to test-drive the proper comparison operator (> instead of >= etc.).
Also, if the value is (fully) randomly generated I don't expect it to influence the test outcome. Similarly, any value within an equivalence class should yield the same result. If it doesn't, maybe that's an indication of some bad code smell.
So yes, I thought about this kind of generation of Example Constants whenever I used the name like INCOME (or before that even SOME_INCOME or ANY_INCOME). But I haven't really tried it out, needless to say in the long term.
It would be great if you could share some examples where this approach prevented you from bugs :)
Well, supposing the test checks that the code works when the value 10 is given. We might have forgotten to write a test to check that the code should work for 11, 12, 13 ...
DeleteThe major concern with generated arguments is the lack of repeatability. You might introduce a bug and not find it out because the test run with an another argument. Or the other way around. How does that map to the TDD cycle where you want to run your tests very often and make sure you didn't break anything?
DeleteMy assumption is tests never give me 100% coverage of all possible cases. That is, I will try to cover all the scenarios (pick one argument out of each equivalence class) but obviously something might slip my mind. Once a unforeseen bug pops up, I will address it and add a relevant test to make sure it never comes back again. This builds a safety net I can rely on.
Sure, under circumstances it may lead to big number of test cases. If your test needs to work with lots of input arguments, what you can do is to use a Parameterized JUnit Runner Class (in Java, or an equivalent in any other language) and run the same test multiple times with different arguments (but in a repeatable manner).
All in all, I believe the drawbacks of generating arguments outweigh the benefits. But again, I could think of it as some form of complementary testing (not it the standard TDD-cycle test suite), similarly to Jester (see my first comment).
I agree with the distinction you make.
ReplyDeleteBut I disagree with using constants in tests taken from the implementation code. The duplication that you see is not duplication because the values represent two distinct things: (1) In the test, what the value SHOULD be; (2) In the implementation, what the value ACTUALLY is.
You want the tests to catch problems when you change things in the implementation. If you change the value of SUCCESS_VIEW to "incorrectPage", the test will continue to pass. That's not what you want.
That is a good point! I agree both constants represent different concepts. In your example the test would pass but when you deploy the application there would be a 404 error.
DeleteOn the other hand, if you work with TDD you don't just change the implementation, but instead you look at the relevant test first. Obviously, we are discussing a simple example here. But if the Constant is reffered to in many tests, it is probable you overlook some of those, change the implementation and the overlooked tests won't show you the problem. Alternative would be to search for usages of Constants before changing them, but that's a bit too manual...
So yeah, I might have pushed it too far but it helped me notice the distinction.