My Resume

  • My Resume (MS Word) My Resume (PDF)


Affiliations

  • Microsoft Most Valuable Professional
  • INETA Community Champion
  • Leader, NJDOTNET: Central New Jersey .NET User Group

Thursday, November 12, 2009

What’s a “Unit Test”?

Photo courtesy of those show cancelling bastards at CBS.

No, I’m not talking about these guys...

Generally speaking, writing any kind of code that exercises the code you've written is a good thing, but the term “unit test” carries with it a very focused and specific meaning. Listed below are what I consider the top-most important qualities of a “unit test”:

  • Atomic


    A unit test should focus on validating one small piece (“unit”) of functionality. Generally, this will be a single behavior or business case that a class exhibits. Quite often, this focus may be as narrow as a single method in a class (sometimes even a specific condition in a single method!). In practice, this equates to short tests with only a few (preferably just one) deliberate and meaningful assertions (Assert.That([…])).

    Common Pitfalls & Code Smells
    • Dozens of lines of code in one test
    • More than 2-3 assertions, especially when they’re against multiple objects
  • Repeatable


    A unit test should produce exactly the same result at any time on any environment, given that environment fulfills a known set of dependencies, e.g. the .NET Framework. Tests cannot rely on anything in the external environment that isn’t under your direct control. For instance, you should never have to worry about having network/Internet connectivity, access to a database, file system permissions, or even the time of day (think DateTime.Now). Failed unit tests should indicate a bug in the code and nothing else.

    Common Pitfalls & Code Smells
    • Tests pass on the first execution, yet some or all fail on subsequent executions (or vice-versa)
    • “NOTE: The XYZTest must be run prior to this or it will fail!”
  • Isolated / Independent

    In a culmination of the first two qualities, a unit test should be completely isolated from any other system or test. That is to say, a unit test should not assume or depend upon any other test having been run or external system (e.g. database) having a specific state or producing some specific result. Additionally, a unit test should also not create or leave behind any artifacts that may trip up other tests. This is certainly not to say that unit tests cannot share methods or even whole classes between each other – in fact, that is encouraged. What this means is that a unit test should not assume some other test has run previously or will run subsequently; these dependencies should instead be represented as explicit function calls or contained in your test fixture’s SetUp and TearDown methods that run prior to and immediately following every single test.

    Common Pitfalls & Code Smells
    • Database access
    • Tests fail when your network or VPN connection is disabled
    • Tests fail when you have not run some kind of external script (other than perhaps an NAnt script to compile, of course)
    • Tests fail when configuration settings change or are not correct
    • Tests must be executed under specific permissions
  • Fast

    Assuming all of the above conditions are met, all tests should be “fast” (i.e. fractions of a second). Regardless, it is still beneficial to explicitly state that all unit tests should execute practically instantaneously. After all, one of the main benefits of an automated test suite is the ability to get the near-instant feedback about the current quality of your code. As the time to run the test suite increases, the frequency with which you execute it decreases. This directly translates into a great amount of time between the introduction and discovery of bugs.

    Common Pitfalls & Code Smells
    • Individual tests take longer than a fraction of a second to run

If one were really clever, they might arrange the above into a cute little acronym like “FAIR”, but the order in which they appear above is very deliberate; it is the rough order of importance that I place on each quality.

Unit Tests vs. Integration Tests

Odds are that if you have written any automated tests recently, you probably violated one of the above guidelines… and probably for very good reason! What you have produced, my friend, is another very valuable form of automated test called an integration test. As opposed to a unit test - whose sole purpose is to validate the logic and/or functionality of a specific class or method – an integration test exists to validate the interaction (or “integration”, as it were) between two or more components. In other words, integration tests give the system a good work-out to make sure that all of the individual parts work together to achieve the desired result – a working application.

As such, integration tests are just as – if not more so – valuable in a business sense as unit tests. Their major drawbacks, however, are their slow speed and fragility. Not only does this mean that they will get executed less frequently than a unit test suite, but the rate of false-positives (or negatives… however you want to look at it) is much higher. When a unit test fails, it is a sure indication of a bug in the code. In contrast, when an integration tests fails it may mean a bug in the code, but could also very well have been caused by other issues in the testing environment such as a lost database connection or corrupt/unexpected test data. These false positives - though a useful indicator that something is wrong in the developer’s environment – usually just serve to slow down the development process by taking the developer’s focus away from writing working code. Assuming you strive to avoid these distractions whenever possible, the conclusion I come to is that you should therefore strive to rely on extensive test coverage via a solid unit test suite and supplement that coverage with an integration test suite and not vice-versa.

References

A great deal of the reason I even took it upon myself to write this blog post was because I couldn’t really find any good online articles or posts concerning “what makes a unit test”!   Below are a few of the great ones I found.  It may seem like I stole from some of them, but the ideas above really are my opinions…  they just happened to be widely shared. :)

However, it seems at this point if you are very interested in learning more about this topic, books are your best bet.  Anything by the “usual suspects” (Fowler, Hunt, Thomas, Newkirk…) is a great bet, but here are a few I have read and loved: