Comment on Testing on High: Bottom-up versus Top-down Test-driven Development by Ryan Kinderman

Sun, 25 Nov 2007 17:45:20 -0800

@Merlyn: I don't think that the outside-in testing approach is necessarily slower in producing a result that has value to the user. That being said, it can be slower at producing a component at any level that is fully-functional. To understand what I'm getting at, imagine that you start by writing a little HTML for a web app feature you're building. Once you get something that looks to the user like it's going to work, you write some tests around it, mocking or stubbing out component behavior that the views delegate to. You then start testing/implementing the functionality that the views invoke, which may result in more stubbing or mocking. It may take a while before you can stop mocking or stubbing components and get to a level where the component is sufficiently small that it doesn't need to call off to other components, and you can fully test and implement its functionality.

If you were to use the inside-out approach, the idea is that you start at the component that is sufficiently small to fully-implement. You then write components that use that component's functionality. Since you start with inside-out precisely where you stop using outside-in, it's quicker to fully-implement a component or set of components using inside-out, at least initially. Does that make sense?

As for whether testing just below the UI layer is not purely outside-in:

I think a decision always has to be made regarding where the "outer-most" point at which to start testing is. This doesn't necessarily have to be at the UI layer. In any such decision, the correct choice is always the point that is the most critical to the user. Often, the user doesn't know where that point is, because they're not used to thinking of solutions in terms of software interfaces. In such cases, the most critical point is probably at the UI layer, since helping the user visualize possibilities in that area provides them with a framework for communicating their own needs, which they need to be able to do before a developer spends too much time writing software for them.

On the other hand, sometimes the user needs are already well-defined. In such cases, the most critical starting point may move down from the UI. This will also happen if the points at which the UI will interact with the lower layers are fairly well-defined, such as in Ruby on Rails, which provides a development process that is more-or-less followed consistently when development begins on each new feature in a system. For instance, in your typical Rails application, there is little question how a resource's CRUD interfaces should be implemented, at least initially, to provide some basic functionality to the user.

Yet another scenario where testing below the UI layer is appropriate is when the user is more concerned about whether the underlying functionality is implemented correctly. I once worked for a client who only wanted a basic UI initially - just enough to show them that some of the underlying system functionality could actually be implemented, given its perceived complexity. In that case, the most critical aspects of the system were, in fact, below the UI.