One of the biggest changes in thinking necessary for a Tester in an Agile world is the concept of Just-In-Time (JIT) Test Planning.
One deliverable that was expected of us (I feel) in the Waterfall model is the Gigantic Test Plan and a HUUGE Test Suite with ‘000s of Test Cases. It is, afterall, one way to guarantee confidence in a product.
But ‘000s of test cases take a long time to plan and execute, which simply does not fit in the Agile world, or indeed in any RAD world. Not only that, but it doesn’t really make sense. As Anne-Marie Charrett said at STANZ 2011, “Why spend all your thinking time planning how not to think?“. It makes us inflexible to change, it reduces the actual testing time (ie: testing, not just following a script) and it reduces the efficiency of the testing when the tester actually gets a chance to test something.
Here’s where you may expect me to go on to talk about Exploratory testing and how we are using it at Seek, but while it is definitely a big part of the way we do things, there is still a higher-level need to plan our testing and where Exploratory Testing would sit alongside Automation and Manual (Scripted) Testing. After all. sometimes we need to script a test for something that can/should not be automated (eg: de-coupled systems, timing dependencies, DB Hacking dependencies, etc). Hence JIT Test Planning.
The concept of JIT as it applies to Manufacturing and Development is nothing new. As anyone who knows anything about the history of Agile development would know, it’s been around since the 60s/70s as one of Toyota’s techniques to meet fast changing consumer demands with minimum delays1.
JIT Test Planning is a strategy of spending more time working with the BA in defining the requirements before taking those requirements and defining the Plan that will define how you test them.
One underlying goal of the way we are doing things is to avoid as much duplication of documentation as possible. ie: if the User Story says Button A needs to do X, do we really need to right a separate document telling the tester to press Button A and to make sure X happens? The manual test case works in collaboration with the User Story. It allows the Tester to define the most efficient way to verify the Acceptance Criteria without repeating the Acceptance Criteria word for word. The test case now focuses on providing instructions on how to test something, not just what to test.
Probably the best way of proving the beneficial effects of JIT Planning is to show an example graph from one of the projects it has been used on. Note: The following graph covers manual testing only.
The main thing to notice (besides the high quality) is the lack of ‘Grey’ area, where Grey is planned un-executed testing. No grey means No wasting of time. The lack of Grey is the result of two things:
- We do not begin planning until all Acceptance Criteria have been detailed and the User Story has been accepted by Dev and QA.
- Because Dev and QA begin at the same time, by the time QA is ready to test, there is already a product to be tested.
In Summary, JIT Planning gives us the clarity, relevance and accuracy of requirements as they are generated in collaboration without months/years between Business Analysis and Development as well as the confidence in Test Coverage. It allows a much greater flexibility and acceptance of change (the Agile way) and it all works in collaboration with Exploratory and Automated testing. I’ll talk more about the end to end process we follow in future posts.