Software Testing Challenges in the Iterative Models

Iterative / Incremental life cycle models, as the name suggests, focus on developing software in increments or chunks. Popular iterative models include the Agile methodologies such as Scrum, XP, etc.

A common thread amongst the iterative models is that integrated and working systems are produced and delivered at the end of each iteration. Business functionality is divided into chunks.

Each iteration delivers a chunk of functionality that is integrated and tested. From a testing perspective, the software testing group will get testable systems early in the lifecycle unlike a sequential approach. However, iterative and incremental models can pose their own set of challenges to the testing team.


Challenges in Software Testing

The main challenge for testers in agile / iterative models is that the system is constantly changing. New code is being checked in regularly and testing cannot wait for or expect software that is in a final state. For the testing team, the software is a moving target. Software Testing teams need to adopt different approaches, tools and strategies to handle this level of change.

In iterative models, the riskier / more important functionality and features tend to be built in during earlier iterations / increments. Subsequent increments add more functionality on top of what was built in the previous increments. Given the amount of changes that can potentially happen in each increment, the risk of regressions is high. With each new increment, it becomes very important for the testing team to regression test all the features provided in the previous increments to make sure nothing is broken.

Another interesting facet of iterative development is that activities across increments can overlap. It is not usually the case that one increment completes entirely before the next one begins. For example, the testing team will generally be testing the current or previous increment’s build in detail while the development team works on the next increment’s features.

Of course, while testers will also in most cases, be required to help with incremental testing of new features in the increment that the developers / programmers are working on, the primary responsibilities of the testing group is to thoroughly regression test the last increment’s build plus the new features delivered then. This overlap can pose challenges in addressing bugs that the testing team identifies and time taken for fixing.

Software Testers also need to be aware that iterative models such as Agile are not really in favor of heavy weight processes or formal methods. There are extreme cases where the value of having testers is questioned. But again, that is the extreme. Most mature agile teams realize the immense value that testers can add in producing a quality product. Testers need to tailor their methods to suit the agile world.

Testing groups need to understand the fact that the organization does not really want “zero-defect” or “defect-free” software. While stakeholders will obviously not state this and will continue to ask for defect free software, the organization will not be ready to make investments necessary to reach that objective or even close to it.

In reality, as we approach the objective of defect free software, costs keep increasing to a level (theoretically to flush out all the bugs in the system will cause costs to tend towards infinity) where organizations may not want to invest further. Finding the right balance between costs incurred and value derived is important.

Leave a Reply