Automated versus Manual Testing – Make an Informed Decision!

Being a manual software testing resource, like many others I have lived the past few years under the constant fear of my job being taken over by the hype of “Automation”. I have worked in domains ranging from telecom, hospitality and retail and have mostly followed the principles of old school manual testing for ensuring quality for a variety of software systems/applications/websites etc. Not that I haven’t had a chance to pick up a few automation skills, but I instead chose to do a rational analysis of manual versus automation rather than quickly switching roles.

After weighing the two sides against different parameters, I realized that there will always be space for both in the industry. And thus, I decided to stick to what I love best, thinking of every possible click, trying to figure out the various turns and U-turns that a software journey can take, put a system through a subjective as well objective analysis and then accordingly designing the manual test scripts. The argument that automation will eventually annihilate manual test practice, I think is way too optimistic. It’s like saying that robots might someday take over everything that a human does. I have my doubts for any of this to happen.

Automation no doubt has its advantages and can complement manual testing. However, for any company, program or project, it is important to continuously evaluate and counter balance the two.  Automation, manual or combination of both, the decision is driven by certain key factors. Some of these are:

Repeatability

  • It is one of the prime factors that drives the automation versus manual testing decision. Automation is no magic wand; it requires specific skill set, tools, time, effort and budget. Is there a good ROI for investing in so many resources? Definitely, yes if there is a high reusability of the automated test scripts. Thus, for a project that has multiple runs of the same test scenarios or that requires regression tests at regular intervals are some of the best candidates for automation.

Cost & Benefit Analysis

  • Automation Testers, frameworks, tools, maintenance, training all add up to contribute majorly in project costs. A Test Lead/Manager has to weigh this additional cost against possible benefits. For example, for a short term project or a minor change it does not make sense to automate. In such a scenario Manual Tests will be more cost efficient and would add equal value as automation might as well do.

Project Type

  • Another key driver when deciding between manual and automation testing is the size, scale and type of project. Automation can be helpful when there are too many data driven tests where same tests need to be run for different data, or there is a multi-browser test involved. On the contrary a project that is highly GUI oriented or lays emphasis on customer experience; manual tests would be best suited to them and would be cheaper as well. Similarly, if it’s a large project with too many testers working on it, with both front end and back end to be tested, it would be perfect ground for a mix of automation & manual testing.

Value Addition

  • And finally as a PM or test manager one needs to ask the question – does spending on automation adds value to the project and does it really saves time and effort. If it does then just go for it. If not, it’s probably not worth it. Adding value can mean different things in different projects – uncovering additional bugs, uncovering bugs earlier in the test cycle, uncovering bugs in lesser time, quicker retests, fewer resources required to test or simply making life easier for manual testers.

In a good sized project with a variety of testing to be carried out, manual and automation testing should tie in together. Instead of dismissing each other, automation and manual tests should be used in a collaborative manner, so as to get the best out of both.

Even though testing as a subject is never really taught in universities, it still can’t be termed as a job that anyone can do. One has to have a specific mindset to be a good test professional. A mindset that looks at a system at large, a mindset that always seeks the visibility to overall objective and purpose of a system under test, a mindset capable of constructive criticism so as to identify the break points of an application, to think from the perspective of not one but various types of users, to simulate a user behavior that the software is not ready to tackle. And this is why automation testing cannot take over manual testing. Automation can provide the tools to implement and execute, but it needs a manual (read human) tester to do prep work.

Automation can be very effective and cost friendly in the implementation of some of the objective scenarios; however, for a subjective point of view there isn’t an alternative to manual testing. It is wise to automate that can be quantified but in my experience quality cannot always be judged by facts and figures. For the qualitative factors companies still need the human interface.

5 Replies to “Automated versus Manual Testing – Make an Informed Decision!”

  1. I agree that manual and automation tests have to be balanced.

    By my opinion automation tests are more important in the world of continuous integration and delivery, but it doesn’t mean that are more important than manual tests. That’s why regression tests have their place in automation (and you mentioned it in first factor).

    I would like to emphasize also the difference between severity and priority. Even a minor change can cause a lot of problems when is in frequently used part of application.

    Thank you for your article. I like especially the sentence „One has to have a specific mindset to be a good test professional.“

  2. Very nicely articulated. And fully agree with the fact that automation testing cannot replace manual testing completely .

    Being in automation , I can vouch for this fact , the main reason being is none of the systems can be fully automated. Also with lots of system / env issues , the scripts are able to run successfully 70% . There are times in automation when we manually verify the results or in some cases we do partial automation only

    As you said , robots can never take over the world , the same goes for this well., we need a human brain to analyze the results considering all the perpestives rather than just giving out a result

  3. The thought of both manual and automation testing approaches is well explained. One cannot replace other. Nowadays sr management all around you think automation will replace and is must. This is must read for such people.

  4. Great post….Usability testing is a wide area in software testing and it requires genuinely high level of knowledge of this field alongside a creative thinking ability. Individuals taking part in the usability testing are required to have aptitudes like persistence, capacity to listen to the advices, openness to welcome different ideas, and the most critical of them all is that they ought to have great perception abilities to spot and resolve the issues or problems.

Leave a Reply