The need to innovate and respond to changes in the context and preferences of customers makes it essential for organizations to update or launch technological solutions much more frequently.
Given this requirement, software development teams respond with the adoption of agile practices, DevOps, automation and continuous testing.
Optimizing testing processes is a key issue nowadays, in a scenario when keeping quality (without increasing costs) is another unavoidable necessity, since in many cases client satisfaction and the chance to bring the client an ideal experience depend on the quality of the software.
Automating QA testing processes appears as a very tempting option to face the demand of faster delivery cycles and error-free launches. This strategy also enables fault detection at the beginning of the software development work, which makes it cheaper to fix them. Now, does testing automation make sense in all cases? To know that, it is important to be able to measure the return over investment (ROI), which, by the way, is not so simple, since many variables come into play.
Return over investment
The formula to calculate QA testing automation ROI does not differ from that which is applied in other situations: net income (or net savings) divided by net investment (i.e., the sum of the tools and resources to be used), multiplied by 100 (since ROI is expressed as a percentage). But, of course, when calculating this quickly, some challenges arise. For example, defining which concrete factors must be taken into account when calculating both income (or savings) and investments.
Calculating the automated testing return over investment is, on principle, important to avoid making some common mistakes. For example, the cost of creating, developing and maintaining automated testing is frequently compared against that which corresponds to manual testing. And although it is true that with automation at least 80% of the time devoted to manual testing could be reduced, there will still be a percentage that must continue to be done the manual way -basically, those tests that call for human observation-.
Besides, it must be taken into account that after automating tests, they will need updating and proper maintenance, which should also be considered in the calculation of the ROI. Tests are software. And when estimating the ROI, one should bear in mind the ways in which tests will change over time. Part of that change will stem from adding new characteristics and functions.
Technological solutions
The key to a successful automated software test is having good tests, since there are currently different automation solutions. However, if said solution will be performed in house, it is convenient that the ROI calculation contemplates the sourcing cost or the training of people who design and develop the adequate automatic testing solutions.
On the other hand, testing automation has some intangible benefits to bear in mind. For example, the chances to avoid delays in error fixing, to perform quicker deliveries, to increase launches per year and to improve the work environment (by replacing repetitive and manual testing cases and focus efforts towards more challenging projects).
There are also a series of variables that will make each case unique and therefore the ROI calculation not wholly transferable from one organization to another. For example, the commercial requirements to be implemented, the test percentage likely to be automated, the complexity of the tests, the amount of settings that need testing, the number of launches per year, etc.
Some estimates indicate that testing automation may make sense for organizations with at least 50 business processes and who launch software products at least 4 times a year.
Automated QA testing
Anyway (and though it is not easy to calculate, since it also depends on each case), the highest automated testing ROI is probably given by reducing the risk of introducing a fault in manufacture (by covering tests close to 100%) and the subsequent fixing costs. By avoiding the introduction of faults, the organization may avoid a potential business interruption.
This being the case, from the savings (or income) side, the formula should contemplate the saving in tests plus the saving in the reduction of risks and fixing costs. From the investments side, for its part, the automated testing tools license will be the cost with the highest impact. But it will also be necessary to consider the resources available for installation and setting and the learning curve.
Has a software testing automation strategy been implemented in your organization? Has the ROI been previously calculated? It would be great if you shared the case!