Test Automation ROI Calculator by Ananth Krishnamoorthy

Test automation is often seen as a way to reduce the costs of testing, increase test coverage and effectiveness, and shorten testing cycles. In fact many software organizations consider automation as a vital step in establishing a mature QA program and it certainly has a lot of value if it can be effectively leveraged. However, it can never eliminate the manual testing, which is crucial for thorough testing of software applications, completely.

Automated testing involves higher upfront costs and should be looked at as a long-term investment where the pay-offs come anywhere between 2-4 years down the road. One has to keep in mind that there are various intangible benefits associated with automation. Performing a return on investment (ROI) for your planned automation can however help you understand right at the beginning the actual returns that you will get from your investments and you can weigh those against the benefits you will gain from automation. An ROI analysis will not only help you determine the various elements associated in calculating the ROI and the approximate cost and benefit involved but it can also help you decide on the types of automation you want, the areas that you can potentially automate, the tools and the skill levels of the testing resources that will be required.

CALCULATING ROI

Quite often, automation is considered a transition from manual testing. As such the cost benefits from automation can be viewed as trade-offs in comparison to manual testing. However, the comparison here is more of looking at the investment and see in a long run how it pays-off. This should persuade your decision towards automation transition.

The key parameters to be considered while initializing the ROI exercise are:

Automate the entire QA system rather than building scripts to test the product

Understand the focus of automation with respect to reducing testing cycle time, maximizing test coverage, reaching the market faster, cost benefits, etc.

Estimate the existing test suite with respect to effort and cost

Assess test automation tools

Understand the product and its technology road map

Test Automation Framework design

While these parameters can aid in doing an initial analysis and determine the high level ROI, the results obtained can be used to delve into the details of calculating the real ROI.

AN IDEAL ROI CALCULATOR

An ROI calculator is the one which helps you understand the possible returns on your Test Automation Investment. Hence, before you decide on this strategic investment, it is vital to know the cost and effort savings that your automation effort can offer.

An ideal approach in calculating the ROI entails analyzing the effort and cost involved in Automation, based on the background information (existing manual testing details) that you provided as input. The practical steps that are involved in the transition process from manual to automation are shown in the block diagram below:

Based on your input, the calculator derives both manual and test automation testing parameters that are the only basis for calculating the ROI.

YOUR INPUTS TO COMPUTE ROI

For a successful ROI computation, you need to have the existing test parameters of the product under test:

Age of the product

Releases planned for a year

Required regression cycle per year

Number of existing regression test cases

Number of configurations to be tested

Cost of the testing resources

Infrastructure cost

Apart from the above background information, the following parameters are needed to be focused on:

Test automation tool cost

Percentage of test cases that can be considered for automation

Time required for building reusable components, automation frameworks and batch scripts

Hourly cost per test automation resource

TEST AUTOMATION ROI

The transition from manual to automated testing has a significant impact on the Test Automation ROI. It is an established fact that Automation involves higher upfront costs, but the optimal usage of this initial cost is very crucial in deciding the returns. The transition process must be a step by step process and has to be done by experts. The first step towards automation is to scope the automation requirements, analyze the technical feasibility for automation and identify an appropriate tool for automation. This will help you to estimate the required effort and cost for test automation investment.

When the estimation is being done, it is vital to consider the effort required for designing test automation framework, which will help you to use reusable components through out the automation and thus helps in reducing the script development time. These factors will help you to come up with a right estimation and you will be in a position to look at the real benefits of automation.

It is always worthwhile to understand and analyze the need for automation, analyze the product test requirements and the existing regression test suite, identify the right test automation tool and then estimate the timeframe required to design and automate the entire testing. This exercise would benefit in finding your potential automation ROI before investing in it.

A higher ROI will also come through with the following factors:

Matured testing process

Maximized test coverage which will assure the quality of the deliverables consistently

Elimination of the risk of over sight of quality in repeated tests

Faster Execution of the tests in multiple environments

More focus on new features

Enhanced product quality

KEY TAKE AWAY FROM ROI CALCULATION

Automation may not give you its benefits immediately. Moreover, you would need ample time to transition your QA efforts from Manual to Automation. Your Manual QA would also have to continue in its own pace in ensuring quality until the automated execution begins. This means though you invest in the first year, you will be able to see the cost benefits tentatively start only in say, the 3rd year. And this includes the additional cost required in the 1st and 2nd year.

After the breakeven of the investment in Automation at 3rd year, you would be getting higher ROI over the years. This is due to the reduced test execution cost and effort. The returns can still be maximized by having disciplined set of processes during the transition phase.

CONCLUSION

Automation is a strategic decision and an understanding on the potential ROI will help in the decision to invest in automation. It is highly recommended that you calculate your actual ROI and understand the factors that will influence and enhance your investment and get the best out of automation. While automation does help in reducing the costs of testing, increasing test coverage and effectiveness, and shortening testing cycles, it is not a silver bullet for all your testing problems and can never completely eliminate manual testing. It is but one means to ensure better quality software. You need to get the Test Automation right which can lead to a successful product in the market.

Aspire Systems has developed a Test Automation ROI Calculator with which you can determine a high level approximation of effort and cost.

It is available at http://www.aspiresys.com/testautomationroi/

About The Author

Ananth has more than 10 years of professional experience in Product Testing, including a substantial amount of time spent exclusively on test automation efforts.

The author invites you to visit:
http://www.aspiresys.com