Traditionally, testing occurs at the end of every development cycle, but as more organizations move towards a continuous delivery model — like DevOps, where software is constantly in development, yet always ready for deployment — this traditional model breaks down. With continuous development, we must also have a method for continuous testing … enter Test Automation.
I had the opportunity to virtually sit down with Henika Tekwani, who leads Appirio’s Testing Automation practice. Over the last ten years, Henika has gained experience in software testing and test automation. Henika shares her firsthand experience of not only successfully running automated tests, but the resources, skills, and technology it takes to get there.
Q: In addition to supporting a DevOps model, what are the advantages to test automation?
A: Test automation is re-usable at it’s core, generating tremendous time savings at nearly every stage of the development and testing life cycle. Additionally, test automation is scalable and customizable, so it has the ability to grow and adapt with an organization over time. Last but not least, it enables you to test consistently between each environments and releases resulting in a consistent and positive user experience from day one of go-live.
Q: What are the contributing factors to a feature being “automation eligible”?
A: Test automation is best suited for functionality that’s likely to cause regression issues. Therefore, selecting test cases for automation that help in detecting regression issues becomes crucial. Automation helps eliminate repetitive tasks from the testing life cycle (such as multiple input data test cases, deployment time testing, daily smoke testing, regression testing of features from previous releases, etc.), so that manual testers can use their skills to focus more and find new use cases and bugs. Human-centered design is integral to delivering a successful solution, so user interface (UI) elements like fonts, colors, and graphics still go through manual testing steps.
For new application functions still in development and those that evolve and change frequently, creating automated scripts may not be worthwhile. Therefore, its advisable to start automating them once the feature is stable.
Q: How can organizations determine the ROI of test automation?
A: There are two primary factors used to determine the relative ROI for test automation — efficiency and risk.
- To evaluate the potential for efficiency gains, we focus on the actual efficiency of the automation, in terms of hours. Typically, eight-hour time blocks are used when calculating manual test efforts — but since automated tests can run up to 24 hours continuously, the factor is increased. Due to interruptions that could occur, 18 hours is a reasonable factor to use when calculating efficiency.
- The risk is more subjective and it can be hard to calculate the risk of not adopting test automation. However, the overall risk is reduced when automated testing is adopted as resources are freed up to focus on more productive tasks, like conducting in-depth analysis of application, test design, and negative testing.
Q: For customers looking to build out an automation team, what characteristics make for a good test automation resource candidate?
Q: What support should be provided by customers who are looking to adopt test automation through Appirio’s TestRight solution?
A: We will work closely with the customer throughout the development and testing life cycle. The customer is a key contributor to understanding testing priorities. Often, customers will share regression test cases, as well as access to any test cases their team may plan to run. We will need support from their business admins and project managers to answer questions from the QA team while conducting tests. If they have an automation team, we will work instep with them as they provide infrastructure support, and cross-train their team to support ongoing test automation. We will work with the customer team to determine KPIs, like volume of test cases in a given time frame, so that they are able to quickly determine if the build is good to go.
While speaking with Henika, I learned that while test automation has it’s benefits, not everything should be automated. The advantages of testing automation are clear, and for companies moving to a DevOps model the move to test automation will be mission critical to success.
Are you wanting to learn more about test automation? Visit our Appirio Hub for more test automation resources and to learn more about TestRight.