The Infosys Labs research blog tracks trends in technology with a focus on applied research in Information and Communication Technology (ICT)

« NextGen User Interfaces | Main | Towards a Consistent and Responsive Web design »

Analytics in Software Testing

Use of analytics in Software Testing leveraging AI, ML & big data analytics etc. focuses on identification of potential problem areas and suggestion of next best actions for high-quality product delivery. Automated software testing using analytics is essentially quality control for operational aspects of the product. With, larger aim to create a robust testing process operating through one or multiple test automation frameworks. Type of analytics used in testing can be categorized into three types namely a) descriptive analytics; to create simple counts, distributions, visualizations describing test data results, b) predictive analytics; to predict organization processes for example predicting types of testing defects at the beginning of release, c) prescriptive analytics; to prescribe corrections and suggest mitigations after identification of risks.

From a market statistics perspective global market for automation testing (AI, Big data & IoT) solutions is expected to grow at a CAGR of 17.7% from 2018 to $19.27 billion by 2023. And, software testing services is maturing market and expected to grow by 6.5% over the 2016-2021 period to $27.9 billion. With specialized testing gaining pace at 48% of total spending, and digital transformation testing is the fastest growing segment at 15% CAGR.

Need for analytics in testing generates from the fact that expected outcome and zero defects has become hygiene factor for mission critical and high-risk applications. In spite crucial importance of software testing there have been many instances where software bugs created major disruptions in business operations. For example, Amazon Christmas glitch, HSBC payment system failure, Microsoft azure crashes, Screwfix price glitch, HMRC's tax blunder etc.

The Traditional Testing methods are static in nature and designed to detect possible defects in early stages of SDLC. With the goal to find lexical, syntactic and even some semantic mistakes. Also focus is more on functional aspects where system is tested around functional requirements and specifications with less focus on the specialized services. Traditional testing methods relies more on descriptive and diagnostics analysis of post testing improvements. And with lot of manual processes and lack of automation traditional methods of testing usually takes weeks or months for software rollout. Also, there are major drawbacks in the traditional approach like less customer focus, reactive to feedback, difficult to detect and isolate defects, and adaptation to new environment etc.

There are few drivers for adoption of analytics into software testing methodologies:

·         Mission Critical Dependency: With mission critical and high-risk applications running on software applications, testing is expected to achieve zero defects and trusted quality. Many cases software bugs created havoc for the customers, companies and its reputation. 

·         Financial implications of bugs: Besides the hit to the customer experience and company reputation, many of the examples of software bugs/failures have resulted in huge financial cost implications for the organizations.                

·         Emerging Tech: Massive adoption of mobile and cloud services have impacted Test Environment Management (TEM) as well as Test Data Management (TDM), however traditional approaches fails to deliver required level of efficiency.

·         Faster on-going testing: Advent of Agile & DevOps has made shorter software development as new normal, but even more is needed to meet today's demand for faster go-to-market (GTM)

The use of analytics in software testing will positively impact efficiency and effectiveness quality process as well as overall release cycles. There are multiple features of analytics which can be leveraged into the software testing landscape like AI and ML led platforms can help in defect analysis, log analytics, test prioritization, improvement in overall test coverage, and automated test case generation. Predictive analytics to spot future failures in view of the past data sources. Enabling pre- cognitive defect healing & proactive defect prevention. Advanced Visualization in form of real time data of test performance, test history charts & Error logs in dashboard can create significant grounds to improve the testing procedures of the future. Touchless fully automated testing across software development life cycle. Along with robotics, ML, conversational AI or natural language processing etc. Use of various analytics techniques such as ANN, NLP, reinforcement learning, deep learning etc. for automation of the testing methods and processes. Use of open source test automation solutions like Selenium and Appium for web and mobile have a high impact in automation and digital transformation.

Many players in the market are showcasing their 'analytics in testing' capabilities via AI/ML. Some of the AI powered tools in the market are NIA, Applitools, SauceLabs, Testim, Sealights, Mabl, ReTest etc. There is huge potential for touchless fully automated analytics based software testing which focuses on automated defect identification as well as automated healing (via bots). Most of the competitors has rolled the solutions under the umbrella of QA services solution or automated software testing solution.

Post a comment

(If you haven't left a comment here before, you may need to be approved by the site owner before your comment will appear. Until then, it won't appear on the entry. Thanks for waiting.)

Please key in the two words you see in the box to validate your identity as an authentic user and reduce spam.

Subscribe to this blog's feed

Follow us on