Testing Services provides a platform for QA professionals to discuss and gain insights in to the business value delivered by testing, the best practices and processes that drive it and the emergence of new technologies that will shape the future of this profession.

« Performance Testing in the Cloud | Main | Role of Validation in Data Virtualization »

Predictive Analytics Changing QA

 Author: Pradeep Yadlapati, AVP

Today's mobile economy is changing the way enterprises do business. A recent survey indicates that the mobile ecosystem generates 4.2% of the global GDP, which amounts to more than US $3.1 trillion of added economic value. It is no surprise that organizations are fast embarking on digital transformations.

The pervasiveness of devices is altering interaction as well as business models. Customers expect a seamless experience across different channels. Everyone wants one-touch information and they expect applications to display preferences and facilitate quicker and smarter decisions. 

A cartoon strip published recently captured these dynamic interactions very well. It showed a street vendor organizing his fruits to indicate how people who purchase mangoes also buy apples and grapes. This is the impact of data and analytics - personalized interactions. And this personalization is changing how business models operate, creating a reverse cycle in the sale of goods and services.

Besides enabling agility, the digital revolution underscores the need for a superior user experience. This is critical in the light of studies showing that the average customer tends to shift to a different provider if their response time is over 3 seconds. Thus, to retain customers, one must provide an unparalleled user experience with lightning-quick responsiveness.

Consider the last time you were dissatisfied with a service/product. Typically, you would express dissatisfaction through online posts on social media and experience some level of gratification for being listened to and empathized with. Today, the urge to share experiences is more prevalent - and much easier - than ever before. 

To keep pace with these changing interaction and business models, enterprises want to know: 

  • How can they listen to customers faster to improve their services?
  • How do they build resilient systems through continuous listening?
  • What self-learning systems do they implement to gain accurate insights into what customers want?
  • How do they ensure that testing ensures high quality and a better user experience

The answer to all these questions lies in data. Data yields actionable insights about customers that can be leveraged by testing teams.

The recommended approach is to apply multi-dimensional analytics on 4 data sources to get accurate data. As illustrated in the picture below, enterprises typically analyze defects to understand failure rates, pass rates, closure times, turnaround times, etc. While some departments such as marketing analyze social media to understand customer sentiment, the most valuable source of insights is from machine logs - and this is where enterprises should focus their efforts. 

Let us explore the four ways that enterprises can leverage effective testing to gain a competitive edge and create relevant user experiences that ensure customer delight and loyalty.

1. Listen to your Customer
According to Bill Gates, "Your most unhappy customers are your greatest source of learning." In an age where every sentiment has a digital footprint, companies can understand and change customer sentiment easily through active listening.

Say, for instance, you purchase a Wi-Fi-extender from an online retailer that was delivered earlier than expected allowing you to get connected faster than planned. You may express your satisfaction through positive online reviews. Alternatively, if you were unhappy with your experience, your likely course of action would be to visit the retailer's social media site and express your dissatisfaction.

Social media analytics can track customer reviews and classify them into 'positive' and 'negative'. Negative reviews provide valuable information regarding functional, performance, security, etc., issues. While several enterprises already conduct such sentiment analysis, they often do not share these insights with the enterprise IT owners or the managers of online and mobile testing teams. Sharing insights about factors that impact user experience enables testing departments to proactively address issues by creating new test cases, automating scenarios and building a comprehensive repository.

2. Learn from failures
Every enterprise has a repository of defects captured during each release/sprint. These defects indicate parts of an application that have failed, helping enterprises to evolve smarter testing techniques based on accurate data.

Let us take the example of a bank rolling out a new core banking platform using agile methodology. Each sprint has logged defects in the application lifecycle management (ALM) tool regarding the user story, backlog, area of failure, etc. Since higher functionalities increase the risk of regression failure, enterprises must identify regression-heavy sprints. Here, machine-learning algorithms can be used to mine the defect data and perform predictive modelling to gain insights into failure patterns, which can be further fed into visualization tools such as Tableau, QlikView, etc., to visualize each defect by sprint and module. Such visualizations can help businesses identify vulnerable modules and choose whether to regress or retain error-free functionalities.

With defect analytics, enterprise can easily prioritize what to test and the sequence of testing based on vulnerability while significantly reducing the cost of testing.

3. Insights from incidents
Customer service representatives (CSRs) who handle on-call issue resolutions often capture valuable information during their conversations. Typically, incident management teams analyze the root causes using ITSM tools, thereby gaining information on how to curtail problem scenarios. However, as a direct interface with the customer, CSRs are privy to insightful suggestions from customers on what impacts their experience and how problems may occur in production.

Recently, I faced an issue using an online application to recharge my travel card from my savings account. Despite feeding the correct details, the transfer was unsuccessful. On calling the customer care number, we discovered there were several issues with the application and while the customer representative could not understand why the application malfunctioned, he captured my suggestions for a support expert from level 2 or 3 to analyze it.

Root-cause analyses on incidents are critical to discovering how IT can prevent incidents during production by understanding failures and proactively creating test cases to address them in the future. Organizations can create utilities tools that continuously read incident records, classify them into different categories (such as functional, regression, performance, etc.), create test cases, and feed these into a repository. The creation of test cases for all boundary scenarios allows businesses to get a constant feedback loop that tracks production activities.

4. Predict application performance
From the time an application is developed, it generates a variety of logs related to application, database, app server, web server, etc. Each log captures details about failed code components, error causes, etc.

By analysing these logs, businesses can get information about areas of failure such as modules, code components, database requests, memory overflows, etc. Further, machine-learning algorithms continuously learn from these logs and predict application performance, which can viewed through visualisation tools that offer hot-spot views on potential failures in each module/code component. Thus, testing workflows become more effective by understanding vulnerable areas, sequencing them appropriately and conducting risk-based testing. Coupled with powerful machine-learning, this approach helps testing teams predict the performance of an application before it reaches testing.

The new paradigm of digital-first creates unique opportunities for testing teams to leverage multi-level predictive analytics and get insights that were previously unavailable. Predictive analytics revolutionizes the role of testing, making it a powerful contributor to the end-user experience. To enable testing success, businesses should leverage machine-learning algorithms and enable rich visualisations for better business decision-making about potential issues, thereby delivering an unparalleled user experience. 

Post a comment

(If you haven't left a comment here before, you may need to be approved by the site owner before your comment will appear. Until then, it won't appear on the entry. Thanks for waiting.)

Please key in the two words you see in the box to validate your identity as an authentic user and reduce spam.

Subscribe to this blog's feed

Follow us on

Infosys on Twitter