Testing Services provides a platform for QA professionals to discuss and gain insights in to the business value delivered by testing, the best practices and processes that drive it and the emergence of new technologies that will shape the future of this profession.

June 7, 2018

Quality Assurance In Chatbot Powered Conversations

Quality Assurance in chatbots powered conversations

Conversational chatbots are in phase of revolutionizing the way that interaction takes place with customers across industries as this would replace websites and some applications since chatbots use the most natural method of human interaction - a dialogue. It is the advent of a technology era and simplifies the traditional interaction processes via phone by providing instant, tailored, two way conversations at scale and all through natural human language. Ensuring quality in these interactions paves a way for testing widely. Though the usual web application testing, like functional, compatibility, security and performance testing is required, it is important and necessary to have usability testing first for a great chatbot, some of the top features to test to create an engaging experience and make users come back for more would be Interaction, Answering, Navigation, Handling errors etc. Agile and continuous testing models are excellent approaches to be followed for chatbot testing as minimum viable product can be enriched during each iteration.

Creating a machine that can be useful with novelty features is challenging and ensuring quality for such an entity comes with different obstacles when compared to existing software. The underlying principle for any well performing chatbot is to keep it simple, stupid which means a bot shouldn't be built to answer any question, instead there should a definition in details as what they need to perform flawlessly focusing first on frequent cases, then possible cases and lastly address infrequent requests. Testing methods defined should address each of its components, starting with input, knowledge base, intelligence and reasoning, infrastructure where bot will be hosted, connectivity and voice communication etc.

Industries across have begun transforming their voice process to conversational chatbots and are considering adoption of Chatbots as modern age contact Centre executives, minimizing turnaround time and cost along the way. Due to the novelty of this technology, most users have a tendency to treat the app as a real person during their Interaction with a chatbot, hence testing should be more focused on achieving the consistency of the app which means maintaining the same voice during entire conversation as the users shouldn't feel that they are talking to a different customer representative. Chatbots are required to do more than if/then controls as they can parse text and create their own answers, testers should play with different combinations of inputs and variations of the same input to identify the sytem's ability to understand. Advanced chatbot requires adequate testing and debugging.

A well-developed chatbot should be available round the clock for conversations with customers, provide similar experience regardless of the channel: may it be mobile, web etc., ability to flawlessly interpret various languages commonly conversed, should restrict response format to an easy to understand text, images and unified widgets for better conversations, provide swift response to any query without any delay or making the customer wait, capable of remembering customer interests or preferences and uses order history to recommend products, studies from customer responses to the products advertised, and cross sells effectively, Improves efficiency and reduces turnaround time through speedy information delivery, does mundane tasks at high speed, gives specific user input at each point, learns from client feedback and improvises, thus enhancing the end user experience by understanding the context and responding the user with most appropriate information even to the extent of distinguishing the slang or irony.


Chatbots are excellent examples of software that can be developed using Agile approach, to ensure no bugs are crawling into the bot, testing is good to be performed at each iteration. In the initial stages ensuring business logic, and in later phases automation helps QA teams to deliver improved version into the market, the significant Quality Assurance challenges of the chatbot automation process includes huge number of test cases, lack of previous prototypes, security, and of course lack of business logic prior to starting implementation. Next generation chatbots or interfaces should be more QA tested for behavior, dialogue accuracy, speed, Localization and multi lingual support, compatibility across platforms and collecting the right data and improve etc.  Large number of clients across industries have foreseen the power of these new interfaces and are already investing to create and deliver a value through chatbot powered conversations.

March 14, 2018

Future is Intelligent systems: Are you ready to test them?

Author: Harleen Bedi, Industry Principal, Infosys Validation Solutions

Artificial Intelligence (AI) is frequently making headlines with various possibilities that it offers to make our lives easier and how it is driving innovation in all spheres of our lives. There are opportunities for AI applications in almost all domains/ fields - right from home automation, personal virtual assistants, automated service agents, fraud detection, preventive maintenance, personalized experiences, financial advisory, healthcare recommendations and many more.  Multiple reports or studies predict huge market potential for AI and associated technologies.

Continue reading "Future is Intelligent systems: Are you ready to test them?" »

January 16, 2018

Infosys Business Assurance Store - Fast Forward QA with the power of a Million

Author: Rajneesh Malviya, Vice President and Delivery Head, Infosys Validation Solutions

There is a famous psychological thriller where the protagonist - a patient with anterograde amnesia (short-term memory loss) - sets out on a journey to avenge the death of his beloved girlfriend. To overcome his condition, he uses photographs, notes and tattoos on his body to remind him of various events. I can't help but think that without these cues - these photographs, notes and tattoos - his mission was doomed to fail. So, he needed either his own memory to support him or the presence of some external system that captured memories in different forms. In this particular story, another individual helps him by scanning these notes/photographs to understand the background.

Continue reading "Infosys Business Assurance Store - Fast Forward QA with the power of a Million" »

January 2, 2018

Big data validation for a memorable digital customer experience

Naju D. Mohan, Delivery Manager, Data Services, Infosys Validation Solutions

I have often heard from my colleagues, sales team members and sometimes even clients on what and how exactly do you validate big data and data insights? This is not surprising to me, since big data and occasionally even the insights derived from it is a black box for most of them. For a whole lot of people, it is lots and lots of data which traverses across systems, gets churned by algorithms, which most of us have forgotten after our school years and finally displayed using fancy visualizations making it a mysterious world. Let me make a modest attempt to take you through the journey of big data for a common use case, which we have experienced in our daily lives. I shall take a pause at each step of data flow and explain what has to be tested and how we confirm on data quality. 

Continue reading "Big data validation for a memorable digital customer experience " »

December 13, 2017

An approach for model validation from Functional and Technical perspective

 Author: Kuriakose K. K., Senior Project Manager, Infosys Validation Solutions

Once a model is built and trained using the training data, you need measures to understand the confidence level you can have on the model. In order to gain this, you start using the model in real production scenarios.

In order to validate the model one needs to test the model both from the intended usage perspective in terms of functionality and also validate the models statistical significance from technical perspective.

Continue reading "An approach for model validation from Functional and Technical perspective" »

October 25, 2017

Testing Traditional Databases vs NoSQL Databases

Author: Surya Prakash G., Delivery Manager, Infosys Validation Solutions

When was the last time you heard the words "Big data", "Traditional Database", and "NoSQL database". These words are heard every second now and are widely used terms in every IT organization and across different industry verticals. These terms signify the way technology changes are happening due to huge data that is getting generated through various means (IOT, Social media, Sensors etc.).

Continue reading "Testing Traditional Databases vs NoSQL Databases" »

July 12, 2017

Allure of Cloud may fade away without proper data validation

Naju D. Mohan, Delivery Manager, Data Services, Infosys Validation Solutions

The need to validate integrity, correctness and completeness of data, residing in the cloud is increasing every second with the penetration of mobile devices and the inter connection of computing devices through internet. Cloud seems to be establishing itself as the best possible alternative to meet these data storage and processing demands. Data management capabilities of traditional data stores are revamped to meet the demands for huge volume and variety of data in cloud storage. This calls for new testing techniques and tools for ensuring 100% data validation.

Continue reading "Allure of Cloud may fade away without proper data validation" »

July 6, 2017

Thinking the Unthinkable: An AI approach to scenario prediction

Every now and then QAs are confronted with the uncomfortable situation where a defect is overlooked and it makes its way to the higher environments. This happens despite QA team having supreme understanding of the system under test. Due to the tremendous complexity of the real world applications and the sheer lack of resources (especially time), many flows and behaviors inherent in the application may not be tested and the authenticity of such behaviors remains dubious.  Also, curiously novice users are able to find defects that expert users cannot. Expert users of the application suffer from a syndrome that can at best be described as hindsight bias, wherein they tend to be blind towards the possibilities in the application that they are not used to.

One way to deal with the above mentioned scenario is to hire more and more QAs so that there are more pair of eyes looking at the application hence trying out and finding out more hidden behaviors that can potentially be defects. Due to the cost issues involved, this approach is not very practical. Another approach is that if we can try to simulate the users with different thinking patterns. This approach puts us in the realm of Artificial Intelligence.  We have developed a Genetic Algorithm (GA) for black box software testing that can do just that.

Genetic Algorithms are the heuristic method of optimization that simulates survival mechanics of genes during the course of evolution. They are based on the mechanics of survival such that these string structures that are linear yet randomized, are able to exchange information to form a search algorithm. An initial group of random individuals (population) evolve according to a fitness function that determines the survival of the individuals. The algorithm searches for those individuals that lead to better values of the fitness function through a selection, mutation and crossover genetic operations.

Brief overview of System Under Test:

Our system under test was an incident logging system. There were few requisites  that needs to be fulfilled before user can successfully create an incident. Few of them are:

1)      User Rights and Associations: User with correct rights and associations should be able to create the incident only in their associated domains and should not infringe outside their domains.

2)      Routing mechanism: Un-authorized users should not be able to access create incident screen either through user action on menu items or directly through URL.

3)      Presence/Absence of Data: Presence of data should be there for mandatory fields before user can create an incident. However, there should not be any checks like this for optional fields.

4)      Dependent fields: Dependent fields should not get the data before their parent fields being populated.

5)      Authenticity of Data: User should not be able to create the incident without the correct data in the fields that have look-up value checks.

Synopsis of the Solution:

 Our algorithm placed the factors outlined above as the bits on the string much like DNA. Each factor was recognized by the place it holds on the string structure.

The objective of the GA was to highlight any false positives i.e. the flows in the application that lead to successful creation of the incident when they should not have been created. For this purpose, we gave higher weights to the factors that were less likely to create an incident. For example, a user (A) with CREATE rights is more likely to create an incident successfully then a user (B) that have just READ rights. So, in this case user B will get significantly higher value of weight than user A as he/she is less likely to create an incident. Same kind of weight distribution was done for other factors as well.

Finally, an equation was created that will calculate the fitness value of the individual string structure. The equation was designed to give higher values of fitness to the individuals that resulted in successful creation of incident even though when the odds of such a thing happening was less. The fitness value of first generation of 10 strings were calculated and then mutation and cross-over events were performed to create the offspring of 10 strings. If the parent was having higher value of fitness then it was retained else child was selected. Once this selection process was over, again the same steps were performed to generate next generation. This process was repeated till the n­th generation. After which human analyst can look at the resultant strings and verify whether these flow really are defects.

Advantages of GA based software testing over traditional testing:

  1.  Able to explore various scenarios hence improved coverage of the system, many of the scenarios may not even cross the mind of the human QAs.
  2. Increasing the Defect Detection Efficiency leading to improved Quality of the system.
  3. Easy to change the directionality of search to find new defects by changing the weights associated with the factors.
  4. Substitute for human testers with the advantage to work on non-business hours including weekends.
  5. High risk area in the application can better be covered by increasing the population size and number of generations.

Associated Challenges:

  1. Technical expertise required to build the framework.
  2. Clear understanding of factors affecting the application is required. It is still not Subject Matter Expert (SME) agnostic. The bias of the SMEs may affect the outcome of the runs.
  3. Hardware intensive. Procuring the advanced computation resources may not be easy in the client environment.

This is a humble attempt to increase the software quality and provide end users with less grief due to missed defects without escalating the project costs.  In the last few years there has been significant involvement of Artificial Intelligence related technologies in most of the aspects of software yet software testing seems to have been eluded by the riches it provides. I sincerely hope that software testing has lot to gain from the advances in the AI making software solutions and products more reliable and efficient over the passage of time. 

Continue reading "Thinking the Unthinkable: An AI approach to scenario prediction" »

March 8, 2017

DevOps at Enterprise scale

Author: Varun Rathore, Delivery Manager

The current business environment makes many demands on organizations. With the proliferation of start-ups becoming a game changer to the existing brick and mortar industries, the need to innovate, iterate and stay relevant, is enormous. In view of this, UBS has embarked on a journey to transform itself through DevOps practices

Continue reading "DevOps at Enterprise scale" »

November 26, 2016

Predictive analytics - An emerging trend in QA

Author: Indumathi Devi G., Project Manager, Infosys Validation Solutions

As digital transformation is rapidly changing business operations, quality assurance (QA) also has to change from traditional quality control to intelligent quality assurance. Nowadays, clients not only want to test software adequately, but also as early and thoroughly as possible. To accomplish these goals, it is important to opt for shift left testing and predict the failures even before the applications are handed over for testing. Today's business dynamics require QA professionals to make critical decisions quickly. It is imperative to make use of the avenues, such as customer feedback, defect data, and test results available at disposal to make prompt decisions.

Continue reading "Predictive analytics - An emerging trend in QA" »