Testing Services provides a platform for QA professionals to discuss and gain insights in to the business value delivered by testing, the best practices and processes that drive it and the emergence of new technologies that will shape the future of this profession.

« May 2013 | Main | July 2013 »

June 21, 2013

Dynamic systems can't have static solutions


I recently had an interesting conversation on cost of quality (COQ), where a client wanted a 'simple equation to model COQ'. I am sure it does not require a mathematical genius to come up with an equation to calculate COQ, given the values of all the factors that contribute to it are known. We had a long debate on the reason why the formula to calculate COQ cannot be a model to predict it.


The discussion was focused on using the formula and a linear model for predicting the values of individual contributing factors to predict COQ. All seemed so easy and straight forward - we have COQ which is the sum of several cost components and we know the values of those factors on a time series. All we need to do is use some techniques like a curve fitting to project the trend and predict the future values, add them all to predict the future COQ. It took us a while before we could agree on one thing - we need to look at the problem differently!


One of the key revelations from the brainstorming that followed was that many of the 'variables' that we considered varied at varying rates. Of course, variables are expected to vary, but we assume that they follow a fixed relationship (a static model). Usually we use techniques like curve fitting to arrive at a trend and use it to predict the future, which may not be appropriate. For example, a schedule pressure will increase when there is more pending work. This will push productivity up to certain levels and quality down. We found a few more of such dependent relationships involving skill level, process maturity, quality of inputs etc.; finally, we concurred that principles of System Dynamics are a better fit than the conventional models in such situations.


System dynamics is a perspective and set of conceptual tools that enable us to understand the structure and dynamics of complex systems. It allows us to model complex relationships between system components and model their behaviour based on the different mutual relationships and feedback loops. This makes the model simulate results that are realistic as statistics such as rates at which the variables change, the impact of feedback loops on the behaviour of the system and system's behaviour at each instant are taken into consideration.


We at Infosys are experimenting to model a system that can represent the COQ. For this, we are building the various causal loops and a set of components that contributes to or can influence the COQ. The intent is to use this model to predict the COQ in a given context. It will take a while before we complete the exercise. However, the way modelling of the system is shaping up, we are becoming confident about the way we approached the problem.

June 13, 2013

Testing for an Agile Project - Cool breeze in Summer


For a software testing professional, working in an agile project is no less than a breeze of cool air in the hot summer. The point that I am trying to drive is that for a tester, who works regularly in projects that operate in the traditional model, the agile model is a very welcome change. Let me tell you why...

As testers in non-agile projects, some of the challenges we face are

· Timing of our inclusion in the project

· Utter dependence on documentation

· Coping up with abrupt changes in requirements

· Insufficient channels of communication with the Development Team or other stakeholders

Let me explain how each of these aspects is handled in Agile projects in turn making our life simpler:

· Timing of our inclusion in the project

Normally, we are almost always included somewhere in the middle or towards the end of the development project.  By this time, the development team would have completed most of the coding. We end up uncovering defects in requirements and design when the coding is almost over! With each defect we find that is attributed to an earlier phase in the life cycle, we are pushing ourselves into more trouble! Then we need to plan more re-tests to ensure that the fixes have not broken the other parts of the application. This increases the cost of the project as well as the risk of not adhering to the time-to-market deadlines.

In contrast, in agile projects, we are included in the project, at the same time as developers. The 'how to test' section of the user stories are documented before they are coded. The development team takes guidance from the documented tests while coding.  This approach avoids the number of defects and hence reduces the re-testing effort


· Utter dependency on documentation

In non-agile projects, the entire planning, estimation, scheduling, test case identification, etc. depends on the provided documents. We never get a chance to participate in the requirements gathering exercise. We only get to 'analyze' the requirements after they are documented. If the requirement documents are not detailed enough, we can miss testing something important. Also, it has to be ensured on an ongoing basis that we and the analysts/developers are working with the same version of requirements in the project.

On the other hand, in agile projects, we are an integral part of the team. Estimations and Scheduling for the tests are done in sync with the overall development planning, estimation and scheduling.  We have a chance to participate in the requirements gathering exercise and would have the same understanding of the requirements at any point of time as the developers. Requirements in agile projects are documented as User stories, which are simple and detailed enough by default. So, there is no scope for missing requirements.  For each user story that is being documented, we discuss 'How to test' part with the developers and ensure that these test cases are documented. Developers write the code to ensure that our tests are satisfied.


· Coping up with abrupt changes in requirements

We hate requirement changes because the requirements are not independent by nature and so any change may also impact other requirements leading to significant re-work. We always have the testing window fixed and have to adopt techniques like risk-based testing, to select the tests intelligently for re-testing. There is never the luxury of running all the planned tests for every requirement change.

In agile projects, since the development as well as testing is done in an incremental fashion, handling requirement changes are easier. If the requirement change leads to newer user stories, these are handled in the same fashion as other yet-to-be-code-tested user stories. Care is taken to move them into appropriate future sprints so that the integration testing is logical. If the requirement change is within the user story, handling it is simple as it is limited to updating the test coverage for that particular user story  as each user story is an independent entity. Hence we work in a more disciplined atmosphere where we are always in control of what we do.


· Insufficient channels of communication and collaboration with Development Team or other stakeholders

In non-agile projects, there are multiple challenges with respect to communication and collaboration. Since, we do not have the opportunity to collaborate in real time, we depend heavily on documentation and spend time on emails and phones to sync up with the other stakeholders on the requirements and issue clarifications rather than on any innovations, which could bring down the cost of the project or improve the testing cycle time of the project.

In agile projects, we are engaged with the other stakeholders, right from the Initiation phase. We interact with the larger team, using multiple channels of communication, all in real-time. We have the same view of the application as the other stakeholders  as we participate in Release planning and Sprint planning. We can quickly and efficiently resolve issues as daily standup meetings provide a chance to discuss issues as soon as they arrive providing real- time handling of defects. Constant communication among stakeholders provides a platform to debate each other's perspectives and opens a window for innovation. Surely Agile projects provide myriad opportunities for rewards and recognitions.

Now, isn't all this cool? I am sure most of the tester community would agree with me.


Do let me know your thoughts on the same.

June 12, 2013

Top 4 cost hurdles while doing on premise testing

In my previous blog, I have mentioned how on-premise testing activity brings in delay in an actual testing. In this post, I will uncover expenses suffered by the businesses while doing the same. Let's take a look at what eats away your hard earned money.

·         Testing Tools cost

A sizeable portion of your business expense pie is spent on procuring and maintaining a large pool of testing tools, which come with the hefty amount of licensing cost associated with these. Apart from that, these tools require periodic upgrades to the latest patch or hotfix level to bring it to the latest version. These patches or hotfixes may not be available free of cost incurring extra purchasing cost.


·         Owning Hardware Infrastructure

It goes without saying that any testing project requires a finite number of compute resources (Server, Storage, Networking device) to carry out the job. However, the challenge is to scale it up in case of performance testing. Ideally, to do the stress testing or load testing, a production like scenario is required which is usually not the case. Even to replicate a production like scenario, business needs to spend a huge amount of money, either to procure the infra or to rent it from the market. We have seen most of the organizations test in a scaled down environment. This is not advisable as it can hamper the quality of testing of the applications under test and may result in few of the defects go undetected.


·         Energy Consumption

There are indirect costs associated with the above activities which often get overlooked. Cost incurred due to energy consumption is one of them. Decent testing environments have specific wattage requirements, which if avoided could have contributed to the 'Go Green' initiative that the Global IT corporate houses are vowing to go with.


·         Real Estate Cost

This is another indirect cost that automatically comes into existence when a testing platform is required to be hosted, either in house or outsourced to a third party.



All of the above expenses are seen to be an overhead to the organization as it is not directly linked to its business objective. In my subsequent blogs, I will be discussing "how Cloud Computing has relieved business from these overheads and has ushered in a new business model".

June 4, 2013



Continuing the Next-Gen QA series, we take a look at the fourth parameter that will help in the paradigm shift - QA BASIC TO QA ADVISORY SERVICE.


QA organizations are moving up the value chain and are expected to play an advisory role based on their experience in testing delivery. Advisory services span from strategic (defining testing policy, standards, testing organization structure, business operating model, etc.) to operational like reviewing their process maturity, metrics and reporting.


Ø  Client organizations are increasingly looking for a more proactive and advisory role in identifying the optimization opportunities, not only in testing phases, but also in the upstream and downstream phases. IT organizations are keen on leveraging QA advisory recommendations for internal change management, to transform their organization.

Ø  Demand for QA advisory services will extend beyond IT divisions to cover business units and functions as well. Business units will show more interest in organizing and building their Requirements Definition and UAT capabilities and deriving more efficiency through QA advisory services.

Ø  Thought leaders with a consulting mindset need to be identified and developed by QA organizations to play this advisory role.


QA Managers need to look beyond testing phase. Test Managers' ambit, skills and mindset must be broadened and sharpened to play this advisory role effectively.


In my next and the concluding part of this blog series, we will talk about the fifth parameter - OUTPUT- TO OUTCOME-BASED ENGAGEMENT MODELS AND KPIs.


See you soon.