The three ingredients for a perfect test strategy
Authors: Gayathri V- Group Project Manager; Gaurav Gupta- Senior Project Manager
Last week, we won a testing proposal for a mandatory program that cuts across multiple applications. Although we, the 'pursuit team,' were celebrating the win, a voice in our head incessantly kept saying, "Winning is just one part of the game; but delivering such huge programs on time is always a tall order!"
Over the past years, we have delivered multiple, large testing engagements for various clients. Some of these projects were as large as setting up an entire new bank, while some involved introducing new schemes or regulations. The impact of these programs varied from 'medium' to 'complex,' in terms of applications, rigidity of timelines, impacted lines of businesses (LOBs), and so on. Nevertheless, the challenges were always very similar.
During such projects, there are inevitable environmental challenges that our teams struggle to pull through. Even after they solved these initial hiccups and data set-up issues, they are constantly bombarded with conflicting stakeholders and changes in requirements. However, in spite of such issues, we always pull through, successfully!
There are common underlying levers for the success of any of these programs. As a first step in the testing process, the potential challenges and the strategies to mitigate them need to be diligently thought-out. To ensure a successful implementation every time, our team possesses ample experience in handling three major aspects of any test strategy:
Recent surveys have shown that up to 40%-50% of a tester's time can be consumed by test environment issues, thereby affecting quality and productivity.
In major testing engagements, selecting the testing environment is one of the most crucial decisions, as the entire development and testing process relies on it. Here, we can select:
1. A brand new environment ---- do we need an environment that requires multiple platforms to co-exist or whether we can work in silos
2. An existing environment ---- - is it a shared environment or an independent one
Both these options come with their own set of challenges. For instance, a brand new environment will require a huge investment in terms of infrastructure, time, and budgets; while on the other hand, an existing environment will need an agreement with ongoing projects in order to share the resources efficiently without causing any obstructions.
In some testing engagements delivered for major banking clients, we could not go with a new environment but had to work with the existing one.
These engagements have posed various challenges, wherein we had to share the test environment with multiple programs and had to alter the test data for our needs. We even timeboxed the event, made changes to the test data with prior approval, carried out our testing, reverted the environment to the previous state, and then handed it over to the other program team.
A Strategic approach adopted to mitigate the challenge of a Shared test environment depicted below:
Environment strategy depends on the selected option and has to cover the planning, strategizing, and timeboxing of events as part of the logical day plans. To this effect, we have observed that conducting workshops between all participating platforms in a lifecycle is a very effective mechanism to arrive at a workable strategy.
In the past, we have faced a few scenarios where we had to provision test data from scratch or had to upgrade the test data in the existing test environments. There have also been challenges where we had to share the test data amongst multiple programs and teams. The selected test data strategy has to consider all such aspects of the program and needs to call out the test data mining, timeboxing, dependencies, and ownership details.
The ever-fluidic nature of these programs, coupled with changing business requirements in the form of change requests (CRs), only serve to add more complexity. CRs can break the schedule and hence, implementing them requires a thorough impact analysis that cuts across platforms. Workshops have been quite fruitful in achieving this, wherein we discuss whether the CR and its need to implement in the environment is a 'must have,' or 'good to have' . All impacted stakeholders need to agree upon the exact testing cycle in which the change must go. Additionally, it is crucial to document the exact steps to handle CRs, in the strategy.
The phrase, "Man proposes, but God disposes," is applicable to the pragmatic world of large engagements too. No matter how much we strategize and plan, unforeseen issues can always materialize during the course of the program. It is here that we require a strong management presence; one that can provide course corrections swiftly and align the program with the defined strategy.
To summarize - employing a program-level strategy coupled with a strong management focus, will do wonders and ensure excellent results in large testing engagements.