Unique Challenges in Managing Test Automation Projects
There are many similarities between test automation and manual testing projects from a management perspective. However, there are some fundamental differences which make managing test automation projects very challenging and add to a project manager's existing responsibilities. Some of the most important challenges are:
1. The stakeholders' consensus on the scope for Test Automation
2. Agreement on the goals for Test Automation
3. Agreement on the manual effort required in the Test execution phase
4. The need to constantly update the stakeholders about changing technical risks & their subsequent impact on the final scope
I will explore each challenge listed above in detail, making the assumption that the underlying development methodology of the AUT (Application under Test) is waterfall SDLC, which is non-agile in nature. I'll also explain why and how these challenges are either not applicable or easy to handle in a manual testing scenario.
Stakeholders' consensus on the scope for Test Automation
A business could demand the complete automation of an application undergoing testing, but there could be a test automation expert who is too cautious to commit on any percentage of automation without conducting a thorough technical feasibility study. It's the responsibility of the QA manager to analyze the study and use it to come up with a feasible solution, to finalize the scope for test automation and articulate it to all the involved stakeholders for a consensus.
Whereas in a manual testing project, the QA manager would not face this challenge of defining the test automation scope because the testing scope depends on the scope of release and there aren't any technology specific constraints to deal with.
Agreement on the Goals for Test Automation
All test automation projects are not alike, so test preparation and execution productivity figures might significantly vary from the industry norms/organization baselines. The QA manager needs to ensure that all stakeholders understand the test preparation & execution productivity for the AUT might differ from their expectations, if any.
In a manual testing project, the QA manager does not face this challenge as the test preparation and execution productivity does not have a high dependence on the technology of the AUTs and minor variations are manageable.
Agreement on the Manual effort involved in the Test execution phase
No amount of robustness of the test automation suite can take away the tester's involvement in test execution, completely. The QA organizations need to understand the role played by the machine and a tester during test execution and mutually agree on the manual effort required during the test execution phase.
It is the responsibility of the test manager to clearly distinguish the manual effort (needed for the activities like test data setup, data table updates etc.) and the machine execution time and ensure that the QA organization understands the estimated effort required during the test execution phase and eventually agree to the same.
Once again, this challenge is not applicable in a manual testing project as test execution effort is an integral part of the overall effort estimated.
The Need to Constantly update the Stakeholders about the changing technical risks & their subsequent impact on the final scope
Due to some unforeseen technical incompatibility issues between the application under test and the testing tools, a situation might arise where the estimated percentage of test automation is not met by the project. Instead of waiting till the end of the project, the QA manager can proactively flag such technical risks earlier; keeping the stakeholders informed on the possible decrease in the test automation scope and the need to fall back on manual testing for test cases that could not be automated. The most difficult part of this exercise is to clearly articulate the complications that arise because of technical constraints so that all stakeholders understand and agree with the same.
In contrast, the change in scope for manual testing projects is very straightforward and easy to communicate.
To conclude, QA managers in test automation projects need to be technically more sound than their counter parts in manual testing projects, with excellent persuasion and communication skills, to enable the successful implementations in test automation.