Test Tool Selection for Functional Automation - The missing dimension
"How do you select the appropriate test tool for your functional testing?"
"As per the listed features, the automation tool we invested in seems to be right. But, the % of automation we could really achieve is very low. What did we overlook at the time of tool selection?"
"Both my applications are of the same technology. We achieved great success in automation with one of them and failed miserably with the other. What could be the reason?"
You could keep adding no. of questions to the above list.
Do the above questions sound familiar? There is a lot of literature on the internet that helps compare different automation tools and decide the best one among them. Also, the specific 'Read Me' notes of the tools, list down the Operating Systems and the technologies which the tool is compatible with. We do take into consideration all the above mentioned inputs while arriving at the right tool for our application. Despite all this, where is that we go wrong? The answer lies in the dimension which we miss while selecting the tool.
What is this missing dimension? This is the detailed information about the specific application interfaces that we would be testing with the test tool. We check the compatibility of the 'Operating System' with the test tool (Ex : XXX tool works in Windows only or YYY tool works in Unix/Linux ), we check the compatibility of the 'Technology ' with which the application interfaces are built, with the test tool ( Ex: AAA tool Supports Active X controls or BBB tool supports Java), but, the detail which we fail to investigate is "Whether all the GUI objects of this particular technology are directly supported by the Test tool?"
Whether we achieve success in automation or not, largely depends on the test tool's compatibility with the objects in our application screens. Full support for all the screen objects by the test tool, directly translates into high automation script preparation productivity and greater ROI.
A test tool might provide support for the underlying technology of the application, but no test tool can guarantee that it provides 100% support for each and every GUI object of that technology. The reason being, apart from using the default GUI objects that are provided by the technology to build the application screens, the programmers might use their imagination to come out with their own objects (which can also be called as 'Custom objects') and build the application screens with them. These are the kind of objects for which no tool can provide the built-in functions.
Some tools, allow us to map the custom object to a standard object and proceed using the built-in functions of the standard object, for the same. Many-a-time, these mappings do not work properly for the tests to be automated. In these cases, sometimes, it is possible for the automation engineer to come out with a custom function to work with these kinds of objects. It may not be possible to work with those objects at all, neither through mapping nor through custom code. These are the times when we wonder, whether we have selected the right tool!
So, what's the best process to take care of this dimension while arriving at the best test tool for your applications? One is to do a 'Static Check'. Another method is to do a 'Dynamic Check'.
'Static Check' involves, gathering the information from the Development Team on the kind of GUI objects they used in the application. The success of this technique depends on the communication channel between the Development and Testing teams. This is more suitable for agile projects where both the teams work hand-in-hand. Cross check the tool documentation to see whether the tool supports these objects.
This requires the tool to be run systematically on all the application screens to check whether the tool is able to capture all the objects. This is more suitable for applications that are already developed and the automation addresses regression.
Use the information from both the checks wisely, along with the other parameters (like Cost, compatibility with the OS, Compatibility with the Technology, DB Support, Ease of Maintainability, Ease of Reporting etc.) to arrive at the final decision.
While, it is not possible to figure out all the challenges that we might come across with the GUI objects at the time of tool selection, a conscious analysis of the compatibility of various test tools with different type of objects would help us to understand objectively the effectiveness of the test tools. Make an informed choice and there will be no confusion regarding the tool selection.