SOA Testing and its benefits - Do we really reap them?
SOA being an abstract concept, understanding its core principles is essential to succeed. Stories of SOA implementation failures have been doing rounds primarily due to overlooking of its fundamental principles of business involvement and governance. The case with SOA testing is no different. The term is even more ambiguous, and any misinterpretations can result in failures of test implementations. The foundation of SOA testing lies on three basic beliefs - early testing, optimized coverage and faster regression. When it violates one or more of these, it is more likely to fail. From my experience, here are some of the typical situations where SOA testing does not give its intended benefits:
1. Late start - Services are instituted in the foundation layers of the architecture, with core business functionality. Detection of defects in such functionality prior to integration helps avoid cost escalations due to rework. As you get later into the life cycle for testing these services, this benefit is lost.
2. Testing of non-core services - Certain services are there just for the sake of it. They are neither reusable nor have any core business functionality. Such services can be better covered as part of system integration/end to end testing. Otherwise, it just becomes an overhead.
3. Poor choice of SOA testing tool - Selection of a SOA testing tool has to take into account several parameters like its support for message formats, transport protocols, automation, virtualization and commercial elements. For instance, a tool that does not support the required protocols or one that cannot adapt to WSDL changes can hit the ROI drastically. You can read my article http://www.infosys.com/IT-services/independent-validation-testing-services/Pages/maximize-ROI-through-SOA.aspx on the important parameters for SOA testing tool selection.
4. Ineffective automation - Automation of frequently changing enterprise services is essential to improve the efficiency of regression testing. However, just because you have a tool does not mean that you have to automate everything. Automating the wrong ones, almost static and too application specific web services for instance, can lead to wastage of effort and late completion of testing.
5. Redundant testing - SOA testing is a testing methodology which helps in improving coverage of SOA based systems. At the time of test strategy, it has to be planned on how the system requirements will be covered at various phases such as services testing, system integration testing and end to end testing. However, most projects treat SOA testing as a different track (due to different technical skill set needs of the team) which leads to lack of collaboration and hence redundant testing (the same test cases covered in multiple phases).