Testing Services provides a platform for QA professionals to discuss and gain insights in to the business value delivered by testing, the best practices and processes that drive it and the emergence of new technologies that will shape the future of this profession.

« September 2013 | Main | November 2013 »

October 10, 2013

How different is providing Performance Strategy for BPM applications?

In the domain of Financial Services and Banking, Business Process Modeling (BPM) systems are often at the heart of the enterprise applications as these provide flexibility, lesser time-to-market and better regulatory compliance with local laws to name a few. BPM technology is typically used to implement a 'workflow based application' where different 'tasks' are to be performed by different 'user groups' or 'actors' based on the 'state' of the data. It can have actors as humans or the business processes that can integrate with other systems through Enterprise Service Bus (ESB) for a complete business process or business activity.


A typical BPM system, at a very abstract level, comprises a Process Engine, Process Modeler and a Content Management System (CMS) / Database. Process modeler is used to create business processes, process engine provides the runtime to execute the processes and CMS or DB will be used to store information specific to the Process Engine artifacts.

Now the question arises that how should one look at BPM systems for Application Performance Management (APM)? Is it any different from web-based or batch type applications? How different it is to provide performance strategy for BPM applications? What are the additional factors to be considered while capturing Non Functional Requirements (NFRs) and providing load simulation approaches? Well, here are my thoughts.

·While capturing NFRs, be focused to understand 'business process', 'sub-process', 'tasks' and 'actors' in the system - this basically sets the stage to have clear understanding of the business flows, atomic operations and end-users which makes the performance strategy easy to define.

While defining performance test execution strategy, following should be kept in mind for BPM applications.

·If Actor 'a' works with some set of screens or UI steps, all such screens/steps should be considered as a single business flow for Actor 'a' - similarly all different users/actors working on a set of screens should be considered as one business flow which should be mapped to one 'Test Script' created using any Load Simulation software.

·Secondly, if a single business process comprises two sub-processes, first sub-process acted upon by user 'a' followed by second sub-process  by user 'b', the strategy to load simulation can take either of the two following approaches:

ü Approach 1 à Create a single script with sub-process for user 'a' who will modify the state and submit to user 'b' - and user 'b' will act thereafter à This represents real world scenario where user 'b' does not have any pending items in his inbox and hence cannot act upon unless user 'a' submits something. Here these 2 actions will be sequential in nature.  

ü Approach 2 à Create separate LR scripts for activities done by user 'a' and 'b' and execute them concurrently à This also represents real world scenario where multiple items are pending with user 'b' and hence he does not necessarily act upon the recently submitted items by user 'a' - hence they act on respective actions independently.

In Approach 1, test data flow between the two sub-processes (from user 'a' to user 'b') is apparent, intuitive and easy to handle in load simulation scripts.

In Approach 2, data should be pre-created and kept ready so that steps for user 'a' and user 'b' can concurrently execute, as actions/tasks submitted by 'a' will not be available to user 'b' instantaneously, which is comparatively tedious and time consuming.

In either approaches, one should ensure that the concurrent load created on the underlying BPM system per unit of time should be the 'Peak Load' which can be controlled by adjusting the number of Vusers, iterations, pacing and think time in Load Simulation software.

·One should have clear understanding about the 'data flow' between the 'actors' so that data preparation activities are clearly highlighted in the proposals and right estimates are provided upfront.

·Also, BPM systems in specific, besides having database instances for application data persistence, have separate DB instance to persist configuration details of the Process Engine (specific to processes, sub-processes, tasks, user groups and any administrative details) - hence, ensure to highlight the strategy to monitor process engine DB instance as well.

·Like for online/batch systems, the 'concurrency' for BPM applications is nothing but a set of actions/tasks being generated by different 'user groups' or 'actors' performing respective tasks concurrently - the basis for concurrency should be 'transaction load' but not just 'user load'.

So laying down the performance testing strategy for BPM systems necessitates focusing on few additional aspects without which performance assessment for BPM systems will be ineffective.

Preparing ourselves for Application Security Testing


Haven't we all as functional testers, done 'Authentication' and 'Authorization' testing? Almost every application demands some flavor of these tests. So, are we not already doing 'Application Security Testing'?

Let's explore and see what's the extra mile, we need to traverse, in each phase of SDLC, to say confidently, that the applications we are testing are secured ones.


In this blog, I am going to refer ourselves as 'functional testers' in the context of the regular testing tasks that we carry out in a project and as 'security testers' for the additional tasks which we need to plan, to certify that our Applications under Test are secure.

Requirement analysis phase

As functional testers, our focus is to verify all the requirements for completeness, correctness, unambiguity of the requirements and also derive the test scenarios out of the requirements.

As security testers, we need to review the documented security objectives for the application (provided by client) against the security objectives that can be derived out of the requirements, for completeness, correctness and unambiguity.

To do the above, we must perform techniques like role matrix, CIA (confidentiality, integrity and availability) analysis on each requirement to identify the security objectives. We also need to be aware of the set of common questions against which we need to check the security objectives of the application for identifying the missing security objectives. Some of the questions to ask are: 'what are the tangible and intangible assets to be protected?', ' what are the compliance requirements?' etc.

Design phase

As functional testers, we verify the application design against the functional requirements.

As security testers, we need to verify the application design against the application vulnerabilities and the associated potential problems, in the context of bad design.

To do the above, we should be aware of typical set of potential problems that can arise out of bad design such as 'data tampering', 'cross site scripting', 'buffer overflow', 'SQL injection' and the corresponding good design practices.

Test phase

White-box testing:

As functional testers, we do not typically do the white box testing (code reviews / structured code analysis), as this is mostly carried out by the development teams themselves.

As security testers, our responsibilities might include the white box testing of the code also, but mostly using the automated tools.

To do the above, we should be aware of configuring and executing the tools for secure code analysis, prepare the bug report from the result report of the analysis and track the bugs to closure.

Black-box testing / Penetration testing:

While white box test tools act on the application code, black box test tools act on the executable application.

As functional testers, we do the black box testing either manually or using some automation tools such as QTP, RFT etc.

As security testers, we need to use the penetration testing tools such as IBM Appscan, HP WebInspect and HP QA Inspect etc. These tools are capable of identifying whether our AUT is compliant with respect to our Application security objectives.

To do the above, we should be aware of how to configure these tools to test the security objectives of our AUT. Additionally, we need to methodically identify the business scenarios within the application that can unearth the security breaches and run these tools to crawl and audit each of these flows. We should also be aware of analyzing the results of the scan, understand the recommendations given by the tools and guide the developers on how to update the application code to avoid the security flaws.

Deployment phase

As functional testers, our scope of work in this phase is nil.

As security testers, we need to ensure that the infrastructure on which the application is deployed is secure.

To do the above, we should be aware of the appropriate configuration settings/best practices, for both the network and the host. For example we should know how to check whether right patches are applied on the server, whether log files help in troubleshooting the security problems, validate restricted/blocked remote registry administration etc.


Unless and until we approach the application security testing in a focused manner, in each phase of SDLC, we cannot claim that our AUT is secure. Let's gear up on the additional skills that are needed, to upgrade ourselves as 'Application Security Testers' and help our clients create secure applications.

October 3, 2013

Crowd Testing

The concept of crowdsourcing is not new. The practice of harnessing ideas, services or content from a larger pool of unknown population has existed for many centuries.  For example Oxford English Dictionary got created through an open call to the community to identify all the words in English language along with their usage; this call yielded 6 million submissions over 70 years! The Indian Government effectively used crowd sourcing to obtain entries for the symbol of the India Rupee which finally led to selection of the current symbol. On a lighter note, in India we do see crowdsourcing all around us. A crowd of helpful volunteers trying to help fix or push-start a broken automobile is a common sight here!

Though harnessing the collective wisdom and services of crowd has been in common over for a very long time, the term crowdsourcing was coined only recently in 2005 to describe how businesses were using the internet to outsource work to individuals.


From a testing perspective too, the industry is not far behind. In the past few years we have seen companies being formed which specialize on crowd-testing.


So what is so special about crowd-testing?


Testing is all about assuring readiness of a system or a product to be deployed for end users.  Towards achieving this objective, crowd-testing helps in the following ways:

·         Often the testing function is challenged in its ability to get diverse users to try and break the system in order to unearth hidden defects. A rigid and process driven testing which relies on pre-developed test cases may not be enough to uncover defects and usage issues.  Of course, exploratory testing techniques can be used but this is still executed by a limited number of people. A more diverse set of testers can more effectively find hidden problems.

·         Consumerization of IT and BYOD has led to phenomenal increase of smartphones requiring systems to be tested in multiple devices and networks. Any investment here will be prohibitively expensive!

Crowd testing enables diversity in testing both in terms of users and end devices. It helps in true representation of the user community and thus helps in unearthing defects which otherwise might not have been found through the usual structured testing techniques.


It is to be noted that crowd testing can at best augment the mainstream outsourced testing services and not replace it.


Incentives for crowd sourcing participants can vary depending on the objectives and crowd selection.  While monetary incentives do play an important part here, it is not necessarily the only one. It has been proved time and again with interest groups that the main motivator for participation has always been professional pride, peer and community recognition rather than hard economic considerations. The success of open source communities is a testimony to this fact.


In Infosys too, we are experimenting with crowd testing with a leading bank in APAC region. We hope to tap the wisdom of our crowd testers to unearth usability and functional defects which otherwise would not have been found in regular testing.