Testing Services provides a platform for QA professionals to discuss and gain insights in to the business value delivered by testing, the best practices and processes that drive it and the emergence of new technologies that will shape the future of this profession.

« Testing-as-a-Service (TaaS): Take a Peek | Main | Preparing ourselves for Application Security Testing »

Crowd Testing

The concept of crowdsourcing is not new. The practice of harnessing ideas, services or content from a larger pool of unknown population has existed for many centuries.  For example Oxford English Dictionary got created through an open call to the community to identify all the words in English language along with their usage; this call yielded 6 million submissions over 70 years! The Indian Government effectively used crowd sourcing to obtain entries for the symbol of the India Rupee which finally led to selection of the current symbol. On a lighter note, in India we do see crowdsourcing all around us. A crowd of helpful volunteers trying to help fix or push-start a broken automobile is a common sight here!

Though harnessing the collective wisdom and services of crowd has been in common over for a very long time, the term crowdsourcing was coined only recently in 2005 to describe how businesses were using the internet to outsource work to individuals.


From a testing perspective too, the industry is not far behind. In the past few years we have seen companies being formed which specialize on crowd-testing.


So what is so special about crowd-testing?


Testing is all about assuring readiness of a system or a product to be deployed for end users.  Towards achieving this objective, crowd-testing helps in the following ways:

·         Often the testing function is challenged in its ability to get diverse users to try and break the system in order to unearth hidden defects. A rigid and process driven testing which relies on pre-developed test cases may not be enough to uncover defects and usage issues.  Of course, exploratory testing techniques can be used but this is still executed by a limited number of people. A more diverse set of testers can more effectively find hidden problems.

·         Consumerization of IT and BYOD has led to phenomenal increase of smartphones requiring systems to be tested in multiple devices and networks. Any investment here will be prohibitively expensive!

Crowd testing enables diversity in testing both in terms of users and end devices. It helps in true representation of the user community and thus helps in unearthing defects which otherwise might not have been found through the usual structured testing techniques.


It is to be noted that crowd testing can at best augment the mainstream outsourced testing services and not replace it.


Incentives for crowd sourcing participants can vary depending on the objectives and crowd selection.  While monetary incentives do play an important part here, it is not necessarily the only one. It has been proved time and again with interest groups that the main motivator for participation has always been professional pride, peer and community recognition rather than hard economic considerations. The success of open source communities is a testimony to this fact.


In Infosys too, we are experimenting with crowd testing with a leading bank in APAC region. We hope to tap the wisdom of our crowd testers to unearth usability and functional defects which otherwise would not have been found in regular testing.


Hi Everybody,

Cool Article & Nice.

To Me :: Traditionally the desired results of "Crowd Testing ( As per the terminology described here")are derived from traditional testings like " Beta Testing (Potential Customer,End User Testing ),Acceptance Testing( Customers in General) are already in practice.

When the concern is of exponentially various device platforms - then that could very well be addressed by adapting to "Cloud Technology" - Which ensures successful utilization of any application irrespective of the platform.

So,in conclusion,I will say - To address exponential device platform concern - Mount the application in Cloud . 2. To test the effectiveness of any developed application at exponential user count - Advocate "Acceptance Test" that is developed application could be mounted on Cloud Environment,so that testers across globe can have their share of structuring the Application towards perfection( If that's the desired Intention ).

So,device compatibility test for an application not required,if "Cloud Technology" is administered but In the process of application development,"Beta Testing & Acceptance Testing" can have its go,be it Cloud environment or by traditional means.

Crowd sourced testing is a relatively new software testing innovation that enables the developers to have their software tested across multiple unique devices. Crowdsourced testing is an emerging trend in software testing which exploits the benefits, effectiveness and efficiency of crowdsourcing and the cloud platform. It brings together thousands of professional testers from across the globe to test software on multiple devices, browsers and operating systems. There are several other crowd testing service providers available in the market, I have come across websites which allows its users to select their crowd among the community of testers and tests the app in a real world environment.

Thanks for the article.
Cheers Jack!

Post a comment

(If you haven't left a comment here before, you may need to be approved by the site owner before your comment will appear. Until then, it won't appear on the entry. Thanks for waiting.)

Please key in the two words you see in the box to validate your identity as an authentic user and reduce spam.

Subscribe to this blog's feed

Follow us on

Infosys on Twitter