Testing Services provides a platform for QA professionals to discuss and gain insights in to the business value delivered by testing, the best practices and processes that drive it and the emergence of new technologies that will shape the future of this profession.

« Pragmatic Performance Engineering | Main | Culture is everything in Performance Engineering »

Crowd Testing: A win-win for organizations and customers

Author: Manjunatha Gurulingaiah Kukkuru, Principal Research Analyst
            Swati Sucharita, Senior Project Manager

Software is evolving with every passing day to accommodate newer prospects of its usage in day-to-day life.

Some of the key factors for a new software to succeed in today's market place include:

  1. Faster time-to-market
  2. Collective responsibility of the software's quality, taken up by the development and quality assurance (QA) teams along with the product / application's owner
  3. A strong market survey conducted in order to understand competing products, imitate end user scenarios, and simulate end users in the test environment

Even in today's rapidly changing scenarios, traditional testing with dedicated testers continues to be integral in ensuring product quality. However, given the challenges posed by modern-customers, a traditional testing team is insufficient to ensure a quality product as the team operates within the confines of the organization's processes and has the same mindset as that of the developers or the product's creators. It is to address these challenges that crowdsourced testing is now being leveraged by large, independent software vendors as well as services companies.

Crowd testing brings diversity to testing techniques, works with low-cost testing devices, and ensures better test coverage across multiple geographic regions.

The role and impact of using a platform for crowd testing

Crowd testing's success depends on ensuring better test coverage, along with device and platform diversity. All these factors, combined, result in a high-quality products, faster time-to-market, short release cycles, the ability to execute in parallel with normal project delivery schedule, and an economical process with elastic resource utilization / on-demand testing.

While these success factors encourage us to look at crowd testing as an important and complementary part of our testing services, there are multiple challenges surrounding test coverage, motivation of the 'crowd', the quality of findings, and consistency throughout the testing phase.

In order to bring the crowd together and execute testing, while simultaneously mitigating key challenges; a platform that's easily accessible is absolutely essential. The platform should provide the ability to monitor test coverage and consistency, with different metrics, throughout the testing phase. Additionally, by providing continuous ratings for the contributions of the crowd, which is visible to peers in the test cycle, can create a sense of inclusion amongst the testers.

Infosys Crowd Testing platform and our success stories

The Infosys Crowd Testing platform is our in-house innovation that addresses all the challenges we've discussed so far. It provides features to define the scope of testing along with various details like problem statements, prioritized test scenarios / use cases, timelines, and contact information. The platform has discussion forums where the crowd can share their experiences. It also provides different matrices to track test coverage across the multiple devices used by the crowd. Additionally, the platform also provides standard matrices to monitor testing progress and review its findings. We have also integrated gamification into the platform, which publishes a leader board with 'reputation points' scored by testers and validators. This serves to keep the crowd motivated throughout the testing phase.

We have performed crowd tests for an external facing application by leveraging our bench resources, who were well-versed with Infosys testing methodologies, and an external crowd, who in just four days identified 30 defects / possibilities for improvements across the diverse array of devices that they used for testing. This created an opportunity for Infosys to participate in a global 'request for proposal' (RFP), in which we were selected as one of the two partners who would deliver crowd testing services. We have also successfully adopted this concept in the delivery practice for a leading telecom client, wherein we performed user interface and usability testing of the application by leveraging the Infosys talent as crowd and fixed all issues well before the application went into production.

Conclusion and call-to-action

In most cases, crowd testing is classified as working with freelancers or a testing organization whose business model involves pooling in a crowd of testers. However, although this is one form of crowd testing, it is typically called 'public' crowd testing. There are other creative ways of crowd testing that include leveraging the crowd right within an organization by:

  • Sourcing people with relevant skills, who otherwise might not be directly working on the same project. This is typically called 'private' crowd testing
  • Building a pool of end users with a mix of internal resources and external / private MVPs, who can continuously provide feedback in all phases of product development. This is called a 'hybrid model'

Many organization fail to reap the benefits of their internal, diverse talent pool due to their reluctance in accepting new and emerging trends, and their resistance to transformations. With crowd testing, however, organizations can build a whole pool of talents with diversity of wisdom, which can help sustain them through changing scenarios.


A very good read.

Post a comment

(If you haven't left a comment here before, you may need to be approved by the site owner before your comment will appear. Until then, it won't appear on the entry. Thanks for waiting.)

Please key in the two words you see in the box to validate your identity as an authentic user and reduce spam.

Subscribe to this blog's feed

Follow us on

Infosys on Twitter