Crowdsourced Testing: Leveraging the power of a crowd to do the testing
What is crowdsourcing?
The term 'crowdsourcing' was coined in 2006 when Jeff Howe wrote an article for Wired Magazine - "The rise of crowdsourcing." Crowdsourcing is a combination of two words - crowd and outsourcing. The objective is to get the work done from an open-ended crowd in the form of an open call.
It is the combined effort of a crowd working at different places through an online medium - leading to significant results. The most common example of crowdsourcing is Wikipedia - the most comprehensive information created by crowd. According to Jeff Howe - "The amount of knowledge and talent dispersed among the human race has always outstripped our capacity to harness it. Crowdsourcing corrects that - but in doing so it also unleashes the forces of creative destruction."
What is Crowdsourced Testing?
Testing has always been a painful area for development companies. This is primarily because of a huge gap between planning and reality for the development of a project - as shown in the figure below.
One possible way to solve the above situation is to increase the resources and address the scheduling issues - but this may result in huge costs. Another way is to outsource the work to other companies. This requires a fixed schedule, lengthy sales process, and sometimes you end up with the same challenges faced by the internal team.
However, both these solutions have their own pitfalls. In today's competitive world, organizations are aiming to deliver a good quality project with lower costs and delivery on time.
Therefore, the solution to the above problem is - crowdsourced testing. It is an emerged trend in software testing, which allows individuals to participate at reduced costs and with a better testing quality. Testers are paid for their testing skills and capabilities. The key advantage of crowdsourced testing is that it enables organizations to get their applications tested by skilled people who are closely similar to their end users.
Crowdsourced testing on an average delivers 300 percent more work in the same amount of time*. As a result, it is faster and addresses the time parameter. You have skilled testers testing your application across the globe and as rightly said by James Surowiecki - "Large group of people are smarter than an elite few, no matter how brilliant." Therefore, we can count on a high quality product. At the same time, it keeps your cost down, as you do not have to go through a lengthy sales process.
Stages of crowdsourced testing:
Stages of crowdsourced testing is not different from a software testing lifecycle. The only difference is that resources are taken from a crowd.
- Gather the requirements from a client and prepare a software requirement specification (SRS) document
- Prepare test strategy / test plan document, decide on testing activities, and the approach to be followed
- Requirements for the staff with the right skill sets are posted on the company portal
- Based on the responses received from a crowd, the hiring process begins
- Once hiring is done, details such as requirement specification document, application URL, environment details, and other testing details are provided
- Test cases are designed
- Crowd testers are assigned with different modules for test execution
- Tester executes system test cases and does a bit of exploratory testing
- A project lead - monitors the testing activity of crowd testers
- All the testers share the daily status report with the project lead
- The project lead also acts as a mediator between crowd resources, client, and clarifies all queries from crowd testers
- The crowd testers raise defects in a defect management tool
- A project lead validates the defects raised by the crowd testers
- Crowd testers are paid based on the number of valid defects raised
- Defect summary report is prepared.
- After a defect is fixed the crowd resource does the retesting
- Testing activities are closed once all the milestones are met
- Once the test cycle is closed - then all payments are settled
Some crowdsourcing examples:
Awardesigns.com is a Spanish crowdsourcing platform launched in 2011. More than 2000 designers and writers from over 15 countries (Spain and Latin America) help entrepreneurs, startups, and small business with logo design, Web design, and other types of graphic design.
uTest is a community of more than 20,000 testers who charge on - per bug basis. A client specifies to uTest the type of tests needed and the devices to be tested.
Pros and cons of crowd testing:
- The Costs are maintained:
- It is cost-effective - as you are paying only for the valid bugs reported
- It is cheaper than hiring engineers, designers, and specialists
2. Shorter time-to-market:
- It is faster as you have a large number of testers across the globe testing the software simultaneously
3. Multiple language testing - very easy to get it done:
- It helps you to test the application based on localization as there are testers across the globe with variations and skills in different languages and locales
- As there are more hands - the job will be completed faster within the stipulated time
1. Management issues:
- Managing testers across the globe and working in different time zones can be challenging
2. Compromise on security and confidentiality:
- Your application and the other environment details are exposed to strangers
- As the crowd has access to your software and application, any breach of customer data can lead to security damage
3. Communication issues:
- Communication issues may pop-up when an urgent response is needed from the people working online.
In today's competitive world, there is a lot of pressure on an organization to deliver good quality products in a timely manner. Crowdsourced testing approach is more cost-effective and involves skilled testers working from different places to ensure that all the testing areas are covered. It is faster as you have more hands to get the task done, while maintaining lower costs at the same time. This proven method is an also an effective way to get a multilingual testing done - as there are testers working across the globe.
*This information provided by Crowdtesters.com