Testing Services provides a platform for QA professionals to discuss and gain insights in to the business value delivered by testing, the best practices and processes that drive it and the emergence of new technologies that will shape the future of this profession.

« November 2012 | Main | January 2013 »

December 31, 2012

6 Steps to Test Games Effectively

Have you ever wondered how games are tested and delivered? Effective game testing strategy leads to a hassle free and responsive gaming vigor to the end users. Gaming industry is growing exponentially due to increase in apps usage in mobile and tablets. It is hard to define a standard strategy for game testing and testing methodologies need to be tailored as each game differs from the other. To start with, aspects like Functionality, Usability, Multi-player functionality, Regression, Endurance, Compatibility, Localization / Internationalization (L10N, I18N), Performance, Content Testing, Hardware and Recovery Scenarios have to be analyzed. Phases of game testing almost align with the standard Software Testing Life Cycle (STLC) and based on my experience, below is the approach to test games effectively.

1.       It starts with requirements elicitation. A thorough understanding of Storyboard, Game Architecture, Game Features, Characters, Concept, Rules, Game points and Game stages/levels helps testers to create an effective game testing strategy.

2.       Game test strategy preparation is the next phase. The Test Strategy document must include sections like Timeline, Testers, Number of Testing Cycles, In-scope, Out-scope, Types of Testing to be conducted, Risk based Test analysis, Service Level Agreements, Risks and Mitigation, Defect Logging Process and Reporting Process. For a typical action game,

·         In-scope tests could be Functionality, Performance, Compatibility, Compliance etc.

·         Validation of hardware components such as Joy stick, Console, Wi-Fi devices etc. as they are outside the game testing environment.

3.       In third phase, testers should design test cases. Testers should create positive as well as negative test cases.  Following are some effective test case design techniques:

·         Critical Path Test is a technique to identify the critical paths in the game. Critical Path testing helps to reveal the dependencies which causes the failures.

·         In contrast to Happy Path testing, Exception Path Testing helps to document alternate paths and exceptions path.

·         Also, boundary value analysis, equivalence partitioning, and error guessing are the traditional techniques of black-box testing.

4.       Executing game test cases in the identified environment with different software and hardware configuration reveals more defects.

·         Alpha, Beta Testing and respective age group testing helps to improve the quality of the game further.

·         Content Testing plays a significant role in game testing, often partially ignored in many projects. It validates the following aspects of the game:

o Is the content appropriate for the gamers?

o How easy it is to understand the content?

5.       Test results should be recorded as a movie or as a series of screenshots. This helps the development team to review the behavior effectively.

6.       All the defects should be logged, prioritized, categorized and tracked effectively. Defect Logs help developers for the next title work.

Once the game is created, it is delivered to entertainment companies like Sony, Electronic Arts, and Microsoft for their feedback and production. And finally on the approval of respective Censor boards of the game, it is ready to be distributed commercially.

I would be more than happy to know your view points as well towards effective testing of games.

Happy Gaming & Testing!


December 26, 2012

8 challenges one must know before testing SaaS applications

Software as a Service (SaaS) is a software delivery model in which the software and the associated data are hosted centrally and made available to the end users over a network, typically the Internet. Due to its exclusive model, testing of a SaaS implementation from the client's point of view possesses unique challenges. Following are the 8 unique challenges one must know before testing a SaaS application:

  • White box testing is not possible since the SaaS service Provider would only provide access to their test environment and not to their code base.
  • Access to the database will be restricted and hence verification of test results needs to be done only from front end.
  • Testing team has no control over the scheduling of jobs and needs support from the vendor.
  • Testing team will not have access to communicate with the Technical Team of the vendor which might lead to ineffective communication.
  • It is always a tussle between the testing and development teams in seeing eye-to-eye for what a defect is and what is not.
  • Another potential source of miscommunication could be in pointing out the difference between what a configuration defect is and what a code defect is.
  • Since SaaS applications come bundled with several features, it is imperative that testing team is aware of defining the scope to avoid spending effort and time on testing unnecessary functionalities.
  • The last challenge stems from the fact that SaaS providers keep making major updates to their existing clients on a periodic basis. Testing teams in general are not accustomed to see the complete overhaul of the User Interface or new functionalities in the middle of a testing cycle. However while testing SaaS Application; the team needs to be well aware that such possibilities exist.

In a nutshell, if the testing team acknowledges the existence of unique challenges specific to testing SaaS Implementation and plans accordingly, a lot of surprises would be avoided at later stages. "Failing to Plan is planning to fail" - Hence plan to address these challenges before testing begins and a smooth roll out would surely follow.

December 12, 2012

What is a Test Factory Model and how can we gain the maximum value from it for SAP applications?

Let me start with why do we need a test factory. With the ever-growing need for resources and rising costs in the running of enterprise applications (such as SAP and Oracle applications), testing of enterprise applications has become fairly complex and is a major contributor to burgeoning IT costs. Test Factory is a unique concept and model that allows us to address this problem.


If an organization plans to implement the Test Factory model, it can turn to vendors who have the expertise to test enterprise applications in a more efficiently-governed and cost-effective manner. At the same time, this model allows the day-to-day operations of the organization to be run effectively, peacefully and uninterrupted.


A recent POV - co-authored by practitioners from Diageo and Infosys - helps you to understand the various challenges faced by SAP-enabled organizations, and provides an in-depth look at the process of setting up a Test Factory as well as the benefits thereof.  The POV can be accessed at http://www.infosys.com/IT-services/independent-validation-testing-services/white-papers/Documents/test-factory-setup.pdf