Testing Services provides a platform for QA professionals to discuss and gain insights in to the business value delivered by testing, the best practices and processes that drive it and the emergence of new technologies that will shape the future of this profession.

« The Importance of Service Level Agreements (SLAs) in Software Testing | Main | Performance Testing for Online applications - The Cloud Advantage »

Enabling Effective Performance Testing for Mobile Applications

Mobile performance testing approach/strategy is closely similar to other performance testing approaches. We just need to break the approach in order to ensure all facets relevant to performance testing are noted and taken care off.


Understanding technical details on how mobile applications work
    

This is a primary step. Most mobile applications use a protocol (WAP, HTTP, SOAP, REST, IMPS or custom) to communicate with the server, using wireless devices. These calls get transmitted via various network devices (e.g., routers, gateways of wireless service provide or ISP) to reach the mobile application server.


Performance test tool selection

Once we know the nitty-gritties of how the mobile application works, we need to select or develop performance tools which mimic the mobile application client traffic and record it from the mobile client or simulator. There are several tools available in the marketplace to enable the same - HP's LoadRunner, CloudTest, iMobiLoad, etc.   Besides this, the mobile application provider will not have control over network delays; however it is still very important to understand how network devices and bandwidths would impact performance and the end user response time for the application in question. Shundra plugin with HP LoadRunner or Antie SAT4(A) , have features that mimic various network devices and bandwidths.

Selecting the right monitoring tool

 

Once we have zeroed in on the load generating tool, we now need monitoring tools to measure client and server performance.


We can use DynaTrace, SiteScope or any other APM (Application Performance Monitoring) tools to measure server side performance. These tools will capture and display, in real-time, the performance metrics such as response times, bandwidth usage, error rates, etc. If monitoring is in place on the infrastructure side, then we'll also be able to capture and display metrics such as CPU utilization, memory consumption, heap size and process counts on the same timeline as the performance metrics. These metrics will help us identify performance bottlenecks quickly which eliminates the  possible negative impact on the end user experience.
   

Performance of mobile app client is also critical due to resource limitations with respect to CPU capacity, memory utilization, device power/battery capacity, etc. If the constructed mobile application consumes a lot of CPU and memory, then it will take more time to load on devices. This would in turn significantly impact the speed and the user's ability to multitask on the same device. Also, if the application consumes a lot of power/battery, it would also reduce the user acceptance for such mobile applications.  For this, app plugins can be developed to measure and log mobile client performance as well. We can install plugins in mobile devices and encourage users to use it when loading is being simulated.  Possible tools that can be used are WindTunnel, TestQuest, Device Anywhere, etc. Plugins can capture performance data and the same can be sent to a central server for analysis.

 

In nutshell, with the right performance test strategy and tools in place, we can ensure effective Mobile application performance testing. This would ensure that the organization is able to deliver high performance and scalable apps to businesses which positively impacts the top line growth

Comments

Thanks for a concise summary of what is proving to be the next boom in internet - more than that,an explosion! Whether its "boom" as in "crash" or as in "boon" all depends on companies getting much more proactive in understanding that the meaning of "full scale" testing is no longer measured in the hundreds or thousands of simultaneous transactions or users..but often in the millions. Any solution chosen to test and measure at modern scale needs to be able to meet the demands of modern usage and at the pace of modern development.

Questions every solution must answer today include: Can we fully test in the dev lab and in live production?, Can we get instantaneous feedback about internal and external performance metrics, and can we measure the functionality and performance of real devices under load?

Brad, I completely agree with you that the load on mobile applications will no longer be measured in hundreds or thousands of simultaneous users, but in millions. But, the real challenge will be in simulating this full scale user load. Given traditional test environments, replicating such full scale loads will require huge investments in infrastructure to support the mobile applications and the process of generating this full scale load using tools. Thus a lot more effort and cost will be needed to help organizations set up and maintain this infrastructure, thus making it a less viable option to explore.

What we need is a smart performance test approach, and associated tools, that can help resonate the same level of confidence as full scale testing would do. We can plan to run performance testing in scaled down test environments and run what-if analysis using performance modeling tools which would simulate the various hardware, software and user loads. There are several performance modeling tools available in the marketplace today like InPres by Infosys and ePASA etc. Added to this, performance predictive tools can also help us in replicating the capacity needed to support millions of users or transactions. We can also monitor performance and analyze the workload and performance trends in real-time using APM tools like DynaTrace.

Post a comment

(If you haven't left a comment here before, you may need to be approved by the site owner before your comment will appear. Until then, it won't appear on the entry. Thanks for waiting.)

Please key in the two words you see in the box to validate your identity as an authentic user and reduce spam.