Challenges during performance testing in a shared environment
Recently in one of our performance testing project, we were given a shared environment for conducting tests. During this exercise we faced multiple challenges while loading the system, measuring the outputs, successfully completing the tests and almost all the activities.
The following points highlight some of the challenges that we faced during performance testing of a web application in shared environment.
1. System performance metrics could not be measured in consistent and accurate manner due to varying external load on the system.
2. It was hard/ not possible to identify bottleneck conditions in most of the cases. Due to this, overall usefulness of the test results was coming down drastically.
E.g. highly varying system utilization was observed on DB server due to external load. Thus, it was not possible to identify the workload at which the threshold of 40% utilization of the server was being reached, due to application transactions.
3. In some cases as part of testing activities, server instances were required to be restarted. This was not possible at times as multiple applications would be deployed on the same server instance. Restarting would affect multiple activities in the environment.
4. Things are bound to go wrong while performance testing at higher user loads. There will be high probability of environment getting clogged/ failed. As the same environment was being used for other activities, performance tests were affecting these planned activities.
5. Validations of the test results were failing big time and re-execution of the tests were required quite often.
6. There might have been a major impact on effort and timelines.
My subsequent blog will provide details on the method we formulated for overcoming these challenges, while performance testing in the shared environment. (To be continued...)