Testing Services provides a platform for QA professionals to discuss and gain insights in to the business value delivered by testing, the best practices and processes that drive it and the emergence of new technologies that will shape the future of this profession.

« Crowd Testing: A win-win for organizations and customers | Main | Agile and testing: What banks need to know »

Culture is everything in Performance Engineering

Author: Sanjeeb Kumar Jena, Test Engineer

We are living in a bulging knowledge economy where anyone can access information from anywhere with a device that fits into their pocket. We are living in a hyper-connected world via the World Wide Web. Today, business is not the 19th century one-way 'producer-consumer' relationship. Instead, it's a two-way communication. An effective business model is 'not about finding customers for your products' but it's about 'making products for your customers'.

How does this model fit into performance engineering? And how do performance engineers provide insights or values and grow businesses accordingly?
Isaac Asimov once said, "The saddest aspect of life right now is that science gathers knowledge faster than society gathers wisdom." It holds true in performance testing (PT) as well.

As an independent PT quality assurance (QA) team, we plan and execute load tests and report the results to developers. That works well if the changes in the application stack are not frequent and applications are not continuously evolving (frequency of changes in codebase) as per end user interaction with the application.

With the introduction of low-cost mobile devices and cheaper bandwidth, another web penetration of users (around two billion) is expected by 2020. With more users interacting with the applications (web and mobile) on these devices, it's not only about scalability, but also about generating insights on application behavior at scale. That's the demand for the 21st century economy and challenge (or opportunity) for performance engineering.

Building a product is the business need, but building the right culture in the team is the human need that drives business in this fast-changing world.

How do we empower our team to create such a sustainable and learning culture?

Let's borrow an idea from human psychology and a performance engineering point of view.
To quote the iconic inventor, Steve Jobs, "The things you call life around you are created by people no smarter than you, once you realize that, everything will not be the same again."

In other words, people are the critical factor in any domain of engineering. In an organization, typically 'extrinsic motivators' (bonus, rewards, recognition, etc.) are used to drive innovation and build a similar culture. However, things are changing. With all kinds of information just a click away, people in any organization are considering 'intrinsic motivators' as well. These include:

  1. Mastery
  2. Autonomy
  3. Purpose

1. Mastery
Mastery is not only about becoming an expert on the bits and atoms of a tool or technology, but understanding how using those tools can help you learn about the domain and its users. Nowadays, Design Thinking is involved in every process of development. To me, it's not new. It's just another name to solve a problem in a quick and effective way with empathy towards the real user.

When we indulge in Performance Engineering, we get requirements, business transaction lists, and access to tools. After that, we test and report the performance analysis results from tools like log analyzers (Splunk or ELK), application performance management (APM) tools (AppDynamics, Dynatrace), live test monitor (Graphite, and Grafana with Jmeter for live test monitoring).

What if, in addition to performance analysis, we can get information on:

  • How our work impacts the code deployment process?
  • How the application is being used to drive business?
  • How important is the application for the entire application stack?
  • How does its performance impact other applications?

This knowledge on business domain brings mastery in understanding the impacting of performance engineering practices in overall business operation and with better quality information on both business process and consumer behavior, better decisions on performance can be taken and applied.

2. Autonomy
It results from the fact that human needs compound every day. Nobody wants tomorrow as it is today. They want better every day. This is where automation comes into play in our work culture. It's a basic human need to thrive for better and give a modular architecture to repetitive actions.

We usually define our performance engineering practices as combination of defined steps in ALM (Application Life-cycle Management). In Agile model, we define steps like stakeholder interaction, design and execution of tests and reporting in iterative fashion. But in today's consumer or people driven economy-- change is the only constant. With new people joining the new economy, business model needs constantly evolving and the only key to success is to adaptability to changes ensuring better performance. 

Every day while working on the same things, anyone can come up with an idea to improve something. As an innovative culture, we need to capture those ideas and have a platform to quickly try them out to see if they work. Innovation is not magic and it doesn't happen in any exact moment, but ideas for innovation can come to us at any time and from anywhere. The question is "Do we have a platform or medium to capture those ideas, and most importantly, try them out on dummy applications to see its full potential?"

The concept of DevOps comes from this thought. DevOps is not just about collaboration between development and operation management. It involves every stakeholder responsible for running the application as well continuous improvement with learning patterns from user experience.
DevOps in Performance Engineering (PerfOps) creates a platform with an automatic provision for regular activities so that engineers can take full responsibility to gather wisdom from collected performance knowledge.

3. Purpose
Purpose-driven work can improve results in a factor of hundreds over task-driven work (without a defined end-goal) What if we can integrate them both? Purpose in performance engineering can be divided into two categories:

  • General purpose: This includes individual career perspective and the impact of business growth on the client. PE is not limited to web apps or data warehouse applications. It's growing rapidly in areas ranging from wearable technologies to the Physical Internet around every physical object. Performance engineering as a discipline is not limited to what we do today. Its potential lies in the near future where everything is connected and with such scale and speed that it will penetrate every field  

  • Daily purpose: How do we share our best practices across teams so that we do our best every day? There's a saying in Silicon Valley - 'Fail Fast, Fail Often' - to grow faster. But in fact, it's not about failing, it's about learning faster. At the end of the day, performance testing is about identifying bottlenecks and improving your system's performance. If your load testing tool lets you check the scalability of your system in terms number of synthetic users but does not help you much with analyzing results, quickly identifying performance bottlenecks, and pinpointing the location of the problem, then perhaps you're not fully achieving the goal of performance testing. You might then consider integrating with an Application Performance Management (APM) tool for better visibility up to code level and a log analyzer for live visualization of your system while the test is running.

What if we don't have these tools in our project?

Most of these commercial tools provides free-trial versions for a limited period and many popular performance profilers are available as open source.

So the real question is: How do you create a real demo using these tools and show (not only tell) clients the benefits of adapting this trend?

We can leverage cloud computing to test our assumptions rapidly because rapid prototyping needs the absence of development environment complexities such as resource (servers, third party libraries, etc.).  Cloud solutions such as AWS and Azure on the Public Internet can be useful to create a rapid prototyping test platform to try out available solutions in PE and then integrate it into the existing toolchain, if it meets both technical and business criteria.

In the next blog, we will discuss about building a PerfOps framework that can be a part of pragmatic performance engineering culture.

The following blog elucidates further on this framework:
Performance engineering in Agile landscape and DevOps


Nice write up sanjeeb.

Performance Engineering with Purpose(Business Goal) as target- we need to change our PE goal from responsetime, resource usages to how atcually it impacts business

Post a comment

(If you haven't left a comment here before, you may need to be approved by the site owner before your comment will appear. Until then, it won't appear on the entry. Thanks for waiting.)

Please key in the two words you see in the box to validate your identity as an authentic user and reduce spam.

Subscribe to this blog's feed

Follow us on

Infosys on Twitter