The Importance of Service Level Agreements (SLAs) in Software Testing
It's pretty well understood in the software industry that Testing is a specialized area which helps organizations reduce risk and derive greater business value across the entire software development lifecycle. However many organizations continue to struggle with figuring out the best way to define service-level agreements (SLAs) and the outcomes that can govern testing relationships. Through my experience over the years I believe it's extremely important for customers to define SLAs upfront in order to ensure 100% alignment of goals between service provider and customer and to accelerate trust in relationships especially with first time partners.
Before we go on to define the SLAs, it's important to define the Key Result Areas (KRAs). These are broad level areas where the SLAs will be measured and they could be in areas like governance, process, resources/staff and transition. Once these are defined, we can define the SLAs within each KRA. It's Important to choose SLAs which are relevant to the engagement (managed service, co-sourced, staff augmentation) or the type of testing (functional/automation/performance/SOA etc.). A common mistake made while defining SLAs is not defining the criticality of a particular SLA. This is important because it's not necessary to have the same level of criticality for all SLAs. Some are more relevant than others and hence we can use a classification like critical, high, medium or low for the same. Once the level of criticality is assigned to the SLAs, we need to decide on how they would be measured. I have invariably seen that customers are unsure about the measurement of SLA's. However, deciding the tools and the methodology of calculation for measuring the SLAs is imperative. Finally, a decision on the frequency of the SLA capture (release wise, monthly or quarterly), when (release wise, monthly or quarterly) and how (spreadsheet, sharepoint, document) will it be decided upon.
In any new engagement where SLAs are defined for the first time, there will invariably be questions about the targets like how do we determine these targets? In such situations, it's always important to define a "Nursery Period". The purpose of this "Nursery Period" is to benchmark the targets for those SLAs where a period of demonstration is required before it can be set. At the end of this exercise, all SLAs should be specific, quantifiable and measurable.
The commercial framework for a risk and reward model is the key component of SLA definition process. Before delving into commercials, it's important to decide how the SLA scores would be computed. Each individual SLA should be measured and a weighted score determined based on the SLAs criticality weighting. The individual weighted scores should then be averaged. The final average weighted score is then used to calculate the commercials "Debits" or "Credits". To make this less complicated it may make sense to include only the critical and high SLAs in determining the score. Working out modalities (frequency, process of payments) of "Debits" or "Credits" is the last leg in definition of R&R model. My recommendation is to implement separate governance for risk and reward model to facilitate a collaborative and transparent relationship for this key aspect. The governance framework should have clearly defined procedures for issue resolution and escalation to allow both parties to efficiently work through issues inherent in a risk and reward agreement.
To continue the relevance of SLAs, it's important that it is reviewed regularly. I think it should be reviewed every quarter. The SLA's that do not serve their purpose need to be eliminated. "Raise the bar" for SLAs which are being consistently met for consecutive periods.
Lastly, it is extremely important to create awareness about the SLAs amongst internal and external stakeholders and project participants. This is necessary so that everybody understands the SLAs and its objectives. Communication of the SLAs is very critical and essential to the successful execution and completion of the testing assignment at hand.