Testing Services provides a platform for QA professionals to discuss and gain insights in to the business value delivered by testing, the best practices and processes that drive it and the emergence of new technologies that will shape the future of this profession.

« August 2013 | Main | October 2013 »

September 30, 2013

Testing-as-a-Service (TaaS): Take a Peek


In my last post, I wrote how market if adapting to TaaS. Here I am talking "What is TaaS?" and "Why do we need to evolve?".

What is TaaS?

Considering all prevailing aspects of New Age QA, major QA service suppliers are now formalizing their own TaaS model for test outsourcing. Fundamentally, TaaS doesn't necessarily represent a true cloud service, but it incorporates aspects of cloud. This paradigm shift in thinking has made every QA service provider build and continuously mature their TaaS model. TaaS requires us to introduce testing infrastructure, test tools, accelerators, human effort in delivering testing services, delivery methodology, framework and best practices bundled together and apply it uniformly.


TaaS enables predictive delivery quality and cost. It is a process driven approach independent of people. TaaS is the holistic amalgamation of the following elements:


1.       Test methodology

2.       Testing infrastructure for tools and applications

3.       Testing tools, accelerators and framework

4.       Testing team

5.       Application specific approach, best practices and accelerator pack


Why TaaS?

Where at one hand customer is looking at business outcome based testing delivery model to reduce the total cost of ownership, on the other hand he wants to ensure that there is no quality issue related to functional or non-functional requirements and application is adhering to regulations such as SOX, HIPAA, PCI. There is increased focus on software quality to enhance customer experience.


Some of the key drivers for TaaS are:


ü  Predictable and on-demand service, accessible, available without compromising on quality and cost

ü  Flexible and scalable delivery model (Pay-as-you-go)

ü  Outcome based sourcing and pricing

ü  Enabling better estimation and budgeting

ü  Eliminating need for capital expenditure on infrastructure, software and human resources

ü  Improve overall QA processes and delivery framework

ü  Holistic automation

ü  Risk-minimization

ü  Volume discount


I will write on testing alliance collaboration strategy in my next blog.


September 27, 2013

Boons of Cloud Testing--Agility and Automation

In my previous blog I have discussed how traditional way of testing makes business less agile and less adaptable to automation. Here I will explain how Cloud Testing is delivering manifold advantages, creating a testing model that is more agile and is full of automation and orchestration.


Improvement in business agility

In a typical IT organization that practices legacy testing methods, there exists a gap between IT and the business. IT finds it difficult to cope up with all the requests it gets from various internal teams and struggles in prioritizing jobs. On the other hand, business has to bear the brunt of the delay as it causes losing out on precious time and money. With the advent of Cloud based Testing, these problems are eliminated with the following advantages related to agility:

·         Instant Provisioning

With rapid provisioning, business can access the cloud based test services instantly eliminating delay. This reduction in wait-time can directly influence the business growth in terms of revenue and volume.

·         Self-Servicing

With this feature, the test users can directly create their own test environment without depending on IT team. This will help them start their activity at an early stage of testing cycle. The tester will get enough time to comply with near 100% of test coverage. With the reduction in release cycle and with the rapid delivery and full test coverage the business can make huge profits and can improve on quality as well.

·         Flexibility

Cloud testing has the ability to make the test environment a close replica of the production environment. The load generated to test the Application performance can equal the actual scenario removing any geographical constraint. The test infrastructure can be scaled up or down rapidly without any constraint on demand. Any changes in the test plan or test infrastructure can be incorporated instantly making it a highly flexible framework.


Impact of Automation

One of the advantages of Cloud based testing is rapid provisioning of test infrastructure which can be attributed to automation. Unlike the legacy testing method of preparing the test infrastructure right from the scratch for every user, Cloud does it in a smart way. The image of the common test platform is taken, also known as Template, and is kept in a common repository called Service Catalogue. Whenever the requirement of a new test environment arises, a short script will start provisioning the server automatically and it would be ready within minutes. This process can be followed multiple times without human intervention saving considerable time and manual effort.


Also, any complicated test scenario, a particular build or a hotfix release, can be frozen as a Template for future bug fixing after a release. It is not required to prepare the environment and the customer scenario can be replicated in-house within minutes. This is unlike the legacy testing scenario which can take several hours to days to create an exact replica of the customer environment. After every release, the test environment can be returned and gets deleted immediately without maintaining any lengthy process of re-imaging it. This makes the IT operations competitive and up to date.


Agility and Automation are the reasons why Cloud Testing has gained such a momentum in the industry generating revenues for the business and facilitating end-user empowerment. Stay tuned for my next blog in which I will discuss the advantages of Cloud testing in detail.

September 23, 2013

Testing-as-a-Service (TaaS) - Changing Times

In the recent third party survey of Infosys clients, one message is coming clear that customers are moving away from traditional staff augmentation model towards a single custodian for their entire testing needs. There is increased focus on making testing services predictable, accessible, available without compromising on quality and cost. Customer is looking out for an outcome based sourcing and pricing while outsourcing testing services. We can see how New Age QA is evolving from testing phase to Business Assurance. Customer loyalty has become very important and one can't compromise with the quality of an application and hence in application delivery, software testing remains an essential element of business operational efficiency and risk management.

One key reason triggering this change and mind set, is current challenging economic environment across the globe. Customer wants to account for every single penny they spend. There is increased focus on defining business transactions based on business outcomes within the defined quality limits. Key ask from vendor is to account for associated risks while partnering as a QA partner. There is a need to innovate newer delivery models continuously, increase productivity and cut down the cost of operation. Customers are looking for holistic optimization including human effort, infrastructure cost, tools and technology spend.

Prevailing Thoughts on TaaS

The buzz word 'Testing-as-a-Service" (TaaS) has undoubtedly captured the mindset as a concept and paving the way for newer testing models. Any on-demand service has an extension of 'as-a-service' and TaaS is no exception. New cloud based delivery models are gaining traction in an effort to optimize testing costs and increase agility. All major testing tools vendors are also moving towards cloud based offerings enabling testing service providers to evolve newer TaaS models. For instance, HP SaaS has wide adoption now across the globe. Performance testing is now widely used in SaaS model where customer can procure and consume services online. Service providers are coming up with outcome based model for some of the specialized testing offerings such as automated regression testing, performance testing, security testing, product testing and cloud-based applications testing.

Although TaaS encompasses all the aspects of testing services, each implementation has a key driving objective and matures from there. Some of the key driving objectives we see in the market are:


1.       Efficiency: Focus on low cost, centralized testing, tools optimization and overall testing effectiveness.

2.       Transformation: Focus on the testing of major internal transformational initiative(s).

3.       Revenue: Focus on key business outcomes to launch new business services quickly through effective and fast testing.

4.       Quality: Focus on quality of application where it will be used and implemented by thousands or millions of end-users.


In my next blog, I will be revealing more about what is TaaS and why use it. Stay tuned!

September 18, 2013

Retail industry challenges that demand a more mature Testing organization


We live in a digital world and mobile society. It has changed our lives completely, more so for retailers. One bad experience, online or on mobile devices and retail customers are gone forever.. Unlike most industry domains such as banking & financial or healthcare, retail industry is uniquely challenged with high operational overhead and lower profit margin. Overheads include maintaining large number of retail stores where as profit from selling a merchandize could be as low as few pennies. Plus, IT systems are still expected to provide the same level of quality what banking or healthcare industry may require.


In this blog, I am presenting challenges that specifically demand an efficient SDLC process backed by faster software development as well as efficient testing process.



· Legacy applications supporting huge merchandise data


Today's customers demand one-stop shop making top retailers as well as supermarkets, to carry as many as 300,000 merchandise items. Concern is - most large retailers' IT systems were designed 20-30 years back and endless software releases have resulted in spaghetti code making testing, complex and code coverage - ineffective. Hence, retailors ought to institute an end-to-end testing process and provision variety of test data in non-production environments to successfully test all applications, especially in medium to large sized programs.


· Relatively new concept of online grocery


As mobility is spreading its wings, retailers are required to host entire physical store on customers' fingertips.  Mobile and online shopping are no more an option but preferred shopping methods. This trend necessitates a robust mobile application. Hence, requires a relatively new type of testing i.e. Mobile Testing.


· Retail promotions


Unlike other domains of industry, most retailors publish marketing fliers for daily or weekly merchandize promotion. In addition, coupons and price match policies now require IT applications be available on multiple retail channels.  Any delays or error in configuring and testing the promotion systems, can impact speed to market of the product or promotions. On top of it, intense competition demands rigorous and efficient regression testing.


· Global presence, e-commerce and online shopping trends


Global presence, ecommerce and online shopping trends are pushing retailer's corporate security departments to inject secure software development practice. From a testing perspective, QA departments are now on the hook to develop security testing framework and tools to ensure web applications are safe.


· Retail - Mobile POS Security and Infrastructure 


The nature of retail sales processes and devices being used makes infrastructure & security testing, crucial. Recently a top US supermarket chain had major security issues with its POS (Point of Sale) system. Credit card reader was storing and transmitting credit card number to hacker's site. Result - Scared customers and a dent in the organization's reputation. Increasing POS applications, in-store mobile/ POS device, POS credit card devices, credit card enabled mobile apps, credit card reader devices as well as new NFC technology - these demand an effective security testing and a thorough testing of the underlying infrastructure.


· 24/7 Uptime for POS application


24/7 uptime for POS applications is another unique challenge for retail IT.  POS applications such as store billing, back office and pharmacy are, top customer facing applications that need to function accurately all the time. This requires application availability testing to be performed on all retail POS applications.


· Global and large employee base demanding robust HR payroll applications


Large retailers may process as many as 200,000+ pay checks per week. In addition, HR/ Payroll systems are subjected to government's HR regulation - on a constant basis. This demands efficient and effective regression testing. HR Regulations may be country and/or language specific. This brings additional testing needs i.e. localization testing, specifically for the global retailors.


· Supply chain and ever changing retail suppliers & vendors


One of the most critical functions of retail industry is Supply chain management (SCM), which is hugely dependent on IT as it tracks even the minutest detail of raw material to finished products.  Any abnormality or error in order management, payment processing or merchandise tracking, has huge potential to bring the businesses down in minutes.


Unlike many other industries, retailers are constantly required to induct, maintain and retire suppliers, ensuring no payments done for the retired ones. This warrants a robust regression testing else retailers may end up making false payment to incorrect suppliers.


Although challenges of test data, new testing types, or vendor driven testing process is not specific to retail but the underlying business reasons are different. So, no matter what industry sector an organization is, an organized and independent software testing concept is not an option but need. Such a need can be very well, fulfilled by effective implementation of Testing Center of Excellence (TCoE).   

In my next blog, I will talk about best practices and testing tools that can help overcome above challenges.

September 15, 2013

Different BPM Products: What difference does it make from a testing perspective?

This blog is the third in the 3-series blogs on validating BPM implementations. Here, we focus on the common products that are available in the market, their common use in enterprise implementations from a testing perspective.


According to a recent Forrester report on BPM, (The Forrester Wave: BPM Suites, Q1, 2013) in the year 2013, BPM suites are to take a center stage in Digital Disruption. The Disruptive forces of change include technical, business and regulatory changes.  It is a well-established fact that key to any change management or strategy implementation is to start small, think big and move fast! When it comes to the validation of BPM products, the story is not much different, except that we typically see testing organizations fail when it comes to scaling and moving fast. Business Process validation might be a success in silos or at an application/project level, however when it comes to Enterprise processes or integration involving several systems, across LOBs, we just cannot move at the same pace. One key reason is the lack of understanding of the overall BPM picture at hand.


Organizations embark upon automating process workflows (BPM) for many reasons. It could be aimed at streamlining some of the major workflows, to increase predictability in the workflow or it could be to reduce costs, manpower reduction, improve process efficiency etc.  BPM thus helps bridge the gap between Business and IT. To an extent, it equips the business team with the knowledge of IT implementations and helps act accordingly.

From testing perspective, an understanding of the different Business Process Management products will help us test applications/projects correctly. In other words, you understand what you are testing!  It will also help senior management to gain a seat at the table with the business leadership, as it equips them with the vocabulary of business and outcomes. Some of the major product vendors in the BPM space are:  IBM, Oracle, PegaSystems, TIBCO, SoftwareAG, Cordys and Appian.

Forrester (The Forrester Wave: BPM Suites, Q1, 2013) classifies the BPM products as belonging to 3 market segments:

a)     Human centric workflow

b)    Document centric workflow

c)     Integration centric workflow


a)    Human Centric work flows are those that require manual attention. Most of the back office processing, work flow items requiring several approvals, manual intervention etc. fall under this category. Replacing human actors in an entire work flow might seem a herculean task, but it is not impossible. It improves the process predictability. Workflow progress is not hidden at the desk of corporate staff, but it is traced in the systems, visible to the management. These processes are not generally voluminous in data terms, but they might involve several 'stages' in their lifecycle.


b)    Document Centric workflows are those that involve heavy paper work / documentation. This is common in the FSI segment. Banks and Insurance houses have several correspondences / letters that have to be generated, sent to customers, to other banks / organizations etc. Examples include monthly statements, address change notifications, loan approvals, claim processing details, premium renewals etc.  Products such as Thunderhead, HP Extreme are good examples but they focus only on Document Generation and not the entire workflow. IBM Business Process Manager is another good example.


c)    Integration Centric workflows are those that are at a still smaller granularity. This involves great amount of decision making, based on events and up/down stream system extracts / directives. Examples include workflows in Telecom, Banking and Insurance domains. Integration between legacy back-end systems and SOA are very common in the FSI space. Products like TIBCO Active Matrix focus more on this segment.  TIBCO provides this as an extension of its SOA suite of products.

Over a period, there has been a significant change in the way a workflow is implemented. End users want to do most of their banking via smartphones / tablets. They have become demanding and mercilessly switch business providers in search of more value and better servicing. In an attempt to answer this, businesses embark upon workflow integration and automation making BPM Product vendors sell their products in this space. As testers, we validate it!

You can access the other blogs in this series at:

Business Process Management (BPM) Implementation - An Introduction

Testing of Business Process Implementations

September 3, 2013

Is your Performance Testing in an Agile SDLC really 'agile'?

Agile software development is seen as a value-driven method where cross functional teams are expected to deliver and respond to changes quickly and for the iterations happening throughout the development cycle. I have come across performance testing team working in agile projects complaining about burnout due to excessive testing and effort overrun. While an agile method promotes incremental development to changes, it creates quite a bit of challenge for the performance test team. Performance testing team expects a functionally stable environment to carry out tests and investigate performance issues but with continuous changes happening in the code it seems a distant dream. Once bottleneck is identified, performance tuning is done that involves isolating the code segment causing performance issue and rectifying it. However, continuous changes in the code and the functionality make it difficult to pin point the code causing performance degradation. Moreover, short sprint timeline for performance testing does not leave a room for deep dive analysis.

Nevertheless there are benefits too when performance testing goes hand-in-hand with agile development cycle. It is well known that the cost of fixing a defect is less when detected early in the life cycle stages. The objective of testing in the sprint is to test performance early and often. It becomes more costly to incorporate the changes related to performance when testing takes longer time. Performance testing team becomes an important part of the collaborative team and a continuous feedback to developers, architects, project team on application performance is provided which finally results in the delivery of a superior quality product.

To quantify Agility in a project a metric called Agility Measurement Index (AMI) is used which can be derived based on five dimensions - Duration, Risk, Novelty, Effort and Interaction. A low value of AMI indicator is a combination of short duration, low risk, low novelty, limited effort and with minimal customer interaction and that project is more likely suitable for waterfall model. To read more about AMI refer to the research paper titled METRICS AND TECHNIQUES TO GUIDE SOFTWARE DEVELOPMENT published by Florida State University. I believe similar agility assessment is required separately for the performance testing as well to take a decision as to whether the performance testing piece is really a right candidate for the agile cycle.