Testing Services provides a platform for QA professionals to discuss and gain insights in to the business value delivered by testing, the best practices and processes that drive it and the emergence of new technologies that will shape the future of this profession.

« Extreme Automation in ETL testing | Main | Assuring quality in Self Service BI »

Role of Open Source Testing Tools

Author: Vasudeva Muralidhar Naidu,Senior Delivery Manager

Quality organizations are maturing from Quality Control (Test the code) to Quality Assurance (building Quality into the product) to Quality Management. In addition to bringing Quality upfront and building Quality into the product, Quality Management also includes introduction of devOps principles in testing, optimization of Testing Infrastructure (Test environments and tools).

Even though Quality organizations were the primary consumers of test environments, test data and tools, it had limited control on its optimization and rationalization. The usage of open source testing tools was not only restricted by lack of skilled resources, but also by enterprise architects in approving the tools to be installed in the network. The need for higher Quality product is increasing in the ever increasing complex software topology, cost of Quality increase is driving test managers to evaluate and use open source tools for their projects.

Organizations continue its struggle in finding practical ways to improve test effectiveness at reduce cost and improved Quality. Before, I could consider open source tools as one of the potential means; I wanted to first understand if "Testing Tools" is a qualified parameter for reducing cost of Quality.

How much do we really spend on tools as compared to returns? How many tools do we really need to operate efficiently? Once you start listing, the list really looks big. You would need: 

  • Test Management tool (Test Case repository and defect management) - This is an enterprise level, globally used tool with more than 1000+ users
  • GUI test automation tool - Atleast 2 tools in many organizations
  • Requirements management tool (Not many organizations use it)
  • Performance testing tools
  • Data testing tools
  • Data provisioning tools
  • Mobile testing tools
  • Middelware testing tools (SOA)
  • Custom tools 

Many organizations have all the above tools. But have we utilized them well? Can they talk to each other seamlessly? What is the % utilization of your tool license in a given year? What % of tool features do we really use as compared to what is provided by tool vendors?

Now, we might qualify "Testing tools" has having good potential to reducing cost of Quality. The next obvious question is the "Buy vs Build". My focus should be more on my application quality rather than building a framework using open source software. My investment in vendor licensed tools is justified as they help me keep my primary focus intact. I will also not be bothered with the need for highly skilled tool specialists.  This thought process has become old school now. The reasons are obvious:

  • It is more than 10 years since open source community became popular. The community has grown and significant contribution has been made to mature the same
  • Widely tested and validated by multiple user community
  • Cost pressures has forced many organizations to adopt and contribute towards improving open source frameworks and building skilled resources
  • Quality organizations have heavily focused on capability uplift of their resources, many programmers have moved into the testing profession
  • Many enthusiastic sponsors in the organizations promote open source testing tools due to their engineering background

The popularity of some of the open source frameworks has grown significantly over the years. With the advent of Agile and its increased adoption has further pushed open source adoption. The open source test automation tools will also force automation engineers to develop scripting skills and understand the underlying technology better. This will improve the success factors of your automation projects. Also read my (http://www.infosysblogs.com/testing-services/2010/08/need_for_effective_use_of_tech.html) blog 'Need for effective use of technology in testing projects".

I have listed few frameworks/tools below in the table which has already been extensively used and has been accepted as enterprise standard tools. Several frameworks are readily available:


Features and Strength


A power web testing automation tool. Available since 2004. Many new libs and utilities has been added over years. Database utility has also been added. The framework is highly extensible and suitable for today's extreme automation needs


Jenkins is a java based continuous integration tool and works across platforms. It is mainly used for monitoring jobs. He has many testers friends features and helps to achieve seamless integration for end to end automation


Cucumber is a popular and a powerful automation framework for BDD.  Over years it has extended its support to many languages and platforms including Ruby, Java and .NET.


"Web application testing in Ruby" is a popular automation framework which is a collection of ruby libraries. It uses ruby as the scripting language which makes it power browser automation tool. It is widely used in Agile projects as it support Behavior driven development 


Freeware web services testing tool with rich graphical interface, with ability to support functional testing of services, service simulation, Security testing, load testing and many more. The tool has a huge community and good support. If your need is to test only web services, this is the tool.

 This is just a sample of several automation frameworks that are available and built for specific use. If you have already noticed, the common characteristic of each of these open source frameworks is that there is limited IDE and involves good amount of scripting. This helps you in customizing the framework to your needs and integrates with multiple technologies seamlessly. This is the need of the hour whether you are automating a web page or a database. The two basic limitations for commercial usage is availability of skilled resources and dedicated technical support for enterprise usage. Once the user base increases, both the challenges will have its natural death.


The popularity and capabilities of open source frameworks is increasing with every passing day. Software services organizations are creating dedicated user groups to support the framework to increase its commercial usage. With extreme automation being seen as the only solution for catching up with the pace of technology adoption, open source frameworks will play a major role in achieving continuous integration and hence end to end automation. It will also help reduce cost of Quality in a noticeable way. If you have not already starting looking at open source test automation tools for your organization, it is the right time as you are already a late starter in open source adoption.

Post a comment

(If you haven't left a comment here before, you may need to be approved by the site owner before your comment will appear. Until then, it won't appear on the entry. Thanks for waiting.)

Please key in the two words you see in the box to validate your identity as an authentic user and reduce spam.