Testing Services provides a platform for QA professionals to discuss and gain insights in to the business value delivered by testing, the best practices and processes that drive it and the emergence of new technologies that will shape the future of this profession.

« Is Performance Testing really Non-Functional Testing? | Main | Manage tests, the automation way »

Validate to bring out the real value of visual analytics

Author: Saju Joseph, Senior Project Manager

Everyday enterprises gather tons of data streaming in from all directions. The challenge lies in taking this huge volume of data, sometimes unstructured in nature, synthesizing it, quantifying it, and increasing its business value. One way to achieve this is by moving from traditional reporting to analytics.

And a valuable form of analytics is visual analytics which helps enterprises visualize their strategic and operational goals, and drive future financial success. The innate tendency of humans to draw insights from visual representations makes data visualization powerful and visual analytics an effective arsenal in business decision making.

Visual analytics differs from traditional BI reports in three key ways:


Traditional BI reports

Visual analytics reports

           Indicates the status at a specific point in time

-         Displays progress over time towards specific goals of the enterprise

     Translates raw data into information

          Transforms information into insights

   Follows a push approach, where reports are passively pushed to users

        Allows end-users to both enter and retrieve data from enterprise systems, by following a pull approach to answer specific business questions


Visual analytics - Validate to enhancing credibility

Visual analytics operates with three major goals in mind:

  • Efficiently communicate the right information to the right user
  • Enable better decision making
  • Tell the story

Analysts need assurance about the reports presented to them. Specifically, they need to know how accurately the report showcases the results and account for that information when deciding how to use the reports. Establishing such credibility is not easy. There are two ways to enhance the credibility of analytics reports:

  • Transparency - Describes  how the analytic model / algorithm is built
  • Reliability - Describes how well it replicates reality

A validation process in visual analytics helps achieve these two goals and enhances the credibility of the analytics reports. But the strategies applied for validating analytics reports should be over and above the strategies applied for validating traditional BI reports as analytics reports transform information into insights in addition to what traditional BI reports are meant to achieve.

Strategies for visual analytics validation

In general, the following four strategies can be applied for validating analytics reports to impart user confidence in them:

  • Expert validation
  • Predictive validation
  • External validation
  • Cross validation
Strategies for visual analytics validation.png

Expert validation: This phase of verification includes evaluating the model / algorithm used in the analytics report for structure, data sources used, problem formulation, assumptions made, and results expected by experts in the problem area. Expert validation is a subjective evaluation and the process increases the credibility and acceptance of the report results.

Predictive validation: In this process, the analytics report results are generated by prospectively varying input attributes. This is the most desirable type of validation, predicting what will happen in the future. Sensitivity analysis can be performed to explore how results change with variations in the input parameters.

External validation: This method makes use of the huge amount of historical data available within enterprises before implementing new analytics reports. A strategy is devised and applied to feed the historical data as report inputs and for comparing the results with real world results. It is important to perform multiple validations that crisscross a wide range of scenarios. This validation tests the analytic reports ability to calculate actual outcomes. External validation results are limited to the extent to which the data sources are available and the types of data within data sources.

Cross validation: This is the most crucial part of the validation process - validating the analytics reports results independently with similar / comparable model / algorithm addressing the same problem. The difference in the results and their causes are then examined. A high degree of dependency among the models / algorithms reduce the value of cross validation.

Analysts and senior management use analytics reports in the areas of consumer analytics, experience analytics, risk analysis, fraud analysis, etc. for better data-based decision-making, better enablement of key strategic initiatives, better sense of risk, and the ability to react to changes. Application of proper strategies for analytics reports testing increases the credibility and transparency of reports and establishes quality standards by optimal practices. It also gives information on how accurately the reports predict the outcome of interest to account for that information when deciding on how to use the reports results.

In the world of visual analytics, validating the data represented and performing eye-candy tests done as part of traditional BI report testing are just not enough. Visual analytics reports are developed to help the decision maker when the questions are too complex. The team responsible for validating analytics reports should strive to ingrain these strategies in their visual analytics process. A well-validated analytics report can provide invaluable insights that cannot be obtained otherwise.

Post a comment

(If you haven't left a comment here before, you may need to be approved by the site owner before your comment will appear. Until then, it won't appear on the entry. Thanks for waiting.)

Please key in the two words you see in the box to validate your identity as an authentic user and reduce spam.