Testing Services provides a platform for QA professionals to discuss and gain insights in to the business value delivered by testing, the best practices and processes that drive it and the emergence of new technologies that will shape the future of this profession.

« Role of Open Source Testing Tools | Main | Ensure the Quality of Data Ingested for True Insights »

Assuring quality in Self Service BI

Author: Joye R, Group Project Manager

Why self-service BI?

Organizations need access to accurate, integrated and real-time data to make faster and smarter decisions. But, in many organizations, decisions are still not based on BI simply due to the challenges in IT systems to keep up with the demands of businesses for information and analytics.

Self-service BI provides an environment where business users can create and access a set of customized BI reports and analytics without any IT team involvement.

self-service-bi-testing.png

In Self-service BI, the IT team involvement is limited to creating the semantic layer (useful report data) and business users can dynamically slice and dice different data and look at multiple views of the same data to make better informed decisions.

The key objectives of self-service BI are:

  • Fast-to-deploy and easy-to-manage data warehouse.
  • Simpler and customizable end user interfaces.
  • Reduced IT Costs.
  • Easy access to the source data needed for analysis and reporting.
  • Easy-to-use BI tool that helps in supporting data analysis.

Given the advantages, self-service BI is becoming popular in many organizations and this is disrupting the traditional BI implementation which needs major involvement from IT. Quality assurance plays a key role in successful implementation of self-service BI.

Self Service BI - QA Challenges:

Quality assurance needs a customized approach for self-service BI as compared to traditional BI validation. The reasons are:

Traditional BI testing

Self-service BI Testing

Fixed number of reports, fields and metrics with limited combination of attributes and metrics which are easy to identify for validation

Dynamic report creation and access by business users with large number of possible combinations which are difficult to identify for validation

Report layout is mostly fixed and validation of report layout is easy

Report layout is dynamic due to variety of report creation

Reports are created for end-users and anyone without data knowledge can use the reports

Reports need to be created by users to extract useful information and hence usability of attributes and metrics becomes a key consideration


Due to the above factors, self-service BI creates addition QA challenges as given below-

  • Ensure that the attributes and metrics data provided is usable and easy to understand for business users and that users can generate correct reports as per their custom reporting needs.
  • Ensure that the metrics and attribute data provided to business users is complete and will meet the business need of target business users.
  • Ensure accuracy of attribute and metric data. There are too many combinations of metrics and attributes that need to be verified.
  • The reporting performance is difficult to assess as the combination of reports are too many. Testing needs to replicate the business user behaviour in test systems.
  • Testing has to ensure that the system is user-friendly and intuitive for the business users to create and analyze the correct reports and data

Quality assurance solution:

The quality assurance solution has to address the specifics of self-service BI considering the ad-hoc data modelling, variety and volume of reports by different users considering real time reporting needs.

QA has to focus on multiple dimensions given below.

Validation

Detail

Benefit

Structure validation

ü  Validation of correct folder location for different metrics and attributes

ü  Validation of unrelated metric and attribute combinations to ensure that the metric and attribute combinations are correctly reported

ü  Better Usability

ü  Better quality

Data validation

ü  Validation of correct attribute values

ü  Validation of metric calculation

ü  Validation of dashboard data

ü  Better report data accuracy

Functional validation

ü  Validation of commonly-used attribute and metric combination to ensure data accuracy

ü  Validate the metrics across all levels of aggregation to ensure correct data aggregation

ü  Export, print, email scheduled reports

ü  Drill down validation

ü  Better report data accuracy

ü  Better quality system

Data security

ü  Ensure right access to right roles

ü  Validate that user roles without access to certain data cannot access the data

ü  Better data security

Performance

ü  Ensure report generation within expected time considering the production usage in self-service

ü  Report performance as expected

Test Automation

ü  Automated data validation for attributes and metrics

ü  Automated business rule validation

ü  Improved quality

ü  Reduced regression effort/timeline


Summary:

The QA challenges and solution in self-service BI are different from traditional BI solutions and will need self-service-specific test strategy, planning, preparation and execution for successful self-service BI implementation. So, usability, structure, data, security and functional validation approaches are specifically designed for self-service BI. 


Post a comment

(If you haven't left a comment here before, you may need to be approved by the site owner before your comment will appear. Until then, it won't appear on the entry. Thanks for waiting.)

Please key in the two words you see in the box to validate your identity as an authentic user and reduce spam.