The Infosys Labs research blog tracks trends in technology with a focus on applied research in Information and Communication Technology (ICT)

« How do web 2.0 technologies really impact business processes? | Main | Can BPMS expedite Application Development? »

Agile BI - Why it makes Business Sense

I, like most of the BI dreamers and practitioners who get challenged by latest in the BI world, have been trying to make some sense out of the mad rush towards host of BI solutions which are claiming to reduce latency between data creation to decision taken. Making an effort to simplify the whole perspective of Agile BI.

For years faster processing, reduced costs, automations have driven the IT budgets of organizations. The result of this demand from IT is: increasingly complex world of ERP, Supply Chain, CRM, Financial applications, BPO based applications. However, in today’s fast changing world where latency between data creation to decisioning leveraging that data is reducing - those applications in itself are no longer serving purpose of competitive advantage for corporates - because that every other corporate too have.

Reduce the IT investment and increase the operational efficiency - two inversely proportional aspects theoretically, but then again isn't that what every CEO/CIO is demanding? We have been hearing terms like Real-time BI & when someone realized that's not practically possible it was re-framed as near Real-time BI. If that's not enough to confuse you, let me add another term "in Memory solutions" - which will act like a blackbox between source systems and end-reporting systems, and with a magic wand, provide you with an integrated live transaction data to do BI reporting on.

We already understand and know about the Agile development methodology of Software Engineering principles. Typical Agile methods break tasks into small increments with minimal planning. Iterations are short time frames, called timeboxes, of few weeks. Each such iteration involves a team working through a full SDLC including planning, requirements analysis, design, coding, unit testing, and acceptance testing via which a working product is demonstrated to stakeholders. The key to success of agile methodology was the IT teams were able to demonstrate tangible results to stakeholders in shorter spans of time, and allow them to add value to enhancing solution towards their needs.

Effectively all those new terminologies/solutions are result of the pains the organizations have been going thru to deliver effective decision systems in shorter span of time, and thus the advent of term "Agile Business Intelligence". Agile BI can be defined as inexpensive (in terms of lesser resources or time or infrastructure), quick access thru visualization to the wealth of information lying in any corporate that can be brought to the desktop of business users for which its relevant, context sensitive and timely. Benefits:
1. Low upfront investment - fraction of overall BI budgets
2. Immediate yields for better business insights
3. Daily non-technical users of business data can work directly with the data effectively, intuitively, and yet have better insights towards solving their business challenges - instead of requesting IT to provide those solutions
4. Allows for data modifications, drilling, mining, filtering etc
5. Focused on business users rather than technology - equipping users to be innovative, creative and find their way towards answers to business problems

Today the target audiences for Agile BI are small to midsized organizations, and even the smaller departments within few large corporates, as for them agility and responsiveness to the data needs is utmost priority.

Expectations from an Agile BI - Business Perspective
1. Give me tools that i can navigate and work my way around - with minimal to know help
2. Should have the insights into context sensitive data - drill down/up, slice/dice, filter, pivot, compare, merge etc functionalities
3. Flexibly add more data sources to increase scope of analysis
4. Least dependency on IT for any data need
5. Better visualizations, intuitive dashboards/scorecards, graphical representations etc

Expectations from an Agile BI - IT Perspective
1. Tools that can integrate with variety of data sources, scalable, handle larger sets of data etc
2. Require least involvement and technical support once implemented
3. Scalable model from department to Enterprise level strategy and solution

The concept of Agile BI can be related to the maturity of a product wherein companies provide early adopter mechanisms to their customers - giving two fold benefits, organization adopting early can tune their solution on the product & product matures when it gets real life taste. Similarly exposing BI systems to stakeholders, using Agile BI, quite early in the game, will help business stakeholders shape their vision and provide concrete feedback about the system. This in turn will allow the developers to respond by continuously maturing the system to align with the stakeholders' vision.

In my view the Agile BI is definitely going to shape the future of BI in a big way, bringing down the cost of IT and yet remain operationally efficient. Especially when we have burnt our fingers in the recession times recently. I would like to hear your experiences in speeding up decisioning for your customers, and thoughts on Agile BI – the way forward.


The part that I liked the most in this post is the articulation of the evolution of the term "Near Real-Time". I must say it is true and I completely agree to it. IT and Business perspectives of Agile BI have come out well. I don't have experience on Agile BI though, I have a view point to share. I agree that Agile BI exposes stakeholder to BI quite early and it's of huge advantage (as opposed to conventional BI way of doing it) to ensure the usability of the BI system being built. I'm afraid that, due to its nature (such as shorter: time frame, team size, budget and etc) agile BI can not handle larger scope per se enterprise wide. To serve enterprise wide audience, it will have to undergo multiple iterations. Thus, in my opinion, agile BI is poised to limit the architectural options of BI system that is slated to be built.

I would agree with the concept of agile BI presented by Yogesh, Agile BI is not a sequence of things that you must do in order to call agile. Customer collaboration is one of the important attribute in converting agile method success. Important point in working with iterative or incremental fashion is development team has to get started right now because within 2 -3 weeks business will be expecing some new feature.

I will agree with Yogesh that Agile BI is a must have in the changing business scenarios where business is not willing to wait for months for 2 columns to be added to their reports. However there has to be checks and balances when implementing Agile BI in maintenance kind of projects where multiple enhancements are being tracked. Few problems when we have multiple enhancements are changing scope, external dependency resulting to change in acceptance criteria, inter enhancement dependency effecting schedule. Few handy tools can be used to overcome these challenges like “Dependency Tracker” which tracks the external as well and internal dependency for 1-2 iterations. “Iteration end Retrospective” where we analyze the reasons for acceptance criteria changes, Iteration beginning scope and effort Vs iteration end effort and scope etc. Being said that we all must agree that without Agile BI business will loose edge over competition.

Nice article, Yogesh. Got to see many concepts covered well. In Agile, it is believed that business users get what they want quickly. Do you think Interactive Visualisations and Mashups will help here...

BI application development has never been a one sided affair. It involves teams from all walks of business, software and services vendor. The key concept for any application or support project to be agile enough is the involvement of expertise. A quick turnaround cannot be achieved with limited expertise and resources. Agility demands knowledge expertise, tactics and a good degree of business need prioritization, suing the concepts of six sigma and lean management.
Many models have been proposed for agile software development and the one that fist BI is the SCRUM model. The approach, unlike typical Agile which may not follow a highly process oriented and planned work breakdown, follows a highly organized methodology of delivering results with a few iterative process. Scrum-ban model is highly proposed for maintenance type projects.

Agile methodology is not new. It was used for decades in the development of DSS technologies serving ill structured decision making processes. Original work by Courbon, J.C. Bourgeois, (1980) Keen and Scott Morton (1978) and all basic principles of the system approach . (before 50') .Known as evolutive approach or Middle out design it formally reinforces the need of incremental progress led by end user as claimed by agile.

Post a comment

(If you haven't left a comment here before, you may need to be approved by the site owner before your comment will appear. Until then, it won't appear on the entry. Thanks for waiting.)

Please key in the two words you see in the box to validate your identity as an authentic user and reduce spam.

Subscribe to this blog's feed

Follow us on