External Data Analytics: Data from AC Nielson and IMS (medical POS data) is analysed by most of the organisations already, but mostly in silos at country level.  This way the metrics and other parameters they measure are usually country-specific, leaving very little information for sharing across countries.  Organisation-wide implementation of the external data analytics is gaining ground mainly in increasing the productivity, optimise costs and increase information sharing.  Since there is a lot of hesitation for org-wide adoption, cloud plays a very important role in providing scalable infrastructure.  Cloud also provides anywhere-access, thus making information sharing even more easy

Business Intelligence always evolves at an organisation, hence is best described as a journey, rather than a reaching a destination.  The side effect for this evolution is that it accumulates lots of data over a period of time, and generates attention across the organisation.  Our experience shows that data accumulation has serious consequences on server scalability and slows down future developments.  Its popularity among the other functions across the organisation attracts more data usage and building extra data marts/ data pens etc.

Let me start with the multiple hierarchy requirements.  The SKUs are supposed to be rolled up as per the Product Group (main one: based on type of material), Brand (different names for different customers, regions), Sub Brand (based on grade of the material), Packaging size (based on the quantity packed), Business Unit (whether selling directly to big retailers, retail consumers etc.), Category (manufactured / bought), Recipe (to indicate main ingredients), and SKU Case (trollies, packs, SKU etc.).   A huge number of hierarchies as you can see....  These are the ones recognised so far.  There are many to be defined yet.  

This is about the same customer about whom I described earlier as:

Right...we are living in an age where social networks play a big role in the rise and fall of political leaders, ideologies and institutions...we all vouch by the significance of this phenomenon in our daily life...we all have our virtual identity as our alter ego. But when posed with the question of Enterprise Social networking, Shakespearian style soliloquies and musings continue to live on in the minds of Senior Management..."To B(log) or Not to B(log)...that is the question..."

Is this too powerful to be contained within the enterprise? (Is 'contained' an old economy expression...That is yet another discussion). Does it appear to be too destructive (to the corporate fabric) and distractive (to the corporate citizens)? Is Enterprise 2.0 trying to wish away Social Networking - its very own bedrock?

May be it is time to take a relook into the whole phenomenon....

There was a time when Enterprise Social networking was all about throwing in a set of the so called "Enterprise 2.0 applications". We all have gone past those ages of infancy. We have started to appreciate the deeper facets and cultural hues of Social Networking and that is all to do with the way people communicate.

In this new paradigm of collaboration, an organization's "information fire hoses", transforms into "Channels of Expression". The seemingly casual, huddle formation but very relevant and useful exchange of information that normally happen at the water fountains now becomes the norm. Likeminded people flock together and start to express freely and Interesting topics attract user attention and people pour their hearts out. Discussions change course through the conversations...really interactive and participative. In this construct 'Organization' became incidental and the organizational hierarchy breakdown. Wear your creative hat for a second and try to visualize a day in this 2.0 Enterprise - You meet the CEO at the water fountain almost every day and even the newest employee get an opportunity to walk the corporate corridors. Right...in this world 'Personalities', 'Topics' and 'Need for Self Expression' take the center stage.

So let me go back to the earlier question...Does this appear to be too destructive (to the corporate fabric) and distractive (to the corporate citizens)? With all the perceived benefits around the "breaking down of Information asymmetries", "Employee Engagement" and "Knowledge Sharing" it IS in a way triggering off '"too many changes too soon". And this IS the cause of concerns for the wannabe Enterprise 2.0 CIOs. The organization needs to be primed to handle this cultural shift. The organization need to transition though a set of changes based on its state of affairs and priorities.

If we look at Social Networking or Enterprise 2.0 as a dynamic and synergic co-existence of 3 key entities - People, Purpose and Information (People, Intent and Content), on a framework that enables 'Self Organization' and 'Participation' the jigsaws suddenly falls in place.

Now for the curious and the new entrants into this space, let me explain what this "animal" is

1. Search as a "technology" is revolutionizing the way Information would be Organized Managed and Used within an enterprise.
2. Search & Insight Driven Computing team is focused on leveraging this technology to build the Next Generation information Management Solutions.
3. We are one of the very first System Integration firms to have a set up full-fledged service line in this space and that way we are ahead of the curve in this area.

We started this incubation almost 3 ½ years back and now this Service line has successfully moved out of its incubation phase with a bouquet of case studies, multiple account openings and very strong GTM relationship with the leading product vendors in this space

Now, you may ask what led us into this...A few trends that we saw in the Information Management space, almost 4 - 5 years back, caught our attention and those and, in a way, formed the key rationale for the formation of the Search Practice.

Let me try to recollect them for you...

Trend 1: Organizations had started to realize the value hidden in the unstructured information

It is a known fact that 85% of the information within the firewalls of an enterprise is in the unstructured form. It is more interesting to note that most of the IT investments over the last 4 decades and the entire information management industry had been focused on building solutions to address the structured information. The opportunity and possibility to open the vault of unstructured information and unlock the insights from that excited us and we were thrilled to envision the possibilities that would open up for an organization like ours.

Trend 2: The enterprise boundaries had started to blur

We saw the "post dot com" world slowly crystallizing around "consumers". Organizations were (and are) trying to be more "consumer-centric" or outward focused. The concepts of Web 2.0 and Enterprise 2.0 were emerging and with all these the information that is outside the firewall was becoming significant and there was a need to access and leverage that.

While these were going on there were some interesting development within the Enterprise as well

Trend 3: "Formation of Mini Internets within the Firewall"

Within the enterprise space, due to obvious reasons, we were witnessing the growing heterogeneity, replication and fragmentation of information. The enterprise information domain had started to behave like a "mini internets" within the firewall. For any information architect the challenge would be to see how the most relevant information can be provided to the business users comprehensively and at the right time.

And successful information management frameworks would be ones those are "aware of this chaotic situation and can address that"

Meanwhile on the Technology front a new kid had entered the block, changing the way we looked, accessed and managed information

Trend 4:  Technology Trend: Search has transformed from a keyword lookup mechanism into a contextual information integration engine.

We see this as a journey that would involve

1. Seamless Access to Information
2. Enrichment of Information and User Experience
3. Extraction of Insights &
4. Applications on top of that

Based on our perspective the potential impact this approach would have on the Enterprise Information Management space is very similar to what RDBMS created 4 decades back.

1. We look at Search as a unified information access platform that would lead the users to the right information
2. Search as a virtual data warehouse will help users slice and dice information in a rather contextual manner
3. Search can be your Knowledge Platform that would enrich information through contextual content association
4. Search will eventually be the gateway for all user interactions and will have a  pivotal role to play in user experience management

It is quite heartening to see how valid these thoughts are...even now and to move on with our onward journey

Our search for solution / salvation continued. . . .

We once again looked at the Option 1.  This option is very much valid given a green field implementation.  This follows a classical DWH implementation, but the reconciliation mechanism will be like quids chasing pennies.  Then we looked around for the work that is already done, by going through our Minutes of Meetings and other customer supplied documents. 

We realised that the customer had already implemented Vendor Spend Analytics, Customer Web Analytics, Spare Parts Inventory Analytics and Profitability calculations using some old / proprietary technologies.   Repeating these in the Option 1 is not going to be helpful, other than consuming the precious budget.

The event - Discover 2011 - the annual User Conference hosted by Endeca Technologies, Inc., a search and business intelligence (BI) company featured dozens of customer-led sessions, that deliberated on how the simplicity of Search combined with the power of BI can transform their organizations by enabling business users, partners and end users to make better decisions.

The sessions opened with a bang with Paul Sonderegger (Chief Strategist & Ex Principal Analyst at Forrester Research) who set the right context to the event...He painted a picture of

• A world that is getting more digitized every day
• "Abundance of choices" and the resulting "Scarcity of attention" faced by fellow human beings in this Digital world
• The importance of sifting through the ever growing, abundance of data and to "Guide" the users in their daily decisions

This was followed by and that too quite seamlessly by another insightful talk on "How we take decisions". Jonah Leher author of best seller "How we decide" took the audience into the realm of neuroscience to explain the complex events happening in our own 3 pound supercomputer - Brain - during our mundane day to day decisions, like picking up a box of cereal from the store shelf... How the cognitive centers and centers of emotions work together in guiding us to our choices...

Let me take a pause here...Shouldn't the "Information workbenches" mimic or facilitate this process? Every application that we build thus becomes an information exchange that will help users to experience predictability in their daily transactions.

On the same note, it was quite heartening to see a world that we visualized...a strategy that we charted out almost 4 years back, turn out into reality. This approach called for a fresh look at information management and attempted to perceive next gen information management as

An art of intelligently delivering the most "relevant" information to the user for that specific context

• An infrastructure that can
• Connect with the User,
• Sense the potential intentions
• Predict and Persuade the users to the desired outcomes
• A realm of complete user empowerment, where the users interactively discover their needs and choices

Interesting isn't it...Now the questions in front of us will be, "Will this impact the IT fabric of an organization?" or "Will this change that way we design and develop applications?" The short answer is "Yes"

• The locus of control will shift from Information to the User and User experience will have to be addressed and managed comprehensively
• Analytics and Digital Channels will converge - The technology domains, those were kept in administrative silos rather artificially will come together to address the need of the Digital User, whether it is an employee or a consumer
• Next Generation Information management initiatives will call for even closer functioning of Business and IT
• Explosion of customer touch points will mandate organizations to develop "Seamless and Consistent" Customer Experience Management as a core capability

This organisation has dozens of depots to service far flung customers, many central warehouses to supply the depots, major customers, and also store low shelf life and high shelf life products. It also maintains a huge fleet of vehicles to help in raw material collection and FG distribution.

When an organization decides to implement Data Quality strategy, there are several design patterns that can be considered. The data quality goals should be the key driving factor in choosing the design factor.