The Infosys Labs research blog tracks trends in technology with a focus on applied research in Information and Communication Technology (ICT)

« Location Intelligence - Part 2 | Main | Parallelism - Scalability and Amdhal's Law »

NextGen Data Warehousing Trends - Part I

"Necessity is the mother of all inventions" - this quote holds true today as well, except the fact that we are starting to realize the necessities based on the inventions that are shaping up. Data Warehousing is certainly no exception, and over the past years we have seen various avatars of Data Warehousing shaping up organizations, and driving their growth. To name a few - Enterprise Information Management, Operational Intelligence, Real-time/Near Real-time Data Warehousing, BI As a Service (BIaS), in-Memory analytics, Master Data Management etc.

And mind you this trend is not going to stop here, and I am trying to provide few trends which are potentially going to shape the Next Generation of Data Warehousing in coming years. This blog will be a series, explaining each of the trends separately to ensure due credit and focus is provided to each platform/trend.

The various trends shaping the NextGen Data Warehousing are listed below, and this is not an exhaustive list:
1. Data Warehouse Appliances
2. Open Source Databases, Integration and Reporting solutions
3. Advanced Analytics - Predictive Analytics
4. Massively Parallel Processors (MPP) architectures
5. In-Memory solutions with larger data caches leveraging 64-bit platforms
6. XML based/Web Services or SOA based Interfaces
7. Columnar Databases
8. Real-time Integration between Data Warehouses and Operational systems
9. Saas and Cloud Computing transforming Data Warehousing and BI reporting
10. Multi-domain Master Data Management, Model Driven MDM solutions

On a careful observation of the trends, a common focus and thought process of ensuring the improvements in query performance and platform scalability is quite visible. In addition to this, there is a strong need to loose couple the systems/platforms/applications without compromising on the data integration & quality aspects allowing to better leverage the existing investments to the best usage. Idea is to expose the services and work in a collaborative model.

Lets talk about one of the trends "Data Warehouse Appliances" to start with, and will follow other trends in successive blogs. The term "Data Warehouse Appliances" was coined by Foster Hinshaw, Founder of Netezza . These are typically used in large Data Mart implementation where people are expecting use multi-TB's of live data.

What is Data Warehouse Appliance? - An integrated set of Servers, Storage Media, Operating Systems, Database Systems, ETL/Reporting/Metadata Softwares pre-installed/configured for Data Warehousing platform. The platform definitely involves the underlying networking layer as well.

Who all are providing Data Warehouse Appliances? - Teradata, Netezza, DATAllegro, Kickfire, Kognitio, IBM Infosphere Balanced Warehouse, Oracle Optimized Warehouse to name a few.

What are the benefits of Data Warehouse Appliances? - Actually several advantages, and few of the key ones are:
1. Out of the box performance delivered - Entire platform with performance built out of the box for usage, no need to go on shopping spree separately for hardware, Softwares, ETL, Reporting environment etc
2. Offload your Enterprise DWH platform with high performing ad-hoc queries, thereby freeing the Enterprise Data Warehouses for the power users
3. Single vendor - translating to single point for any administration needs, support services done via one support center
4. MPP Architectures help achieve high query performance, high availability & scalability options - all built in

What are the areas of application of "Data Warehouse Appliances"? - Data Marts with large querying needs for analytics that typically would put Enterprise data warehouse under pressure for performance, short term based deployment projects requiring little data integration, isolated query intensive and ad-hoc based analytical solutions requiring tera-bytes of live data - are few of the examples where DWH Appliances would fit well.

That's it for the Part I, watch out for the other trends in coming blogs.


Good article Yogesh it's a summary of all the upcoming things in DW.

Post a comment

(If you haven't left a comment here before, you may need to be approved by the site owner before your comment will appear. Until then, it won't appear on the entry. Thanks for waiting.)

Please key in the two words you see in the box to validate your identity as an authentic user and reduce spam.

Subscribe to this blog's feed

Follow us on