Let's talk about BI Layers- Series 2 of 3
In the earlier blog I covered the names of the various BI layers and the need for a logical model/data architecture in an enterprise data warehouse.Now let us get into the details.
Let’s go bottom-up starting from the data source layer.
Data source Layer – This layer depends on the source of data . Source could be operational data maintained by an organization’s existing systems or could be external data where data is provided by external sources or non-operational data where the information required by end users is not maintained in a computer accessible format.
Core data warehouse Layer- This is the most granular layer and I will also agree with BI gurus who argue that this layer should hold fully normalized data to give it the much needed flexibility to deal with complex and ever changing business structures. Last but not the least should support historical data , and must have dimensions/attributes with drill up/down/across facilities.
Data Mart Layer – This layer has subsets of information from the core data warehouse layer and is organized to meet needs of particular business units or functional streams and allows users to slice and dice the data .When compared to the normalized warehouse layer , this will involve a fact table connected to de-normalized dimension tables .
Data Staging and Quality Layer- This layer has the responsibility of copying data and transforming it to the data warehouse format and also for quality control .
Data access layer – This layer is between the data staging/quality layer and the data source layer and helps connect with data stores in the data source layer. This layer is very helpful as it avoids the need to know how exactly the data stores are organized.
I will cover the rest of the layers in my series 3 of 3 blog .
One parting shot -> now that storage capacity is no longer an issue , the ultimate goal should be to have atomic(most granular) data in the warehouse layer as this can be built further up for analysis purposes .Having summarized data at the bottom layer defeats the purpose of analysis.
