Data Lineage and Impact Analysis
Traditionally BI implementers have had to in a way fight for scraps of information pieces contained within multiple applications or documents. This would then be loosely coupled together to cater to a quick fit-gap requirements exercise or in other cases a more detailed impact analysis job.
With the introduction of these new features, Oracle has ensured a highly integrated and transparent environment providing metadata to put together metadata from the data warehouse, reporting and configuration layers and creating a holistic and end-to-end view of the BI Apps dataset.
How Data Lineage Works in BI Apps
In BI Apps data traverses through multiple layers before actually reaching the dashboards and reports. From a reporting perspective the physical model of the data warehouse is mapped to a business model which forms the logical or semantic layer. This in turn is mapped to the OBIEE presentation tables and columns which are used for the dashboard and report creation. The warehouse itself is loaded in two stages, first using source specific data mappings (SDEs) to extract data from source to staging tables and then using source-independent mappings (SILs) which are used to populate the final target data warehouse tables from these staging tables.
With ODI used to extract metadata from the BI Apps Configuration Manager and its own tables, Oracle has integrated this quite seamlessly and combined the OBIEE metadata through use of RPD and catalog metadata documentation. An ODI load plan then is executed to combine the data contained in these metadata repositories and stored in warehouse tables which are then used for the OOTB Data Lineage dashboards and can also be used to create one's own data lineage and impact analysis reports.
Benefits and Improvement Areas
Once the initial setups and configurations are done and the data lineage tables have been loaded, the OOTB model itself provides enough metadata to cover all the lineage and impact analysis reports that might be required. As is evident, this reduces analysis effort drastically and also reduces dependence of any kind on the ETL team to trace back columns. Anyone and everyone is now enabled to look at and understand where their data is coming from! The metadata provided covers all the main stages in the data load process and also extends to OTBI if that is being used along with BI Apps.
Even though we are still in the early days of exploration, there are a few areas which we are yet to discover. There doesn't seem to be any support for formulas applied in analytics and temp interfaces do not seem to be covered. Data lineage doesn't seem to work if views/procedures are being used in the ODI interfaces. Additionally any customization done in interfaces doesn't seem to be captured. We are in the midst of taking a deep dive on these and will post an update ASAP!
The Final Verdict
The Data Lineage functionality provided is definitely a good start and the belief is it would continually evolve to provide an even more integrated, robust and automated solution to information discovery within BI Apps.