Virtualized Data and Automated Discovery
November 6, 2009 9:26 pm BI Infrastructure, BI Strategy, DataIn an IT world that is rapidly becoming virtualized at the hardware and software levels it is not too much of a stretch to envision virtualization at the data level – easy to dream about, not so difficult to create, or is it?
As businesses continue to struggle to capture, clean and transform their data into a format best suited to BI tools, the adoption of BI in critical decision making is stalled.
BI visualization tools are being increasingly integrated directly to applications, relational databases and cubes, using web services and SOA, with innovations such as columnar databases are promising to overcome the format and power constraints that are holding BI adoption at sub par levels.
Virtualized data would abstract the data from its source silo structure, and instead present as a consumable entity regardless of ETTL processes it may have to pass through to become usable to the end BI tool. This abstraction supports the concept of automated discovery, where data from any source, in any format is consumable by BI applications.
With over 80 percent of information relevant to daily business decisions now unstructured, such advances in data management innovation are critical to overcoming current constraints. Omniture, Web analytics vendor are about to release a product to monitor API traffic on the Web, and a lot of keyword tracking to measure application traffic and consumption patterns. This would, for example, allow online retailers determine the best page layouts to sell more products. This comparative intelligence can be fed into BI analytic or visualization tools to add to customer profiling data.
SAP’s Business Objects Explorer also tracks end user activity across related topics at one location and aggregates it with related data feeds. Explorer is data feed agnostic – leaning towards the type of abstraction that defines virtualization. Information may be drawn from text, voice, video, transaction data or anything else as a mashup of structured and unstructured content with mapping providing contextual relevance.
No amount of ‘intuitive interface’ design will match human capability, but a lot can happen behind the scenes that surpasses the ability of humans to correlate relationships between massive volumes of data in very short time intervals. This contextual mapping has advanced far beyond the traditional integration of data warehousing and is heralding another major leap in BI infrastructure capability.
