Mounds of Data Vs. Useful Knowledge
Introducing the Knowledge Graph [http://www.youtube.com/watch?v=mmQl6VGvX-c]
The volume of digitally available data is growing by leaps and bounds. Estimates of this volume have exceeded astronomical scale and introduced words in our lexicon such as peta, zeta and exa. But at the same time, this abundance of data has widened the gap between available data and the knowledge we're able to glean from it.
Not surprisingly, there has been a steady rise in the number of Internet experts who think there is a tectonic shift from online search to online discovery. If this shift is indeed taking place, enterprises will begin to experience a boom in actionable intelligence. To put it another way: They will be able to transform data into knowledge and actionable intelligence.
As it stands now, social media produces an enormous amount of unstructured data within both companies and society as a whole. Add to that the semi-structured data that companies find scattered across emails, activity logs, and enterprise applications. It's important for organizations to understand the meaning of all this data and to discover its various connections and relationships in order to derive value.
We're constructing what we describe as an advanced knowledge-graph technology stack to track events originating in different data sources. This technology has its antecedents in the work on 'semantic networks' in Artificial Intelligence dating back nearly five decades, and it shares features with Google's use of knowledge-graphs. This technology allows systems to connect the dots, so to speak.
At the heart of this graph technology stack is a data set. It stores entities, events, and their relationships in a graph. A graph database - as opposed to a normal relational database - provides us greater flexibility in discovering new dependencies and relationships. Surprisingly, only a few companies are currently looking to automate their operations in a way that exploits knowledge buried across multiple sources. True, it's ambitious to resolve a majority of issues using these techniques. But even automating 30 to 40 percent of the resolutions will significantly transform IT operations by shifting to near-real time issue resolution.
Today, when someone raises an alert via search, a subject matter expert is assigned to the ticket and resolves it by manually reviewing various system and device logs and consulting with other experts within the IT operations team. Now, imagine instead of a subject matter expert the system identifies similar problems and their solutions by scanning the graph technology stack. This is the future we are building.
When used in conjunction with an Internet search, visualization, semantic and inference technologies can help boost operations risk management. Some of the world's largest telecom companies have already deployed graph-based technology to solve connected data issues. They've applied the technology to areas like customer accounts, social gaming, and contextual messaging.
In the old days (meaning up until a year or two ago), finding things online required using Web-based string and keyword searches. That meant matching user queries in very large, indexed data sets to provide the user with a near-match. Google's Knowledge Graph is attempting to make its searches smarter by shifting focus from strings to things. Knowledge Graph uses a combination of advanced technologies like Natural Language Processing and Semantic Extraction to understand the searcher's intent.
Here's roughly how it works: Google Knowledge Graph encodes data about entities such as individuals, places, organizations, and teams. It not only shows information about famous people; it tracks items about you and me. In fact, Google says it tracks some 570 million entities connected by 18 billion facts. Recently, Google released a new set of improvements. The system, for example, will anticipate what your next question will be and add relevant statistics to the response. If, say, you want to know more about how many people live in India, Knowledge Graph will also show you statistics for another large, Asian country like China or Indonesia.
These exciting developments bring to reality what Tim Berners-Lee, considered the father of the World Wide Web, elaborated more than a decade ago. He believed that computers would someday be able to understand the meaning of things. Over the last few years, this idea has gained more and more ground, primarily with the advent of dictionary-like knowledge websites like Wikipedia, not to mention the growth of social networks and the adoption of shared vocabulary standards.