Google search improving by baby steps

Yesterday Google demonstrated experimental search extensions that utilize semantic information contained in Gmail messages about things like Amazon purchases and airline flights, extending their earlier work on semantic search.

Semantic information enables Google to provide better search results and answer specific questions. For example, as shown here, Google "knows" the values of some of George Washington's attributes, including the fact that Mount Vernon was his home.

It also knows some of the values of some of Mount Vernon's attributes:

Google will test the use of semantic information in Gmail messages in a trial limited to 1 million volunteers. (You can sign up here).

If you sign up, Google will use information about your airline flights and Amazon purchases that it finds in Gmail messages. For example, it will use the values of the attributes of airline flights like airline, departure and destination airport, and flight date to find things like your "flight to Cancun last year."

Google's strategy is to improve search a little bit at a time -- baby steps.

Google goes beyond text search with their Knowledge Graph

Soon after he created the World Wide Web, Tim Berners Lee turned his attention to the semantic Web -- a Web of data rather than documents. Google is now rolling out their first step in that direction, the Knowledge Graph.

Google's 2010 purchase of Freebase and Metaweb, the system used to create it, was a key step toward Knowledge Graph. Freebase is a semantic database, which knows the attributes of entities and the relationships between them. For example, Freebase knows that Larry Press is a person and the value of his city of birth attribute is Pasadena, California (not Pasadena Texas).

Google started with the Freebase concept and added data to create the Knowledge Graph database, which now contains 500 million entities with 3.5 billion attributes and connections.

Let's look at an example. I started with a vanity search for myself, and the following profile was displayed on the right hand side of the screen:

Note that it did not know the value of any of my attributes, it just returned a link to my Google Plus profile and the first few sentences of my most recent posts. I guess I am not one of the 500 million entities included in Google's Knowledge Graph.

Next I searched for George Washington, who is a bit better known than me, and is included among Google's 500 million entities.

In this case, it knows his nicknames, date of birth, etc. Since he is not just a person, but a president, he also has a vice president attribute.  It also knows that he died at Mount Vernon, which is another entity that is included in Google's Knowledge Graph:

While the Knowledge Graph was developed using the Freebase tools, Google did not import the user-contributed Freebase data. (I am in Freebase, but not in the Knowledge Graph). That says Google is abandoning the Wikipedia-like openess of Freebase, in which users could add entities and change the values of their attributes, for a database that is currated in house. That will limit its growth and its "Internetness."

This is an interesting announcement, but Google is not the only player in the Web of data game.

Apple has attracted a lot of attention with Siri, a speech-driven application that answers questions by querying Wolframalpha, another semantic database system. Knowledge Graph gives Google an answer to Siri and Wolframalpha. (Wolframalpha goes further, incorporating a powerful symbolic math engine).

Microsoft is also working on the semanticly rich Web of data. They characterize Bing as an "answer engine" rather than a "search engine," and Microsoft Research has a Semantic Computing Intitative. Microsoft will no doubt incorporate their work into Bing.

The Web is getting smarter -- we may move from today's Web of documents to a Web of data and eventually a Web of knowledge (an ill-defined wannabe buzz word I've heard).  It makes you wonder what it will be like in fifty years.

The Web of Data -- WolframAlpha is two years old

We talk about different ways to retrieve stored data. Text search ala Google is most common, but we also have relational databases like Zoho Creator, hierarchical taxonomies like the Yahoo Directory or the Dewey Decimal system used in libraries, and keyword tags like the label terms in the right hand column of this blog.

Wolframalpha, which just celbrated its second birthday, stores and retrieves structured data, but it goes beyond retrieval, using the data for computation. For example, when I entered "New York to Los Angeles at 100 miles/hour," it computed the straight line distance using geo-coordinates and then computed the time to travel that distance at 100 miles/hour. (Click the image to enlarge it).

It also showed the assumptions it made -- that I meant the city New York, not the State or Financial Note and I meant Los Angeles, California, not Chile -- and it inferred that I wanted it to compute travel time from the fact that I had included a velocity (100 miles/hour) in my query. It also "knows" that a velocity is an example of a broader class, rate.

It easily handed unit conversions. When I modified the query, asking it to calculate "Los Angeles to New York at 1 inch per hour," it reversed the direction of the arrow in the diagram and told me the trip would take 17,804 years 3 months 24 days 22 hours 35 minutes.

My next query was "calories in 2 slices of bread and 2 tablespoons of peanut butter and 2 tablespoons of jelly." It calculated the weight of the ingredients, looked up the number of calories of each and displayed the result: 440 calories. It also displayed other nutrition facts like the amount of fat, cholesterol, and saturated fat.

WolframAlpha has information on many types of object, but it is limited. When I asked for the "calories in a peanut butter and jelly sandwich," it overlooked the bread, and assumed two tablespoons of peanut butter and one of jelly. It does not have data on "sandwiches."

But, the next version may. Wolfram is constantly adding new data. During their second year they added data in these categories: US Economy,International Data, US Social Statistics, Culture and Media, Geography, Astronomy, Chemistry, Earth Sciences, Engineering, Health and, medicine, Life Sciences, Materials, Physics, Money and, Finance, Units and Measures, Math, and Technology and Computer Systems.

It is noteworthy that Wolfram has decided their staff would add new data. This is in contrast to Freebase, a structured data storage and retrieval system, in which any user is able to add data, wiki style. (Freebase lacks the computation ability of Wolframalpha).

Freebase and Wolfram are building a "web of data" as opposed to a web of HTML and javascirpt pages. They include semantic information -- they know about the data they are storing.

Is this the future of the Web? Tim Berners-Lee, the inventor of the Web protocols, is now focusing his attention on the semantic web and Google has acquired Freebase. Google knows you are an instance of the class person -- what can they infer about you?

Freebase -- Wikipedia for structured data?

We have discussed varying degrees of data structure on the Internet, ranging from free-text pages like Wikipedia to rigidly structured relational databases.

Freebase is like a structured Wikipedia, combining the advantages of free text and relational databases. Like Wikipedia, Freebase is open -- any user can create and change content. Unlike Wikipedia, Freebase data is structured so users can query it as they would a relational database.

This example illustrates Freebase. "Topics" are instances of formally defined "types." Here the type is person, and persons have formally defined properties like height, weight, and place of birth.

I am adding a new person (a new topic) to Freebase, myself. I was born in Pasadena, California, and when I began typing a value for place of birth, the system quickly showed the "Pasadenas" in its database with their structured descriptions. It knew that the value of the property had to be a location, and immediately displayed names and property values of locations named "Pasadena." This mechanism both saves time and assures that users have a common vocabulary.

Any Freebase user can create a new topic or change the property values of existing topic. For example, you could add yourself (a new person) to Freebase, and you could also change my place of birth from Pasadena, California to Santiago, Chile.

Thus far, Freebase sounds like a fancy relational database. However, qualified users can also change the structure itself. If one is qualified to be a "type administrator," he or she can create new types and modify the definitions of existing types. For example, one could add the property eye color to the definition of a person.

The developers of Freebase hope that, like Wikipedia, it will attract many users, some of whom are motivated to become type administrators. That way, both the content and structure of a domain of knowledge will evolve over time.

If this sounds interesting, listen to this wide ranging interveiw on Freebase and Metaweb, the system used to create it.