What next in search?

What happens when you press the "search" button on an Internet search engine and how will the future be different? To find out, this column met Qi Lu (pronounced Chi Loo), President of Microsoft's Online Services Division. The answer is actually a lot more complicated than we thought.

The reason is simple: search engines need to be able to sift through exabytes of data-trillions of megabytes, for those who came in late-and pull out relevant results in next to no time. And, as search engines evolve, the accent is increasingly on relevance rather than plain old information retrieval.

"The early days of Internet search were just information retrieval," explains Lu, who was with Yahoo between 1998 and 2008. He credits Google with changing the search game with its Pagerank algorithm that started using the number of times a webpage was linked to or viewed as a proxy for its popularity.

Lu thinks that search engines are still evolving and Microsoft's "Bing" search engine may have an edge. "The search engine has become a great 'site finder', but it does not always match user intent with the search." To demonstrate, he uses the example of the search string "Chicago". "What are you searching for? Tourist information, the music band or the movie?"

Drilling down earlier meant either going through several pages of pointless results or refining your search string to something like "Chicago Music" if it was Peter Cetera's band you were indeed looking for.

Bing changes that, Lu argues, by adding a third column to the results page, which can guide users to the "intent" of their search. Users approve: Bing has a 11.5 per cent market share in February-up from 8.4 per cent in June 2009, according to tracker ComScore.

But it isn't just user intent that search engine algorithms need to be able to handle. Think how "real time" is being redefined in an era of Twitter. Searches now have to pick out the most relevant tweets on the search topic. In fact, Microsoft has tied up with Twitter to get real-time feeds and analyse them. All this while making sure that copious amounts of junk on the web is kept out of the search results.

What about the advertiser, the guy who is bankrolling free search? Lu believes that a better understanding of "user intent" will also help advertisers because they can target advertising better. "A large travel website would bid for keywords such as Chicago, making the term unaffordable, for say, a small record shop," he cites an example. With a better idea of intent, even the small guy can.

The really big thing next in search, according to Lu, will be mobile search enabled not just by devices but location-awareness. "Suppose you walk into a mall, you can just search for 'sales' on your smartphone and all the results will be from stores near you. This is coming sooner than you think."

Lu also knows that this transition won't be easy. While Bing has gained share, Google still accounts for two of three searches worldwide and is evolving. We bet you didn't know that nearly one in 10 searches everyday on Google runs on a new algorithm the search engine is testing. So, while Bing has started with a bang (and Yahoo search is transitioning to Bing), it has a long way to go.

HOW SEARCH HAS CHANGED!

• User-intent: Better algorithms and understanding of user's behaviour will allow search engines to figure out 'what' you are searching for a lot better.
• Real-time: The advent of platforms such as Twitter means that search engines are constantly updating their databases searching through tweets as they are posted.
• Location-based: Search engines are also aware of your location and in future when you search from your mobile devices, the results you get will be highly localised.