The future of search

Businessworld

John Battelle calls it the ‘database of intentions’. The co-founding editor of Wired magazine authored the best-selling Search (2005), demystifying what is universally regarded as the Internet’s biggest killer application. Consider Battelle’s underlying opening message: every search query, fed in by one of the hundreds of millions of people worldwide who use search engines like Google, Yahoo!, AOL, MSN for everything they do, gives us a glimpse into what users are curious to know or find interesting. When all those billions of queries are aggregated, what you get is a goldmine of intelligence into the deepest desires of mankind. And such a ‘database of intentions’ would be useful for innumerable reasons — for just about anybody. For marketers. For politicians. Even terrorists.

The quantum of online information is increasing rapidly. In the past year, the number of Web pages indexed by popular search engines has doubled to about 18 billion. Most of this is perhaps irrelevant, and only some of it is treasure. But what use would the relevant data be if you couldn’t find it through the streams of junk. Search has allowed human beings to fulfil a basic need: to find what they want. By turning one of mankind’s innermost desires — the search for information — into a marketing platform, search has become the hottest business online. To understand it better, just consider the home page of the world’s most popular search engine, Google.

Type a query, say ‘home loans’, onto that blank white bar and wait for the results. What you see on the left side are the organic results. On the right side, you’ll see the sponsored links. Advertisers bid on Google to get onto the right-hand side first page. And that’s how Google makes its money.




Think of the organic results as the editorial of a publication. And the sponsored links that you see on the right side as the advertisements. It costs Google $3.4 billion to have the organic results up and running. It makes $6 billion on the sponsored links (paid search). Each time a user clicks on a sponsored link, Google gets a percentage fee.

This paid search market, also known as search engine marketing (SEM), is fast becoming the most disruptive advertising medium. In 2005, it is estimated to have generated $10 billion worldwide — 40 per cent of the total online advertising market. Google had 64 per cent of that market last year. It is expected to grow 41 per cent in 2006, according to a report by US-based securities firm Piper Jaffray.

In India, of the $35-million online advertising market, search engine marketing is estimated at $5 million-6 million. Most of it is attributed to global advertisers reaching out to Indian users. “Only 10-20 per cent of this is Indian advertisers reaching out to Indian consumers,” says Mahesh Murthy, founder of Pinstorm, an SEM firm. This market is expected to double by end-2006. Search query volumes tripled in 2003-04 and then doubled in 2004-05. However, it is still way behind the Chinese search market that is estimated at $230 million.

The players jumping into search come from a host of seemingly disparate industries. Microsoft (msn.com) is the biggest software corporation, while Amazon (a9.com) is the biggest e-commerce player in the world. They feel the need to enter this market because Internet search is blurring the territories and threatening leadership across various domains. Take Amazon. Traditionally, users have gone onto its portal directly. Now if the user ‘googles’ for Harry Potter, Amazon would be just one of the landing pages that Google offers. Microsoft’s dominance of the desktop is getting threatened as Google bundles in several office applications online, like Google Desktop, for free. So, instead of Windows being the de facto landing page, Google could end up taking over that role.

Left vs Right
Clearly, search is the most popular means for users to find information and buy products. According to an industry report, 32 per cent of all products purchased online were after search queries. “Search is disruptive because the marketing statistics we give the advertiser are quantifiable, monetisable and live,” says Ashish Kashyap, head (sales and operations), Google India. For advertisers, this solves two long-standing challenges that traditional media presented. One, there was too much of a gap between the advertisement’s screening and the consumer’s intent to purchase and his actual purchase. Two, the actual quantifiable link between the advertisement and the actual purchase couldn’t be accurately determined.

Search solves both these problems across categories. Take home loans, again. Marketers say the best time for a home loan marketer to be present in an online consumer’s mind is when he’s searching for products like a home loan. So, if HDFC is on the sponsored links, it knows which keywords are most popular, which give it the most leads, and where they come from. Says Rohit Mull, marketing head, Tata AIG: “Search gives us our highest click rates.” Sharekhan.com has, in fact, increased its search budget 20 times in the past year.

You could, of course, argue that many of these people could just be researching home loans for, say, an article like we sometimes do. Says Rishi Behal, director (search), Yahoo! India: “Over 25-30 per cent of the queries generated are commercially-saleable keywords.” That’s the audience that companies like Tata AIG and Sharekhan.com consider relevant. Further, Behal says that 20 per cent of users querying ‘commercially-saleable keywords’ actually click on the sponsored links (that’s when the search engine gets a percentage fee).

Search has another dimension: search engine optimization (SEO). Many companies use Web analytics to crack Google’s algorithm, so that their website is part of the organic results on Google’s coveted first page. Companies prefer using SEO to SEM. It is cheaper to come on the list and more effective because users are more likely to click on organic results than the sponsored links. In the US, SEO constituted 12 per cent of the search industry in 2004, according to a non-profit SEM association. However, marketers say it will continue its slide behind SEM. Typically, 93 per cent of users don’t go beyond the first 20 results. As more companies vie for the first few and as they find it difficult to keep pace with Google’s changing algorithm, it is becoming tough for businesses to come on the first couple of pages. Even for popular keywords, companies can take 6-12 months to have a chance of being featured on the first page.




Companies are, therefore, becoming aggressive on their SEM strategy. Although it’s far less effective for an advertiser to be on the sponsored links, companies can buy their way onto effective leads by paying more. SEOs are also in conflict with a search engine’s interest. One, because if businesses crack the algorithm of a search engine, it dents the latter’s credibility. Second, because for every dollar an advertiser spends on SEO with an Internet marketing firm, Google loses a higher dollar in its SEM revenues. To tackle these issues, search engines frequently change the algorithm of its search.

To keep costs low, companies are now outsourcing SEO work to small software firms in the developing world. And, India is becoming an outsourcing centre. Says Prabuddha Raychaudhuri, CEO of SEOguru Technologies: “There are about 100 Indian software companies doing substantial work in helping foreign companies come on that first page.”

The Next Battle
Type ‘Tom’ on Google, and scan through the first 10 of the 300 million-odd results you get. Each result on the opening page is different, depending on the context or intention that the search engine interprets. As you move onto the next few pages, you’ll find some of the results repeating. It might take a while before you find what you were looking for. It could be Tom Cruise, Tom Clancy or Tom and Jerry.

Now, try the same exercise on ‘smart’ search engines like Clusty.com or Mooter.com. On the page, you’ll find clusters which group all pages under various categories. So, all scattered results referring to Tom Cruise will be grouped together. That’s the first step. Then, there are new search products (like Yahoo! Answers) that answer questions like ‘How old is Tom Cruise?’ Such a Q&A is the next level of intelligent search. Says Dr Prasad Ram, CTO, Yahoo! India: “The future of search is when you do not have to search, but just look up information. The search engine will need to understand users’ context, intent, community, and presentation preferences.”

That’s where most of the battles will be fought — to make search more relevant, contextual and intelligent. Take Naukri. For the past six years, the website has been on a resume-adding spree — it adds 8,000 resumes everyday. The database now numbers 5 million. “It has become a jungle, and we have to bring some order in all this chaos,” says Sanjeev Bikhchandani, CEO, Naukri. In other words, his customers (recruiters) cannot sift through each resume to find the right people. Intelligent search can allow Naukri to present the top 10 percentile of resumes to recruiters, based on several parameters.

Intelligent search also throws up business ideas. Pinstorm found that the number of people searching for a ‘hotel room in Mumbai’ was 960,000 last year. They also found the number of Indian companies marketing their hotel rooms online to be far lower. So, they launched a hotel bookings portal called instabooking.com after tying up with some hotels.

Search India
Khoj was the first Indian search engine launched by Rajesh Jain under the Indiaworld portfolio in 1997. When he sold the portal to Sify in 1999, Khoj was part of the deal. But it has been nothing more than a directory search. Jain has now invested in Bangalore-based Seraja, which will soon launch Eventweb, a search engine that enables Indians to track events. However, industry veterans feel Indian companies have missed the paid search engine opportunity.

V. Ramani, CEO, Mediaturf, says a vernacular Indian search engine is still 3-4 years away. “First, we need vernacular Windows desktops and keyboards. Then, we need vernacular content. Then, regular search will need to index this content. And once traffic picks up, only then can paid search take off,” he says. In China, regional content (in Mandarin) couldn’t be googled. That’s where Baidu (52 per cent traffic share) found an opportunity. It is bigger than Google (33 per cent) today in China in terms of traffic. In June 2004, Google acquired a 2.6 per cent stake in Baidu.




Rediff is possibly the only leading Indian Internet company that is spending heavily on Web search to further localise and personalise the service for India. It has been beta-testing its faresearch (travel), jobsearch and newshound (news) programs after crawling (scanning) the relevant sites.

Murthy, however, thinks it is difficult for an Indian search engine to compete with the global giants. “No Indian Internet company can match these guys on scale or technology,” he says. Don’t forget, both Google and Yahoo! are ramping up their India operations aggressively. Yahoo! is launching its paid search model called YSM (earlier Overture) in India by May this year, and is also beta-testing a job search.

But vertical or specialised search of the kind Rediff and Yahoo! are attempting is both a threat and an opportunity, says Bikhchandani. “The opportunity exists because users can access our resumes even when people use general search engines to find ‘jobs’... and a threat because, then, some may not find the need to come to our portal directly.”

Wait, then, for the next battle in search to begin!

alternative search engines:


What Indians searched for most in October 2006:

India - Popular Queries: October 2006

indian railways (consistently the top search item for Indians)
diwali (the all-important festival of lights)
rediff (the biggest Indian web portal)
air deccan (one of the many new low-cost private airlines)
indian airlines (the state run airline)
sania mirza (the female tennis star also written about for her looks and style)
wikipedia
aishwarya rai (the highest paid and most beautiful female actor)
cricket (India's madness)
rangoli (the decoration of house floors in colorful designs during Diwali)
nokia
airtel (leading private cellular operator)
katrina kaif (attractive female actor-celebrity)
salman khan (actor and companion of above)
angelina jolie (recently visited India)