This site will look much better in a browser that supports web standards, but it is accessible to any browser or Internet device.

Jay Currie

One Damn Thing After Another

3/02/2004

Google and Yahoo

To a degree this is being cast as a competition and, of course it is. It is also a pretty interesting exercise in deciding how information on the internet is catalogued and organized. And, if Yahoo is ambitious, it could be the moment when the real power of the net is multiplied.

Google has become many net users' default search engine. Which has meant that how Google sees the net is how millions upon millions of people also see it. Entire websites and discussion groups buzz every time Google tweaks its search algorithm. The more or less monthly Google dance, when website rankings twitch and change, causes consternation, paranoia and despair as people see their information ranked lower than their competitors.

Dark theories abound. Google likes commercial sites more than non-commercial, put AdSense on your site and Google will spider you faster and boost your site up the rankings, Google loves blogs. Companies exist for no other purpose than to "optimize" websites for Google.

It is difficult to overestimate Google's perceived importance to everyone from porn merchants to plastics manufactures because if you are not on the first page of the Google rankings for your particular sort of site you are missing traffic. And traffic equals sales.

At the non-commercial level, Google has essentially structured the internet for millions of users. Through a series of partnerships with AOL, Yahoo and other search engines, Google results have become the de facto internet standard.

None of which is a bad thing. After all, one of the biggest issues of the early internet was simply trying to find reliable information.

If Google posed a problem it was that it was too authoritative. Sites which Google ranked low or missed altogether effectively vanished from the net. Indeed there were occasional calls for Google to be somehow regulated. For some people Google looked like an emerging information monopoly and while there was no clear evidence that it was abusing this position, the possibility existed. Worse, as Google headed towards its IPO concern grew that its commercial interests might over ride its original business model of simply being the best search engine on the internet.

Enter Yahoo. For years Yahoo ran an internet directory operation - a list of sites rather than a true search engine - and supplemented its results through a partnership with Google. On ______ Yahoo announced it was cutting its tie to Google and that it would be crawling the internet on its own.

At a technical level the Google and Yahoo crawls of the internet will start out with fairly similar results. After all there are only so many ways to determine the content and relative importance of a website. However, those results will tend to diverge over time and that divergence will come to represent two very different versions of the vast array of information the 'net can provide.

Very subtle differences in the weight the two search algorithms assign to such things as inbound links, keyword densities, size, search engine optimization strategies, internal links and the value of a given inbound link will mean that the results of the two engines will begin to diverge.

Fairly quickly, the Google map of the internet and the Yahoo map of the internet will become maps of different internets. While the major landmarks are likely to be the same, the detail may very well emerge is strikingly different contours.

Other entrants, including Microsoft, are preparing to offer their own search algorithms, their own maps. With each new map, the Internet will be re-invented.