• Register
• Login

SEO Theory - SEO Theory and Analysis Blog

Algorithm analysis, Web community relationship analysis, SEO practices and techniques, industry news, etc.

• Home
• About SEO Theory
• SEO Glossary

Google’s desperate gamble

Posted by Michael Martinez on August 6, 2007 in Web spam

Although it’s still a little too soon to suggest that Google’s downward spiral has finally begun, Google has lost search market share two months in a row. As near as I can determine (based on largely unreliable third-party estimates), Google began losing market share in late May.

Late May. That would be about the time Google rolled out Google Universal Search, which they said would take a little time to be felt in their search results. That would also be about the time that Microsoft and Ask launched new offensives against Google’s market position.

But Ask’s campaign has fizzled, if not actually backfired, and people are overlooking the fact that Compete says Microsoft’s ClubLive was not responsible for their increased search market share. And in my own ongoing analysis of search referrals compared to 2006, I have noticed a significant decrease in the growth of referrals from Google since May.

My personal network’s performance is hardly a bellwether for the Web, especially since I’ll leave my network to its own resources for long periods of time. But I have been actively building out content, redesigning pages, and reoptimizing portions of my network for several months. I’m achieving my targets with respect to Google rankings but the growth in referrals has been cut in half.

But Google also missed an earnings estimate despite increased sales of online advertising. Wall Street may not know anything about search technology, but they can read income and balance statements. On the other hand, Google has been plagued this year by high profile resignations, advertisor pullouts, and lawsuits or government demands for private data disclosure, changes in procedures, enforcement of copyrights, and more.

Everyone wants a piece of Google right now and the users don’t all appear to be happy enough to keep Google in the driver’s seat. When it comes to Web search this is still Google’s game to lose but they have made some spectacular technological blunders over the past year, including:

1. Dumping millions, possibly billions of unique content pages into the Supplemental Results Index with no means of achieving search visibility
2. Indexing thousands of students’ social security numbers from more than one district
3. Granting Wikipedia unprecedented dominance in search results
4. Pretending that Supplemental Results don’t exist and won’t be treated unfairly in search results
5. Forcing logged in users to rely upon personalized search and search history
6. And someone seems to have spammed Google Maps

Google has done some technologically good things this year, of course. They’ve expanded services through Webmaster Central (but that doesn’t help the search experience); they’ve expanded their custom search technology (but that doesn’t affect their Web search interface); and they’ve joined the growing list of search engines to offer integrated search.

But is integrated search all that it’s cracked up to be? I can no longer find any of the news stories about it but an Indiana school district filed for an injunction against Google’s Universal Search rollout within a matter of days after the May 16 Searchology press conference (where Google announced its Universal Search Solution). Was there a connection between that injunction and what appeared to be a rollback in Universal Search behavior (which, in theory, would have allowed Google time to make some technological tweaks)?

Google is juggling lawsuits over copyright infringement, trademark dispiputes, and privacy issues. They have consistently lost legal battles in the European market. They have had to retrench on YouTube content because of actions taken by industry service groups such as RIAA and some foreign governments.

Google has been fighting a losing battle against search engine spam for several years. The quality control measures have, on more than one occasion, forced millions of non-spamming sites to lose visibility. Google usually responds to the consequential screaming by loosening the ropes with which it has tightly bound its data, but they seem to be determined to enforce Web apartheid more than ever.

Google introduced an early form of Web Apartheid with its Supplemental Index in 2003. But they showed that trust is the foundation of Web Apartheid with their unintentional Sandbox Effect in early 2004. By late 2005 many people who had been sandboxed the previous year understood Google’s trust factors well enough to understand they needed more links: the resulting link glut forced Google to change its strategy.

The Bigdaddy update of December 2005 - March 2006 changed the rules by implementing full Web Apartheid. Only those sites that had an arbitrarily minimal amount of internal PageRank were permitted into the Main Web Index. Every other page was sent to the Supplemental Results Index. As Google recrawled the Web throughout the rest of the year, more pages fell into the Supplemental Results Index for lack of sufficient PageRank.

After Google stopped displaying query data in its cache for Supplemental Results Pages, I compared several test queries with early 2006 results, concluding that Google might have demoted up to 80% of its Web index to the Supplemental Results.

My guesswork may not be flawless, but the numbers of unsearched pages were alarming when compared to the immense numbers of complaints voiced in blogs and forums about sites losing traffic referrals from Google from mid-November 2006 onward. It seems obvious that Google had taken a drastic step in its fight against spam (ignoring the fact that a simple study claims 75% of Google’s Blogspot consists of spam).

But how does demoting a majority of the Web to virtual unindexed status fight spam when the spammers are savvy enough to build out link networks that help the spam pages enter the Main Web Index? The real problem here is that Google is fighting two battles and losing both of them.

The first battle is the massive increase in Web spam that Google itself has fueled through its immensely successful AdSense program. It’s easier than ever for people to earn at least a few dollars a month off of nonsense text compiled from search engine alerts that are auto-posted to blogs. The more money spammers can earn from advertising, the more spam Google will have to deal with.

The second battle is the massive increase in artificial linkage that Google has encouraged by misleading people about the importance of PageRank. Spammers and many SEOs value links chiefly for the anchor text they can use to spam pages to the top of search results, but Google has chosen to ignore the fact that people are riding the anchor text wave.

Now Google has made it almost mandatory for pages to get at least some internal PageRank so that they can rank in search results, thus forcing Webmasters to seek links for precisely the one reason Google says it doesn’t want people to seek links: to acquire PageRank. While this mis-shapen strategy may dilute the amount of targeted anchor text being created in link networks, it will only increase the intensity of linking because people will now act with less constraint.

Whereas yesterday’s SEO advice was to acuire only “relevant links”, now people will be seeking “PageRank links” because that is what they need to get their pages out of the Supplemental Index.

Good move, Google. Maybe you should just hide the “Supplemental Results” label and pretend there is no Web Apartheid, for surely instead of dealing with the problem (your allowing links to pass anchor text) you’ve only made the problem worse not only for yourself but for everyone else, too.

For the first time since 1998, pages now actually need substantial PageRank in order to rank in search results — which all but guarantees that large content sites like Wikipedia will dominate search results despite the fact that they are generally unreliable sources of information.

And that may be why people are starting to turn to other search services. Or it may only be a summertime fluctuation and Google will spring back in July’s search market share reports.

Like I said, it’s too early to be sure of anything, but I needed to write about something and wanted to set the stage for a few “I told you so” type posts down the road if Google continues to practice Web Apartheid.

As a searcher I am tired of not being able to find content that I know is out there.