• Register
• Log in

SEO Theory - SEO Theory and Analysis Blog

Algorithm analysis, Web community relationship analysis, SEO practices and techniques, industry news, etc.

• Home
• About SEO Theory
• SEO Glossary

Google broke the URL reference query

Posted by Michael Martinez on July 19, 2007 in Competitive Analysis

Google broke the “URL reference” query functionality. This was actually a technique recommended by Google some time back (maybe 2-3 years ago) when people complained about the “broken” link: query operator (which was never broken — Google has always shown only a random sampling of backlinks).

Up until recenty, perhaps when Webmaster Central Tools started offering backlink reports, you’d be able to look at other sites’ URL references in Google’s search results. Until Google changed the behavior of this query you’d see both explicit references to the URL in on-page text and you would see links.

Now the URL query only returns pages that explicitly mention the URL in indexable text. If a Googler were to stop by and comment, I’m sure they would make a reasonable argument that “you really only want to see URLs that are part of the indexable body text with that query” blah, blah, blah.

And of course that isn’t true. SEOs have been using the URL reference query to look for possible backlinks because they reported far more data than the link: query operator. Not that knowing how many links a competitive site has really tells you anything useful for search engine optimization (it doesn’t) but knowing when and where a competitor gets new links — that’s useful information.

More importantly, when people come looking for help an SEO can do a quick search for URL references and see what the upper limit of inbound links might be. Many a site operator has complained about not having inbound links only to be told (often by me) that, yes, they did indeed have several hundred links.

I know the query used to work as Google recommended because I spot check the results. I never really trusted them to leave this powerful a competitive tool in SEOs’ hands. So I’ve been waiting for some time for Google fix what seemed like a broken query function. Instead it looks more like they have permanently redefined how that query works.

Which only puts the SEO community in a bad position because doing a proper analysis for new prospects’ sites has become impossible. You cannot go to Yahoo! to look for the links that Google knows about. Many SEOs turn to Yahoo!’s Site Explorer to do their link research but they are wasting their time not only because Yahoo! doesn’t tell you anything useful about Google’s database, Site Explorer doesn’t provide accurate information about what is in Yahoo!’s database.

Before Yahoo! began directing site queries to Site Explorer it was possible to compare those queries to Site Explorer’s results, and Site Explorer always reported different results. Now, maybe Site Explorer is more accurate than Yahoo!’s old site: query operator but Site Explorer is just not very accurate at all.

Among other failings of Site Explorer, it reports non-existant links, such as links scraped from Javascript ads. Many ancient dead pages also float around in Yahoo!’s Site Explorer. Do those links really count? They certainly won’t count in Google, I can tell you, as Google doesn’t show those non-existant pages in its results.

Of course, maybe some really savvy SEO sales people can persuade prospects to open up their Google Webmaster Central logins so the SEOs can get in there and look at the link reports (or, better yet, just talk the prospects through downloading the data). But the problem is that just because you see a backlink [in Webmaster Central's link report doesn't mean] that it’s carrying weight.

So it really does you no good to look at Webmaster Central link reports, except to see when links appear in Google’s database — but they only update that report once or twice a month. Twice a month would be okay, except that they do it for every site at the same time (or so it seems). So most of the time you don’t know what links have appeared from the latest crawls.

This is probably a good thing in Google’s view because they want SEOs to see everyone’s link data. Yahoo! probably feels the same way, which for all we know is why they report bogus link data.

So you have to ask yourself: If you cannot get good link data from Google and you cannot get good link data from Yahoo!, where can you get it from?

The answer is not a happy one for our poor link chasers: you cannot get reliable link data.

I’ve been saying for years that you have to use each search engine to report on what it knows about links, because Yahoo! just doesn’t know anything about the other search engine’s link data. And Yahoo!’s overlap in Web coverage with the other search engines’ crawling tends to run to about 50-60% at best. So you don’t know which links count in Yahoo!, which links Yahoo! reports that are real, and which links Yahoo! reports that the other search engines know about.

Nor does Yahoo! report the links that other search engines know about (which Yahoo! doesn’t include in its own database). On any given day, as you drool over those 100,000+ links Yahoo! tells you about, you’re probably only seeing 1/4 of the real links for a site and a whole lot more non-existing or transient (rotating Javascript) links.

Good job, Yahoo!.

But it’s really Google that should be congratulated. Perhaps they inspired Yahoo! to confuse SEOs with bogus link data. Perhaps Yahoo! realized that SEOs who can’t see the difference between Yahoo! and Google were beating Yahoo! to death with pointless, misinformative queries for “competitive analysis”. Maybe Yahoo! decided to laugh at the SEO community for wasting its time with “backlink research”.

Have a good laugh, guys, but you can easily put a stop to this nonsense by discontinuing the practice of allowing links to pass anchor text. Would that hurt me? Sure. But it would hurt a whole lot of other people even more.

In the meantime, those SEOs who understand that there is no reliable method to look at backlinks are leagues ahead of the Yahoo! checking crowd.

Which group of SEOs do you include yourselves in?