How SEOs should and should not use Webmaster Central

Posted by Michael Martinez on March 17, 2008 in Search Engine Optimization

I rarely comment on specific SEO tools as it’s pretty well known that I consider nearly all of them to be worthless, or at least worth less than the knowledge of how to structure informative queries is worth.

Google’s Webmaster Central Tools have received a lot of attention and as I have studied blog and forum evaluations over the past few days I have realized that a lot of people are now using Webmaster Central as an SEO tool. That’s a bad thing because, frankly, Webmaster Central’s data sucks.

Now, I understand a lot of thought and hard word have gone into Webmaster Central. I don’t want to take anything away from the engineers who brought the service to the Web. I’d rather have it than not have it, but there are serious deficiencies in Webmaster Central’s reporting tools that make it almost completely worthless for search engine optimization.

Let’s a take a look at a sample of the issues:

  1. The link data never matches what the Web Search interface shows
  2. The indexing data never matches what the Web Search interface shows
  3. The verification procedures are inconsistent and don’t always work
  4. There is no way to use the reporting tools to determine which of your pages are in the Supplemental Results Index

There are some positive things I can say about the service, too. For example, I’ve recently noticed that the XML sitemaps parsing seems to be more forgiving of injections over which Web site operators have no control. I’m also glad for the capability to designate geographic targets.

And though the user interface could use some work that’s really not bound up with how you should be using Webmaster Central for search engine optimization. I’ve found only limited usefulness in the service but I would recommend to anyone that they do use it for:

  1. Resolving URL canonicalization (although this can be problematic for sub-domains if the main domain is not set up to resolve sub-domains with “www.” inserted in front of them)
  2. Submitting XML sitemaps
  3. Checking what Google sees in your robots.txt (sometimes it just helps to look at it through a different interface)
  4. Setting geographic targets (if applicable)

If you try to use Webmaster Central for anything else related to search engine optimization you’re wasting your time. Here is why:

Link data - It’s not just that the link data is only updated on an infrequent basis. Google admitted up front that it would NOT report all available link data, and despite some claims of improvement in link data coverage I can pretty much find random examples of links (passing anchor text in Google’s search resuts) that are not reported by the tool.

Furthermore, Google admitted that it would not distinguish between Suppelemental Results Index links and Main Web Index links in the reports so even if they were showing us all their link data, it wouldn’t matter. We’d have no way of knowing which links are being counted. So why bother showing us the link data at all? It’s a waste of Google’s resources and SEOs’ time.

But the timeliness (or lack thereof) of the link data is a real deal-killer. It does an optimizer absolutely no good to see linking data that is anywhere from 2 weeks to 1 month old. We need to know if those links are real. As the tool works right now, the data is reporting bogus results (and remember that Yahoo!’s Site Explorer reports bogus results, so there are no good link reporting tools).

Indexing data - You have to dig to find any real information about indexing in Google’s Webmaster Central. It’s there, what little there is, but it’s almost useless. The worst part of this aspect of the service is that it offers a perfectly useless table filled with links to Google queries. Well, most of the SEO industry may not know those queries are broken, but I do. I’d like to know the REAL story about what is indexed.

For example, have you compared the results of your site: query to an inurl: query? Depending on your domain name you may have to modify the inurl: query so that it only shows pages from your site (hint: DON’T modify it with the site: query operator). You’ll find that most domains probably don’t get the same results. Don’t assume that simple explanations will clear up the ambiguities. I’ve looked for the simple answers and they don’t fit the data.

If you look at your sitemaps report, you’ll see that Google tells you a certain number of URLs from the sitemaps are indexed. This is about as honest as the data gets. You cannot use the linking report to see what’s indexed because that is pretty much bogus data (see above).

Verification process - I love verifying Web sites, especially since I can now place an XML sitemap for one verified site on another. Of course, I don’t like having to do that and maybe Google can update its handling of XML sitemaps enough so that we don’t have to cross-map our sites.

But if you’re verifying a sub-domain on a server whose Web admin has not set it up so that www.sub.example.com resolves to sub.example.com Google chokes when you try to set your canonical URLs for the sub-domain. Yes, you can talk to the Webmaster and say, “Hey, you need to accomodate Google on this little issue” but you might hear a tirade of profanity over how stupid Google is.

Look, I don’t run the search engine, I just manipulate its results.

In any event, not every sub-domain can be canonicalized through Webmaster Central and that’s a serious drawback to the service because not every person out there buys whole domains and gets leased servers. Hosting services run their shops their own ways and it would be nice if Google were a little more flexible in this area.

In fact, because of the error messages I have encountered when verifying sub-domains, I know that Google is using at least two different authentication processes. You can verify the site so that you can see the reports but still not be able to verify the site so that you can set the canonicalization preferences. That’s so uncool.

Supplemental Results Index - This would be such a non-issue if Google would only:

  1. Index every word on a Supplemental Results page
  2. Allow Supplemental Results pages to pass anchor text the same way Main Web Index pages do (or stop allowing ALL pages to pass anchor text)
  3. Allow Supplemental Results Index pages to appear FIRST in the search results (ahead of Main Web Index pages) if the Supplemental pages really are more relevant than the Main Web Index pages

However, Google wants to make life difficult for searchers and providers alike so we have no reliable means of determining which of our pages are in the Supplemental Results Index. If all you need for your Supplemental pages is more links, what’s the problem with telling people which pages are Supplemental? The whole “the tags were confusing people in the search results” argument is completely irrelevant to what we see in Google Webmaster Tools.

It’s time for Google to show us a REAL indexing report (allowing for the fact there are different data centers — let us just pick one as our “authoritative data center” for crying out loud) that tells us:

  1. Which pages are in the Main Web Index
  2. Which pages are in the Supplemental Index
  3. When each page was last crawled
  4. When each page was last cached
  5. Which pages have links passing anchor text to them
  6. Which pages are passing link anchor text to other pages

That would be more helpful than some random sampling of outdated linking data that doesn’t reflect where the links currently are. That would also be more helpful than jumbling up all the Main Web and Supplemental Index pages together, too.

But I won’t hold my breath since, as most SEOs (who share their opinions openly) are pretty much clueless about the usefulness of Webmaster Central, it’s highly unlikely that Google will turn it into a really useful resource any time soon.

Oh, and I’m sick and tired of seeing people promote Google’s date search function. You know, where you supposedly use a time frame like the last 30 days to see which of your pages have been indexed? PUH-leeze! That function don’t work, has never worked, and probably never will work.

If you’re basing your analysis of Google’s knowledge of your site and links on the rantings of SEO pundits who tell you to use Google’s date search, Yahoo! Site Explorer, or Webmaster Central’s link data, you’re in a heap’o'trouble. (And don’t even get me started on using Blog Search for linking data — holy mackeral!)

8 Comments on How SEOs should and should not use Webmaster Central

By wibbler on March 17, 2008 at 1:59 pm

Hi Michael,

I realise that there are short falls in the data provided by google and yahoo regarding showing backlinks to a given page. I also “suspect” that this is because they dont want to give away the full link profile of a given page - fair enough.

However given that there are no other sources of finding out the backlinks for a given page, dont you think that using yahoo is better than nothing?

Can it really lead me up the garden path using yahoo to count the links a competitor has - at least on that engine?

Cheers
Wibbler.

By Michael Martinez on March 17, 2008 at 4:19 pm

Wibbler: “However given that there are no other sources of finding out the backlinks for a given page, dont you think that using yahoo is better than nothing?”

Michael: This reminds me of a scene in “Star Wars: The Empire Strikes Back”. Yoda has just finished teaching Luke about how to use the Force to lift heavy objects. Luke tries to raise his X-wing fighter out of the swamp into which it has sunk and he gives up. Yoda sighs heavily, bows his head, concentrates, and lifts the X-wing almost effortlessly and places it on dry land.

“I don’t believe it!” Luke says in awe.

“That is why you fail,” Yoda replies.

So, my friend, you like so many other people in our industry seem to NOT believe there are ways to find and track links. That is why you fail.

Wibbler: “Can it really lead me up the garden path using yahoo to count the links a competitor has - at least on that engine?”

Michael: It depends on what Yahoo! reports for backlinks. If you can be reasonably certain that the links are not coming from bogus pages (indexed because Yahoo! tests for proper 404 error handling and instead finds the classic redirect trick), that they are not coming from Javascript (on-site navigation, Google AdSense, banner ads, etc.), and that they are not coming from dead pages (taken down but still listed in Yahoo!’s index) AND if you can sort the links according to sourcing domain (since Yahoo! claims it gives the most weight to the “first” link from each domain) AND if you can determine whether the links are passing value in Yahoo! (Yahoo! does impose its own penalties and filters, after all) — well, if you can do all that, then you should be able to get some good link analysis (for Yahoo!’s database) out of Yahoo!’s backlink reports.

While I just made it sound horrendously impossible to do a good analysis, a link-rich site (10s of thousands of backlinks) or a link-poor site (few to no links at all) should be easy to analyze in Yahoo!. It’s all the rest of the sites where you have to roll up your sleeves and slog through the data that are the problem. Most of the sites that SEOs compete with fall into that middle category.

There is no easy way to count all of a competitor’s backlinks, but neither is there any SEO advantage to knowing how many there are (or where they are). Each search engine handles the links in its own way, and each search engine has its own idea of which links are there.

I think there are influencer links that help with your optimization better than other links. Most SEOs seem to agree that some links help more than others. But none of us really has a good (public) method for identifying and counting those links. I do feel it would be better to use a search engine-independent metric for link analysis than to rely on any one search engine.

By wibbler on March 18, 2008 at 4:41 am

Ok

“So, my friend, you like so many other people in our industry seem to NOT believe there are ways to find and track links. That is why you fail.”

Well (Yoda) :) , thats not entirely true - I have a “vision” of a database of links and urls which someone somewhere must have created - an entire webmap. No cached pages which would support text search, just a link map.

If such a map exists (it will but I dont know where or how I would get access to it) then I would also “require” it to contain information such as when each link to a given page was found (timeline), what the anchor text is, what the target pages (the page receiving the inbound link) title contains etc etc etc - my list is around 60 items long.

I see a number of possibilities -
a) someone has a crawler which is storing this data somewhere (it would be extremely valuable to SEOs I am sure) - but the data would not of course represent any particular picture held within any given SE.

b) there are search queries other than “LINK:” commands I can use to locate the links held in a given SE (which would naturally provide results aligned to the SE in question (assuming there was no similar filter against the results such as there are restrictions in place against the link: command)

c) Yoda has another answer :)

I have noticed that you have often referred to there being no viable software present which performs analysis of SE results or metrics.

If you provide me with what you would like to see, I can develop a generic system to crawl sites in a serp for any number of phrases and from any SE - and if you can show me how to establish which pages link to those sites within a give serp, I can provide any number of statistics regarding those linking pages and their content.

I would anticipate such a crawler and statistic building program to be around 20 to 30 thousand lines of code - but if you provide a spec - I can do it for free. My benefit would be to gain SEO experience.

Having said all this - humans make up most of the SERPS now - Im certain - therefore I do suspect that all the stats would represent is what a typical site reviewer would “like to see” in terms of onpage factors and “glam”. With this in mind the answer could be just do nice looking content rich sites and forget the SEs and the links.

Dunno

Cheers
Wibbler.

By Michael Martinez on March 18, 2008 at 8:13 am

Wibbler: “If you provide me with what you would like to see, I can develop a generic system to crawl sites…”

Michael: Alas! Any such design specifications which I might devise contractually belong to my employer via an intellectual property rights clause in my contract.

Which is why I am sometimes frustratingly obscure and ambiguous on this blog. But that is neither here nor there.

There are already tools available to SEOs (WebCEO being one that I know of, for example) that will crawl and index the linking pages you know about. It’s the linking pages you don’t know about that these tools cannot track for you. Search engine alerts pretty much suck (I stopped using them after seeing the same “new” sites show up in my email 5 or 6 times in a row).

But, again, you don’t need to know THAT much about links. After all, most of search engine optimization really has nothing to do with links. Links are just one tool in the arsenal and the fact that so many people rely more on links than on common sense doesn’t mean you’re at a disadvantage if you don’t know where your competitors are getting their links from.

In fact, being in the dark about their link profiles HELPS you because you’ll be less likely to copy their links (which is not going to give you a competitive advantage). 10 good links can help you more than 10,000 crummy links. Instead of trying to analyze competitive link profiles, just focus on building visibility for your site in ways that makes search engines less important in your overall marketing plan.

Link envy really is the SEO’s worst conceptual failing.

By Justin-Goldberg on March 18, 2008 at 9:23 am

Hey, Michael, I posted a very interesting blog here, concerning a mathematical paper entitled “Maximizing PageRank via outlinks”. Also the PDF has been uploaded to scribd. I was going to post it on spider food but it’s posting there is disabled.

By wibbler on March 18, 2008 at 10:51 am

Thanks for the reply,
“Alas! Any such design specifications which I might devise contractually belong to my employer via an intellectual property rights clause in my contract.”

Let your employer have a copy of the software then for free - I wouldnt bother about that - why not put it to them?

Ok - assuming theres not really much to worry about with link profiles, the crawler could more specifically gather stats on ranking pages - and onsite factors of ranking pages for comparison?

Items such as word counts, page sizes, LSI/related word factors, positioning of words and surrounding words, images, naming conventions, who ranking sites link to and whats on those sites, affiliate links presence, titles, bold, italics, alt tags, Hx tags. These could all be totted up and compared for ranking pages. Once all these are done - the real nitty gritty could be addressed.

Even if you cant be in on this tool - would it be worth me even going ahead with it anyway in your opinion. In many respects I would always prefer to write my own software for this kind of project - I would not have to worry about things such as phoning home this way.

Ok - re link envy - I dont envy links, in reality, I am just trying to grasp their importance, I think I am aware that they are not as important as they used to be interms of helping a page gain ranking position. Infact I read items like “you only need a few links” quite a lot these days.

All thoughts appreciated
Cheers
Wibbler.

By Michael Martinez on March 18, 2008 at 11:41 am

Justin, sorry about Spider-food. It’s on hiatus while J.K. and I get the transfer worked out (which itself has been on hiatus because of my spam problems).

Wibbler: “Ok - assuming theres not really much to worry about with link profiles, the crawler could more specifically gather stats on ranking pages - and onsite factors of ranking pages for comparison?”

Michael: I very much appreciate your passion for the topic but I’m afraid that I can only withdraw into the employer-stipulated corner of “no content” at this point. I wish I could say more on the issue of “software for SEOs”.

Wibbler: “Even if you cant be in on this tool - would it be worth me even going ahead with it anyway in your opinion.”

Michael: In my opinion, there is always room for a better tool, especially given the fact that there are relatively few good ones out there.

Wibbler: “Ok - re link envy - I dont envy links, in reality, I am just trying to grasp their importance,…”

Michael: Their importance varies from query to query. Most search results are not decided by links, but try getting the SEO community in general to agree to that without hemming and hawing. They manage a relatively small number of query spaces and yet they base their sweeping generalizations upon a very small, extremely biased, non-representative data set.

In some queries, you DO need a lot of links. But after you list about 5 or 6 verticals people start arguing that you should include “this query too” and “that query over there” — largely because they’ve been looking at their competitors’ link profiles on Yahoo!.

So what if the other guy has 50,000 inbound links in Yahoo!? Are they all passing anchor text that is relevant to whatever query you’re chasing? If not, then his links really don’t explain why he’s kicking your butt in SERPs.

Getting links is not a problem for me. I can get thousands of links every day if I want to (I don’t). Determining whether I need to use links — THAT’s the essence of real search engine optimization. That’s what separates the good SEOs from the SEO bloggers who can’t think of anything useful to say (and I have to admit that some days I struggle to come up with anything interesting at all).

You don’t have to be interviewed by the news media to be good at SEO. You just have to get results in the right places. But if you think yours is the best way to do things, you have a lot to learn about search engine optimization (and I’m using the figurative “you”, not referring to you specifically, Wibbler). I don’t marry myself to any one particular optimization strategy. I can’t afford to.

By wibbler on March 18, 2008 at 2:06 pm

“But if you think yours is the best way to do things, you have a lot to learn about search engine optimization”

Understood - my motto is “when you think you have the ultimate grasp in “it” - think again.”

Comment

Log in or Register to post a comment.

More

Read more posts by Michael Martinez

About the Author

Michael Martinez is the Director of Search Strategies for Visible Technologies, Inc. A former moderator at SEO forums such as JimWorld an Spider-food, Michael has been active in search engine optimization since 1998 and Web site design and promotion since 1996. Michael was a regular contributor to Suite101 (1998-2003) and SEOmoz (2006).

Any help for an odd qmail spam exploit? Three things that are changing today’s search environment