Use ‘What Googlebot Sees’ to Find Scrapers :: Daily Search Engine Optimizations News

March 20, 2008

Posted by Brent D. Payne

Most of you are aware I recently took a position at Tribune as their in-house SEO manager.

One of the first things I did was get a current state of the union. After working through the issues of verification for the dozens of sites (another post, perhaps), I realized that ‘What Googlebot Sees’ for a handful of the domains is ‘Privacy Policy’. Now, this is obviously a red flag for us to add a ‘nofollow’ tag on the link (that’s perhaps another post also), but in addition to that, I found something else.

What’s that? What did I find?

Scrapers . . . lots of them. You’d think they’d take the time to add a line of code to their bot to remove such a common piece of code as ‘Privacy Policy’, but . . . nope.

Take a peek at your Webmaster Central console under the ‘What Google Sees’ section. Not only will it highlight what you should ‘nofollow’ but, if you dig a little deeper, you may find some scrapers. What you do next is your decision. Options? Take a legal stance, allow them to do it until you gain some more inbound links, reach out to them with an RSS feed (or other linking option), or do something a bit more blackhat, like making your ‘Privacy Policy’ type links an image with an alt tag for the targeted keyphrase. ;-) I’m not going to share what we’re doing (though it is not the latter), as that would lead to proprietary information; besides, that type of decision is going to be pretty site-specific, anyway.

P.S. They turned the river green on St. Patrick’s Day in Chicago. Interesting, but not sure if it was worth standing in the cold for 45 minutes with my 4-year old son. ;-)

Brent D. Payne

SEO Manager

Tribune Interactive

Do you like this post? Yes No

More SEO News…. (CLICK HERE)
Tags:

Please be sure to visit our new Social Blog Network, as well as our Social Bookmark Site. Both offer services for free!
Get you latest SEO news fix at AutoPrimeMedia.com!

Search Engine Optimization is not an exact science. It takes lot’s of work and research. Trial and Error. Read through our posts here and try to learn from our experience. We offer some of our insight… news and comment. Please feel free to share your thoughts, and ideas. The site does allow “follows” so post your links.

Posted by admin | Filed Under SEO News 

Comments

Got something to say?