Dealing with Crawlers - SEO starter


This post is about dealing with crawlers namely web-bots orelse search engines, this is Part IV of V linked to previous post SEO starter.



#Make efective use of robots.txt :

>Restrict crawling where it's not needed with robots.txt, A "robots.txt" file tells search engines whether they can access and therefore crawl parts of your site.There are a handful of other ways to prevent content appearing in search results, such as adding "NOINDEX" to your robots meta tag, using .htaccess to password protect directories.

>Use more secure methods for sensitive content, and Avoid allowing URLs created as a result of proxy services to be crawled.

#Be aware of rel="nofollow" for links :

>Fight comment spam with "nofollow", Setting the value of the "rel" attribute of a link to "nofollow" will tell Google that certain links on
your site shouldn't be followed or pass your page's reputation to the pages linked to.

>Make sure you have solid measures in place to deal with comment spam!

>Linking to sites that Google considers spammy can affect the reputation of your own site.

Next is Part V which is Promotion and Analysis.



Receive all updates via Facebook. Just Click the Like Button Below

You can also receive Free Email Updates:

Powered By SlashBall Networks

Random Posts

Alexa