Can I use robots.txt to optimize Googlebot's crawl?

Go down

Can I use robots.txt to optimize Googlebot's crawl?

Post by NewsOnline on Thu Aug 19, 2010 3:14 am

Can I use robots.txt to optimize Googlebot's crawl?

Can I use robots.txt to optimize Googlebot's crawl? For example, can I disallow all but one section of a site (for a week) to ensure it is crawled, and then revert to a 'normal' robots.txt?

avatar
NewsOnline

Posts : 853
Points : 5137
Reputation : 0
Join date : 2010-08-12

View user profile http://forum.seoflexforum.com

Back to top Go down

Back to top

- Similar topics

 
Permissions in this forum:
You cannot reply to topics in this forum