Can I use robots.txt to optimize Googlebot's crawl?

View previous topic View next topic Go down

Can I use robots.txt to optimize Googlebot's crawl?

Post by NewsOnline on Thu Aug 19, 2010 3:14 am

Can I use robots.txt to optimize Googlebot's crawl?

Can I use robots.txt to optimize Googlebot's crawl? For example, can I disallow all but one section of a site (for a week) to ensure it is crawled, and then revert to a 'normal' robots.txt?

avatar
NewsOnline

Posts : 853
Points : 4868
Reputation : 0
Join date : 2010-08-12

View user profile http://forum.seoflexforum.com

Back to top Go down

View previous topic View next topic Back to top

- Similar topics

 
Permissions in this forum:
You cannot reply to topics in this forum