Robots.txt file

View previous topic View next topic Go down

Re: Robots.txt file

Post by markcooper006 on Thu Feb 16, 2012 1:30 am

hi there jessica, well this depends on the platform your are using.
if you not want a particular page to be indexed in google then instead of modifying robots.txt try a noindex tag at header. a basic robots.txt should also be there at root all the time.



__________________________
Tier 5

markcooper006

Posts : 12
Points : 2092
Reputation : 0
Join date : 2012-02-08

View user profile

Back to top Go down

Re: Robots.txt file

Post by jackshobs on Thu Jun 21, 2012 4:18 am

Robots.txt file is a simple text file that must be placed in root directory of a website. The robots.txt file is useful in keeping your spiders from accessing parts folders and files in your hosting directory that are totally unrelated to your actual web site content.

jackshobs

Posts : 30
Points : 1980
Reputation : 0
Join date : 2012-06-15
Age : 29
Location : India

View user profile http://www.rtmlguru.com

Back to top Go down

Re:

Post by NileLars on Thu Oct 11, 2012 7:04 am

Robots.txt are really important for make your page private. Robots file are very important to add for every domain. You must add this file to your website.

NileLars

Posts : 15
Points : 1847
Reputation : 0
Join date : 2012-10-11
Age : 31
Location : Chicago

View user profile http://mobile.softwebsolutions.com/

Back to top Go down

robots file

Post by expertSEO on Mon Nov 05, 2012 1:01 pm

Hi

This file helps site owner in restricting the pages he/she does not want to crawl or index for neither as a page or update.It is done for search engines.

Thanks
ESTI

expertSEO

Posts : 10
Points : 1817
Reputation : 0
Join date : 2012-11-05

View user profile

Back to top Go down

Robots.txt

Post by Seo gurgaon on Tue Mar 19, 2013 4:17 am

Robots.txt files are used to give an indication or command to Search Engine Robots that the site or a webpage is to follow or not to follow; index or not to index.
avatar
Seo gurgaon

Posts : 76
Points : 1761
Reputation : 0
Join date : 2013-03-11
Age : 25
Location : Gurgaon, HR, India

View user profile http://www.seogurgaon.com/

Back to top Go down

Re: Robots.txt file

Post by RafaelJames on Mon May 20, 2013 11:15 pm

Robot.txt file is used in on page SEO to indicate Google spiders that which web page is to visit and which page not to visit. The pages which are restricted to be visited cannot be indexed by Google.

RafaelJames

Posts : 80
Points : 1858
Reputation : 0
Join date : 2012-12-08

View user profile

Back to top Go down

Re: Robots.txt file

Post by jesicafrancis on Mon Dec 09, 2013 12:15 am

Dear all,
The Robot Exclusion Standard, otherwise called the Robots Exclusion Protocol or robots.txt convention, is a gathering to exhorting chipping in web crawlers and other web robots about gaining entrance to all or part of a site which is generally openly perceptible. Robots are regularly utilized via web indexes to arrange and document sites, or by webmasters to edit source code.

jesicafrancis

Posts : 13
Points : 1430
Reputation : 0
Join date : 2013-12-02
Age : 29
Location : Chennai, India

View user profile http://www.agriya.com/

Back to top Go down

Re: Robots.txt file

Post by kumar29 on Mon Apr 14, 2014 7:55 am

The Robots.txt File lets you control what search engine spiders should and shouldn’t crawl on your website. It’s a file that many WordPress users forget to add. For those who do, it can be unclear what folders should and shouldn’t be blocked.

kumar29

Posts : 24
Points : 1756
Reputation : 0
Join date : 2013-01-25
Age : 29
Location : India

View user profile http://www.pulsegym.in

Back to top Go down

Re: Robots.txt file

Post by JuanaLott on Tue Apr 29, 2014 1:59 am

Robot.txt file is helpful for you if you want to hide any contents from search engines. Like any personal data you have on your website, but you don't want to show them another world the use robot.txt. Mention your data on reboot.txt then searches engine will not crawl it.

JuanaLott

Posts : 6
Points : 1273
Reputation : 0
Join date : 2014-04-29

View user profile http://www.greencircleagency.com/

Back to top Go down

Re: Robots.txt file

Post by Sponsored content


Sponsored content


Back to top Go down

View previous topic View next topic Back to top

- Similar topics

 
Permissions in this forum:
You cannot reply to topics in this forum