robots.txt file basic setting | Optimizing All in One SEO Pack Pro | Support | Semper Plugins

Avatar

Lost password?
Advanced Search

— Forum Scope —




— Match —





— Forum Options —





Minimum search word length is 3 characters - maximum search word length is 84 characters

sp_Feed Topic RSS sp_TopicIcon
robots.txt file basic setting
July 11, 2018
12:03 pm
Avatar
Gary Leggett
Member
AIOSEOP
Forum Posts: 12
Member Since:
June 8, 2018
sp_UserOfflineSmall Offline

Hi,

Is there a basic setting for the robots.txt file?  In google webmaster I get alot of errors concerning the robots.txt file with different sites concerning the “no index” error.

For example in one site’s robot.txt file there is this line among many:

Disallow: /wp-content   

seems like I would want google bots in the content section of a site?  So this site seems to need just a basic robots.txt file.

—-

In another site this is all I have in the robots.txt file

User-agent: *
Disallow: /wp-admin/
Allow: /wp-admin/admin-ajax.php

Sitemap: http://aidomes.com/sitemap.xml.gz

—————-

and this site has 51 errors concerning mostly concerning  /category

an example:

http://aidomes.com/category/s6…..us/page/2/

google webmaster says this:  No: ‘noindex’ detected in ‘robots’ meta tag

Is this in the above robots.txt file displayed or is there a setting I’m missing someplace else?

Thanks

July 11, 2018
12:20 pm
Avatar
Steve Mortiboy
Admin
Forum Posts: 1896
Member Since:
January 5, 2012
sp_UserOfflineSmall Offline

** You currently do not have permission to see this post. To obtain access to these Premium Support Forums, please purchase a plugin and support subscription from Semper Plugins **

Forum Timezone: America/New_York

Most Users Ever Online: 964

Currently Online: info@jmgvirtualconsulting.com
44 Guest(s)

Currently Browsing this Page:
1 Guest(s)

Top Posters:

Member Stats:

Guest Posters: 35

Members: 20146

Moderators: 0

Admins: 8

Forum Stats:

Groups: 4

Forums: 28

Topics: 1914

Posts: 8169

Newest Members:

Julio Escamilla

Administrators: Michael TorbertPeter BayliesSupport TeamSteve MortiboyMatthew RobinsonSteve KlasenArnaud BroesAshish Ravi

[i]
[i]
Skip to toolbar