SOLVED RT Resources still blocked after robot.txt fix
-
-
- Ana Lázaro
- Jr. Rocketeer
- Posts: 24
- Thanks: 0
-
hello, i am running 3.6 joomla and php7
the site has the myriad template and many RT resources (includind rokbuster) that are beeing blocket by googleboot.
in my txt text a made this changes, but template, modules css, js are still being blocked as you can see the the attached image. well, i am not a developer and i ask for your help.
this is my robots.txt
User-agent: Googlebot
Allow: www.pet-academy.com.pt/sitemap.xml
Allow: *.css
Allow: *.js
Disallow: /administrator/
Disallow: /cache/
Disallow: /cli/
Disallow: /components/
Allow: /components/com_roksprocket/
Allow: /components/com_rokgallery/
Allow: /images/
Disallow: /includes/
Disallow: /installation/
Disallow: /language/
Disallow: /libraries/
Disallow: /logs/
Disallow: /media/
Disallow: /modules/
Allow: /modules/mod_facebook_slide_likebox/
Disallow: /plugins/
Allow: /plugins/system/rokbox/
Disallow: /templates/
Allow: /templates/rt_myriad/
Disallow: /tmp/
i aprecciate your attention
Ana
-
-
-
- MrT
- Preeminent Rocketeer
- Posts: 101084
- Thanks: 13484
- Web Designer/Developer
-
This is because google is not "seeing" your site as end user would - if you look at the image of the site they show you you will see it doesn't look right - this is because googlebot has been prevented from accessing certain directories because of the content in your robots.txt file in your joomla root directory. You will need to comment out the lines that currently specify "disallow" on cache,components,libraries,modules and plugins (add a # at the start of the line). You could also make this change specifically for the googlebots too.
This is what I use:
User-agent: * Disallow: /administrator/ Disallow: /bin/ #Disallow: /cache/ Disallow: /cli/ #Disallow: /components/ Disallow: /includes/ Disallow: /installation/ Disallow: /language/ Disallow: /layouts/ #Disallow: /libraries/ Disallow: /logs/ #Disallow: /modules/ #Disallow: /plugins/ #Disallow: /templates/ Disallow: /tmp/
Regards, Mark. - Please search forums before posting. Please make sure your post includes the version of the CMS you are using and a link to the problem. Annotations on screenshots can also be helpful to explain problems/goals. Please use the "secure" tab for confidential information.
-
-
-
- Ana Lázaro
- Jr. Rocketeer
- Posts: 24
- Thanks: 0
-
Mark, thank you for your answer.
i have applied your suggestion. Still, the both cs and js in template other RT modules and plugins (and even gantry) are still being blocked
example
www.sitept/components/com_roksprocket/la.../assets/js/strips.js
should i write allow do every single url that is being blocked?
like
allow:/templates/rt_myriad/css....
Honestly i have tried to do it in many ways. Can it be the .htacess?
well, if you have any other suggestion i apreciate your help.
Thanks!
Ana
-
-
-
- MrT
- Preeminent Rocketeer
- Posts: 101084
- Thanks: 13484
- Web Designer/Developer
-
In the robots.txt I gave you I commented out the disallow of /components therefore it should be allowed - so why you are blocking that still is puzzling.
Please would you post your URL, superuser id and pswd in the secure tab of your post and I'll have a look for you.
Please post your FTP logon, password and FTP URL in the secure area of your post.
You can't do any of this in .htaccess that's a different thing - it's robots.txt you need to be changing.
Regards, Mark. - Please search forums before posting. Please make sure your post includes the version of the CMS you are using and a link to the problem. Annotations on screenshots can also be helpful to explain problems/goals. Please use the "secure" tab for confidential information.
-
-
-
- Ana Lázaro
- Jr. Rocketeer
- Posts: 24
- Thanks: 0
-
-
-
- MrT
- Preeminent Rocketeer
- Posts: 101084
- Thanks: 13484
- Web Designer/Developer
-
MrT wrote:
Please post your FTP logon, password and FTP URL in the secure area of your post.
You didn't provide ftp login as I requested?
Regards, Mark. - Please search forums before posting. Please make sure your post includes the version of the CMS you are using and a link to the problem. Annotations on screenshots can also be helpful to explain problems/goals. Please use the "secure" tab for confidential information.
-
-
-
- Ana Lázaro
- Jr. Rocketeer
- Posts: 24
- Thanks: 0
-
-
-
- MrT
- Preeminent Rocketeer
- Posts: 101084
- Thanks: 13484
- Web Designer/Developer
-
Ok something else is going on here because I changed the robots.txt to allow everything and still resources are being blocked.
Are you using a CDN? if so, turn it off and clear it's cache. Possibly "cloudflare"?
After that you really need to ask your host about this as something else is blocking access to those resource by googlebot (most likely a server setting higher up in your hosting).
Regards, Mark. - Please search forums before posting. Please make sure your post includes the version of the CMS you are using and a link to the problem. Annotations on screenshots can also be helpful to explain problems/goals. Please use the "secure" tab for confidential information.
-
-
-
- Ana Lázaro
- Jr. Rocketeer
- Posts: 24
- Thanks: 0
-
thank you Mark for you tip
All the best,
Ana
-
Time to create page: 0.059 seconds