Robots TXT File
The purpose of the ROBOTS.TXT file is to enable you to control which parts of your website can be browsed by search engine bots.
In older versions of Joomla! (3.3.0 and earlier) the ROBOTS.TXT file prevented search bots from accessing the following three directories:
- images,
- media,
- templates.
The ROBOTS.TXT file included the following lines of text to prevent access by search bots:
- Disallow: /images/
- Disallow: /media/
- Disallow: /templates/
TIP: Remove the above lines from your website's ROBOTS.TXT
Removal will enable search bots to:
- list your media content in their search results,
- verify that your website is mobile-friendly.
Reason to Check
The above three lines were not included with future releases of Joomla! CMS.
Unfortunately, it is not possible for Joomla! Updates to over-write an existing ROBOTS.TXT file.
You have make the above changes yourself via the File Manager screen of your Hosting Control Panel.
How to change the ROBOTS.TXT file
Sign into your Hosting Control Panel and open the File Manager screen to edit the ROBOTS.TXT file.
XML Site Maps
You can also add references to your website's SITE MAP or SITE MAPS to help search bots find your website content.
