It is a text file that the search engines' spider use when they visit the websites. In it you can indicate which parts of your site are indexable and which robots can access them. It is not necessary for sites, that is, you do not have to have it. More information:

This dictionary contains 3.259 definitions related to the Internet world.
If any definition is not included Tell us, we will be very grateful. 😉

© Copyright IndianWebs 2018 | Legal notice | Clients Area