Robots.txt is a file that exists on the root Listing of each Web site and may be used to instruct search engines like yahoo on which directories/information of the web site they're able to crawl and involve inside their index. In line with Rubright, a greater solution is hand-buying your https://news.rafeeg.ae