.Google.com's John Mueller responded to an inquiry about why Google indexes webpages that are refused coming from creeping by robots.txt and why the it's risk-free to neglect the relevant Explore Console reports regarding those crawls.Robot Visitor Traffic To Query Parameter URLs.The individual talking to the question documented that robots were actually generating web links to non-existent query parameter URLs (? q= xyz) to webpages with noindex meta tags that are additionally blocked out in robots.txt. What triggered the inquiry is actually that Google.com is creeping the links to those pages, obtaining obstructed by robots.txt (without watching a noindex robotics meta tag) at that point getting turned up in Google Look Console as "Indexed, though blocked by robots.txt.".The person talked to the following concern:." Yet right here's the major question: why would Google index pages when they can't also see the web content? What's the perk during that?".Google's John Mueller validated that if they can not creep the web page they can't view the noindex meta tag. He also helps make a fascinating mention of the website: hunt operator, recommending to ignore the outcomes because the "normal" customers won't find those end results.He created:." Yes, you are actually right: if our company can't creep the web page, our experts can't find the noindex. That said, if our company can not creep the pages, at that point there's not a lot for our company to mark. Thus while you may see several of those webpages with a targeted website:- query, the average customer won't see them, so I would not bother it. Noindex is actually also alright (without robots.txt disallow), it just suggests the URLs are going to end up being actually crept (and find yourself in the Search Console report for crawled/not indexed-- neither of these standings result in concerns to the rest of the internet site). The important part is that you don't make them crawlable + indexable.".Takeaways:.1. Mueller's solution confirms the constraints in operation the Website: search advanced search operator for analysis causes. One of those main reasons is actually given that it is actually not linked to the routine search index, it is actually a distinct factor entirely.Google's John Mueller commented on the web site hunt operator in 2021:." The short solution is that a website: question is certainly not meant to become total, neither utilized for diagnostics objectives.A web site query is a details type of search that restricts the end results to a specific site. It is actually basically merely the word web site, a colon, and afterwards the web site's domain name.This concern confines the outcomes to a details web site. It's certainly not indicated to become a complete compilation of all the webpages coming from that website.".2. Noindex tag without making use of a robots.txt is actually fine for these sort of circumstances where a robot is actually linking to non-existent pages that are actually getting found through Googlebot.3. Links along with the noindex tag are going to produce a "crawled/not listed" entry in Look Console which those will not have a bad impact on the rest of the web site.Check out the concern as well as address on LinkedIn:.Why would Google.com mark web pages when they can not even see the content?Featured Image by Shutterstock/Krakenimages. com.