Have you at any time necessary to prevent Google from indexing a particular URL on your world-wide-web internet site and displaying it in their lookup engine outcomes webpages (SERPs)? If you deal with net web sites prolonged sufficient, a day will possible occur when you need to know how to do this.
The three methods most normally utilized to prevent the indexing of a URL by Google are as follows:
Applying the rel=”nofollow” attribute on all anchor things made use of to connection to the site to reduce the hyperlinks from getting followed by the crawler.
Applying a disallow directive in the site’s robots.txt file to avert the web page from staying crawled and indexed.
Utilizing the meta robots tag with the material=”noindex” attribute to protect against the website page from currently being indexed.
Although the variations in the three approaches show up to be delicate at 1st look, the success can fluctuate significantly depending on which system you pick out.
Utilizing rel=”nofollow” to avert Google indexing
A lot of inexperienced site owners try to prevent Google from indexing a certain URL by using the rel=”nofollow” attribute on HTML anchor features. They incorporate the attribute to each anchor factor on their site used to link to that URL.
Like a rel=”nofollow” attribute on a hyperlink stops Google’s crawler from subsequent the url which, in change, prevents them from finding, crawling, and indexing the focus on page. Though this approach could possibly get the job done as a short-term option, it is not a viable lengthy-time period alternative.
The flaw with this approach is that it assumes all inbound inbound links to the URL will include a rel=”nofollow” attribute. The webmaster, on the other hand, has no way to prevent other net web pages from linking to the URL with a adopted url. So the probabilities that the URL will ultimately get crawled and indexed using this method is pretty substantial.
Working with robots.txt to prevent Google indexing
One more popular system utilized to stop the indexing of a URL by Google is to use the robots.txt file. A disallow directive can be additional to the robots.txt file for the URL in question. Google’s crawler will honor the directive which will stop the web page from remaining crawled and indexed. In some scenarios, however, the URL can still show up in the SERPs.
From time to time Google will show a URL in their SERPs though they have hardly ever indexed the contents of that web site. If enough world-wide-web web sites website link to the URL then Google can normally infer the matter of the website page from the connection textual content of those inbound one-way links. As a outcome they will present the URL in the SERPs for associated searches. When making use of a disallow directive in the robots.txt file will protect against Google from crawling and indexing a URL, it does not warranty that the URL will in no way appear in the SERPs.
Applying google scrape tag to protect against Google indexing
If you want to prevent Google from indexing a URL while also preventing that URL from being displayed in the SERPs then the most successful technique is to use a meta robots tag with a material=”noindex” attribute inside of the head ingredient of the net site. Of training course, for Google to basically see this meta robots tag they require to initial be capable to find and crawl the webpage, so do not block the URL with robots.txt. When Google crawls the site and discovers the meta robots noindex tag, they will flag the URL so that it will by no means be proven in the SERPs. This is the most successful way to prevent Google from indexing a URL and displaying it in their lookup success.