How to Remove Website URLs from Google search indexing or crawling

Today in this article, we will see how to How to How to Remove Website URLs from Google search indexing or crawling.

You might find genuine need of not crawling or indexing the website.

But in today’s article, we will see a use case where you already have website URL which is indexed but you want to remove the URL due to some reasons.

Common Reasons for Removing the URLs

Few common reasons for Removing the URLs are as below,

• Private Content: To maintain privacy and security, website owners may restrict search engines from indexing certain pages or sections that contain data, internal documents, or private information.

• Outdated Content: Some websites might have outdated or low-quality content that could harm the overall search engine rankings. User may want no longer relevant or accurate.

• Temporary Pages: Under progress website with temporary pages that are not ready for public viewing might opt to exclude pages from search engine results.

Remove URL from Indexing Immediately – Manual

• Login to Google Search console

https://search.google.com/search-console

• Select your website from the list of websites available

• Under Indexing option please click on “Removal”

• Under tab “Temporary Removals ” please click on “NEW REQUEST”

• Finally Submit the url as below,

Above step will remove URL only for 6 month.

To remove URL permanently please follow below guidelines.

Remove URL permanently from Indexing

The above discussed removals tool removes the URL from google search for six months temporarily .

If you want to remove content or a URL from Google search permanently, please follow the steps as below,

• Remove the content of your site – Please remove images, pages, directories.

• Update the content of your site  to returns either a 404 (Not Found) or 410 (Gone) HTTP status code.

• Update the content to be accessed through password.

• Other possible option is using the noindex meta tag.

Remove URL permanently using Google Indexing API

One can easily remove URLs from Google search using Indexing API and requesting URL to be removed from search results.

This way , you can achieve automation where you can specify which URLs to be removed from indexing or update the URLs etc. or add new indexing URL.

Below is sample code using Python to achieve the same.

We are using Indexing POST API call using JOSN schema where you need to specify URL and type of operation as “URL_DELETED” .

urlList=["url1","url2", "url3"]
for index, item in enumerate(urlList):
    content = {
      "url": item,
      "type": "URL_DELETED"
    }
    response, content = http.request(ENDPOINT, method="POST", body=json.dumps(content))
    print(response)

For more details on how to setup Indexing API, please visit this article –How to Bulk Index and Crawl Website URLs To Google Search – SEO Automation

---

Please bookmark this page and share it with your friends. Please Subscribe to the blog to receive notifications on freshly published(2024) best practices and guidelines for software design and development.

---