Today in this article, we will see how to How to How to Remove Website URLs from Google search indexing or crawling.

You might find genuine need of not crawling or indexing the website.

But in today’s article, we will see a use case where you already have website URL which is indexed but you want to remove the URL due to some reasons.

Common Reasons for Removing the URLs

Few common reasons for Removing the URLs are as below,

• Private Content: To maintain privacy and security, website owners may restrict search engines from indexing certain pages or sections that contain data, internal documents, or private information.

• Outdated Content: Some websites might have outdated or low-quality content that could harm the overall search engine rankings. User may want no longer relevant or accurate.

• Temporary Pages: Under progress website with temporary pages that are not ready for public viewing might opt to exclude pages from search engine results.

Remove URL from Indexing Immediately – Manual

• Login to Google Search console

https://search.google.com/search-console

• Select your website from the list of websites available

• Under Indexing option please click on “Removal”

• Under tab “Temporary Removals ” please click on “NEW REQUEST”

• Finally Submit the url as below,

Above step will remove URL only for 6 month.

To remove URL permanently please follow below guidelines.

Remove URL permanently from Indexing

The above discussed removals tool removes the URL from google search for six months temporarily .

If you want to remove content or a URL from Google search permanently, please follow the steps as below,

• Remove the content of your site – Please remove images, pages, directories.

• Update the content of your site  to returns either a 404 (Not Found) or 410 (Gone) HTTP status code.

• Update the content to be accessed through password.

• Other possible option is using the noindex meta tag.

Remove URL permanently using Google Indexing API

One can easily remove URLs from Google search using Indexing API and requesting URL to be removed from search results.

This way , you can achieve automation where you can specify which URLs to be removed from indexing or update the URLs etc. or add new indexing URL.

Below is sample code using Python to achieve the same.

We are using Indexing POST API call using JOSN schema where you need to specify URL and type of operation as “URL_DELETED” .

urlList=["url1","url2", "url3"]
for index, item in enumerate(urlList):
    content = {
      "url": item,
      "type": "URL_DELETED"
    }
    response, content = http.request(ENDPOINT, method="POST", body=json.dumps(content))
    print(response)

For more details on how to setup Indexing API, please visit this article –How to Bulk Index and Crawl Website URLs To Google Search – SEO Automation

Please bookmark this page and share it with your friends. Please Subscribe to the blog to receive notifications on freshly published (2025) best practices and guidelines for software design and development.

The post How to Remove Website URLs from Google search indexing or crawling first appeared on TheCodeBuzz.

Today we will learn how to bulk Index URL to google search engine.

We will see manual and automated Indexing or Crawling techniques for helping your Website’s SEO optimization strategies.

You may find the need for the automation of this activity due to multiple reasons.

For example, the article is updated or article enhancement, etc.

You often try to index URLs to Google Search Engine using manual ways for your WordPress website or any custom websites.

Bulk Indexing will be a great SEO strategy you can adopt overall.

We shall cover the below aspects in today’s article,

Manual URL Indexing and Crawling

It is always difficult to perform indexing manually like using the below search console window option.

• https://search.google.com/search-console

The issue with Manual indexing

• To submit URLs, you must log in each time.

• Only authorized users, primarily Admin, can do indexing after logging in.

• It takes time, and if you have numerous URLs to send, you’ll have to submit each one manually.

• Crawling or indexing by Google can take some time ( delay may be longer than days or weeks)

• It’s conceivable that you forgot to index URLs (including modified or newly added URLs)

SEO Strategies – Automation for Url Index or crawling using Google Indexing API

Using an automation method, all of the above-mentioned challenges can be resolved.

We’ll develop a script to do authentication and authorization in this section.

Once authorized, the script can be updated with new URLs or existing modified URLs to submit or resubmit as posts are updated or resubmitted.

With this approach, you can submit multiple URLs to google as per the quota limit.

Quota limit is 200 URls as default.

Getting started – Bulk Index URLs to Google Search Engine

We shall cover the below set of steps needed to be performed,

• Create a service account
• Site Ownership- Add your service account as a site owner
• Writing a script and using an access token
• Submit URL Programmatically

A typical manual step involves below,

• Login to Google Search Console
• Submit each URL individually
• Wait for indexing to be effective

Before we start using the Indexing API, there are a few things that need to be set up,

Authorize Requests to Google Indexing API

Application requests must be authorized by an authenticated user who has access to that data.

• Request applications sent to the Indexing API must include an authorization token.

• Make sure to Enable Indexing API using the below link,
  • https://console.developers.google.com/flows/enableapi?apiid=indexing.googleapis.com&credential=client_key

Below we will cover high-level steps for achieving the Authorization.

Create a Project

Before you can send requests to the Indexing API, you need to tell Google about the client and activate access to the API.

You do this by using the Google Cloud Console to create a project, which is a named collection of settings and API access information, and register your application.

To get started using Indexing API, visit use the setup tool, which guides you through creating a project in the Google API Console, enabling the API, and creating credentials.

Create a service account

• Go to  Service accounts and Select your project else create a new project as explained above.

• Click Add Create Service Account, and enter a name and description for the service account. You can use the default service account ID, or choose a different, unique one. When done click Create.

• The Service account permissions (optional) section that follow is not required. Click Continue.

• On the Grant users access to this service account. You can add a specific role or keep the role optional.

• Once done you shall see your service account create as below,

• In the side panel, click on Action or: symbol. Please select Manage keys.

Select the format for your private key: JSON is recommended.

• Click Create. A new public/private key pair will be generated and downloaded to your machine;

Site Ownership – Add your service account as a site owner

For site ownership, please perform the below steps,

Prove as the Owner of the site 

The verification process of proving that you own the property needs to be done by you.

Please note that verified owners will have access to sensitive Search data for a site. Please click here for more details

Grant owner status to your service account 

The next step is to make your Service account(created in the above step) a site owner.

This step is an important step toward getting actual access to the Indexing API.

• Please Open Webmaster Central

• Click on the verified website.

• In the Verified owner list, click Add an owner.

• Provide your service account email as the delegated owner. You can find your service account email address in two places:
  • client_email – This is a field in the JSON private key in the JSON service account file which we downloaded in one of the above steps.

An email has the below pattern,

<your-service-account>@<your-project-name>.google.com.iam.gserviceaccount.com

Final Step- Bulk Submit URLs Programmatically

That’s all, you are now all set to set up the automation.

With required prerequisites to start invoking the Google Indexing API.

As a next step, please use any of the scripts of your choice(you are comfortable with) to call the Google Indexing API.

Please see the below article for the final steps where we will use a script to send the URL for Google indexing programmatically.

• Python script for Indexing the URL

• Using JAVA for indexing/Crawling the URL

• Node js example for indexing API

Please choose the language of your choice to script the indexing or crawling logic using automation techniques.

Do you have any comments or ideas or any better suggestions to share?

Please sound off your comments below.

Happy Coding !!

Please bookmark this page and share it with your friends. Please Subscribe to the blog to receive notifications on freshly published (2025) best practices and guidelines for software design and development.

The post How To Bulk Index URL To Google Search – SEO Automation first appeared on TheCodeBuzz.

Today in this article, we will learn how to use Python script for Bulk Submit Website URLs to Google Search Console for Indexing or Crawling purpose.

We already learned how to set up the service account, project, and other configurations so that we can leverage Google’s Indexing API before.

Today in this article, we will cover below aspects,

If you have not performed mandatory prerequisites, please follow the below article for more details,

• Configuration to setup multiple URLs submit to Google Search Console

You may find the need to submit the URLs due to many reasons like article update due to new data or upgrade or article enhancement, etc.

In such scenarios, it is always difficult to perform indexing manually.

We shall be using below endpoints required for publishing the URLs,

SCOPES = [ “https://www.googleapis.com/auth/indexing“ ]

ENDPOINT = “https://indexing.googleapis.com/v3/urlNotifications:publish“

In your python script please add below “using” namespace,

from oauth2client.service_account import ServiceAccountCredentials
import httplib2
import json

Sample Script

from oauth2client.service_account import ServiceAccountCredentials
import httplib2
import json

SCOPES= [ "https://www.googleapis.com/auth/indexing" ]
ENDPOINT= "https://indexing.googleapis.com/v3/urlNotifications:publish"

# thecodebuzz-da659b2b8b0d.json is the private key that you created for your service account.


JSON_KEY_FILE = "thecodebuzz-da659b2b8b0d.json" #Specify your service account JSON here
credentials = ServiceAccountCredentials.from_json_keyfile_name(JSON_KEY_FILE, scopes=SCOPES)
http = credentials.authorize(httplib2.Http())
url="url-1"
content = {
      "url": url,
      "type": "URL_UPDATED"
    }
    response, content = http.request(ENDPOINT, method="POST", body=json.dumps(content))
    print(response)

from_json_keyfile_name method uses the JSON file service account information which we created in the previous article.

The above code works perfectly fine for one URL which we are providing the variable “URL”

Please install packages like oauth2client or httplib2 as required.

Python script for Bulk Submit URLs to Google Search Console

You can use Python built-in function called enumerate to submit the URL list as below,

urlList=["url1","url2", "url3"]
for index, item in enumerate(urlList):
    content = {
      "url": item,
      "type": "URL_UPDATED"
    }
    response, content = http.request(ENDPOINT, method="POST", body=json.dumps(content))
    print(response)

Once successfully submitted the URLs for indexing, you can see the status code as 200 for the request.

References :

Please follow the below article for more details on prerequisites,

• Configuration to setup multiple URLs submit to Google Search Console

• Using JAVA for indexing/Crawling the URL

That’s all! Happy coding!

Does this help you fix your issue?

Do you have any better solutions or suggestions? Please sound off your comments below.

Please bookmark this page and share it with your friends. Please Subscribe to the blog to receive notifications on freshly published (2025) best practices and guidelines for software design and development.

The post Python – Submit Website URLs To Google Search Index first appeared on TheCodeBuzz.