Google Search Console Coverage Report: New Filter for Unsubmitted Pages

Google Search Console is an integral tool for any webmaster, providing vital information about their website and how its performing.

One of the key areas that the Google Search Console reports on is how your website looked when it was crawled and indexed, plus your organic performance too.

Understanding the reporting available in the Google Search Console is imperative for all webmasters, and the recent change will make it simpler to find exactly what you need.

What is the Google Search Console Index Coverage Report?

To understand the Google Search Console Index Coverage Report, you’ll first need a rudimentary understanding of crawling and indexing, and why it’s so important.

Before a site can be crawled, it must first be discovered. Google does this by following links, which can be offsite or onsite, and by processing XML sitemaps.

Once a URL has been discovered, it’s then set aside for crawling. When a site is being crawled, Google requests URLs and gains data from them. All of this is then passed to the indexing phase.

During indexing, Google (or any other search engine) assesses the data gleaned from crawling. The process is complex but essentially, the goal is to look at the authority of a URL and determine how relevant is to queries.

Once indexing is complete, a URL can appear in the search results. In other words, your web pages cannot appear in the search results on Google until it has been fully crawled and indexed. And that’s why the Google Search Console Index Coverage Report is so important.

When crawling and indexing is complete, you’ll be able to see the results in the Index Coverage Report in Google Search Console. All of the technical data found will be contained in the report, with the feedback organised into four possible groups:

• Valid - these pages have been indexed
• Valid with warnings - these pages have been indexes but require some attention
• Excluded - these pages haven’t been indexed as there were indications that they should not be indexed
• Error - these pages couldn’t be indexed due to a specific problem

You should receive prompts if there are any major issues with the crawling and indexing, but these notifications can be unreliable. It’s therefore not ideal to rely on them to let you know when something is wrong.

The New Simplified Report

Google is keen to make changes to its reporting to give users the very best possible data in a format that’s easy to navigate.

Some of the most recent changes have been to fix the validate fix button (a known glitch) and launched the more simplified version of the coverage report.

If you previously used the “indexed, not submitted” filter button, you will have noticed that it’s disappeared. But don’t worry - you can still retrieve results that relate to unsubmitted pages only.

Google hasn’t gotten rid of the unsubmitted pages as being identifiable, but they’ve changed the way users find them. When running the coverage report, you’ll notice a new filter. This provides the option for either:

• All known pages
• All submitted pages
• Unsubmitted pages only

Therefore, when looking at the indexed pages report, by running this new filter it’s still possible to view the “indexed, not submitted” pages as you would have done previously.

Why Is the Filter Important?

The indexed, not submitted criteria is a vital one for any webmaster to understand. This is because action may be required if the pages on your website are being indexed but not submitted.

Both indexed and submitted plus indexed, not submitted will be included in the Valid group of pages.

No action is required on the pages which are indexed and submitted.

On the pages which are indexed, not submitted, webmasters may want to dig into the site in more detail. Pages that haven’t been submitted were not identified for crawling through an XML sitemap. Google would have found them a different way and opted to crawl them anyway.

You should check whether these pages need to be indexed, and if so, they should be added to the XML sitemap.

If you have an XML sitemap but haven’t yet submitted it to be registered by Google Search Console, all of your URLs will be classified as “indexed, not submitted in sitemap”.

Not all pages will need to be indexed and added to the sitemap. If they should legitimately be excluded, you should use the robots noindex directive. They should also be excluded specifically within your robots.txt if there’s any chance they could cause problems for your crawl budget.

Understanding the Coverage Report

The dropdown filter will enable you to see all of the pages, or just the ones which Google found which hadn’t been submitted.

If Google discovered the page because it was organically crawling a different URL, it will still be classed as being included in the sitemap.

Depending on the size of your site and the number of issues there are, there many be multiple entries in the Coverage Report. To drill into any of them in more detail, just click on the relevant row. This will then open up a whole page with further data.

There are a number of common reasons why a page has been indexed but not submitted. These include:

• Server error (5xx)
• Redirect error
• Blocked by robots.txt
• Marked “noindex”
• Soft 404 response
• Unauthorised request (401)
• Access forbidden (403)
• Blocked due to 4xx issue
• Blocked by page removal tool
• Crawled not indexed
• Discovered not indexed
• Alternate page with canonical tag
• Duplicate
• Page with redirect

If there is a particular issue affecting the page, you can check for all pages which are affected by the same issue. Doing this will enable you to correct all of the same warnings simultaneously. Once you have corrected the error, you can ask Google to take another look by clicking on the Validate Fix button.

If you notice a URL that has an error which you have already corrected, it’s worth checking the crawl date. If this is before any fixes you implemented, the only remaining issue is letting Google know. Confirm your fix and then re-submit the page to Google for indexing; this should correct the issue.