TL;DR:

  • Use a page monitoring tool such as VisualPing
  • Grab Google's Cache of a web page (cache:http://example.com/example)
  • Use the tool to detect changes in the date and time of the snapshot

Why?

A client recently made changes to some specific pages on their site.

The site is not crawled very often, so it can take a little while for changes to be indexed.

They asked if it was possible to know when Google had updated their cache of those pages.

This is because they wanted to have a rough idea when Google indexed the changes they had made.

Considerations when discussing Indexing and Google's Cache of a Web Page

The cache of a page is what it looked like last time Googlebot visited a page.

Edit, thanks to Gerry White: The cache is the snapshot of a page as it appeared the last time Google visited a page.

The logic behind Google's cache is to keep a record of the page for users in the event the site is down.

Putting aside intentional attempts to remove pages from the index, such as:

  • Robots.txt, Disallow & Noindex Directives
  • <meta name="robots" content="noindex">
  • X-Robots-Tag HTTP header

Most pages that are important to your SEO efforts, you usually want to be indexed.

If they are important and recieve traffic from Google, they usually have a Cache version.

Pages that are deemed low value by Google might be indexed and retrievable using the info: search operator.

However, they will not be assigned a Cache by version.

Finally, if this meta tag is present, there will not be a cache version:

<meta name="robots" content="noarchive">

In this instance, all of the URLs were indexed and had a Cache version.

When the URLs were updated by the client, I used the fetch tool within Search Console and submitted the URL to the index.

I also used Google's Submit URL Tool, it can help with getting web pages into the index quickly.

VisualPing Webpage Monitor

I remembered that Gerry White had tweeted about a website called VisualPing.

It's a tool that will monitor changes on a web page for you.

The most obvious application is to track when competitors update their websites.

Or when a developer is feeling particularly YOLO. For example, I've set it up on client robot.txt files.

However, I wondered if I could use it to track changes in the time and date on Google's cache of a web page.

If I were able to automate checking for a change in the time and date of the cache, I would be able to notify the client exactly when the changes were indexed.

How to use Visual Ping to monitor when Google updates the cache of a web page in their index

Step One - Sign up to VisualPing

Sign up to visualping, if you use my code we both get credited 100 checks for free. Shameless, I know.

Step Two - Grab Google's Cache of the Page

Find Google's cached version of the page. You can do this one of two ways:

1. Google Chrome

Open Google Chrome and add cache: to the start the URL.

I'm going to use a client-neutral URL for this example.

cache:https://www.amazon.co.uk/

2. Google SERPs

Search the URL on Google and then click the small arrow by the URL.

Google SERP Cache Selection

Either of these methods will give you the cache URL.

Step Three - Setup VisualPing to Monitor when the time and date of Google's cache snapshot changes

On the cache version of a page, Google gives the following information on when their cache was updated:

This is Google's cache of https://www.amazon.co.uk/. It is a snapshot of the page as it appeared on Apr 3, 2017 15:38:44 GMT.

We are going to use VisualPing to monitor when the time and date changes.

This will tell us when the changes we have made to a web page have made it into Google's index.

Login to VisualPing and create a new job:

Visual Ping Create New Job

Select Web to detect changes in text on the page:

Visual Ping Web Selected

Paste in the Google's cache URL for the page and select VisualPing to monitor changes to the time and date of the cache.

Creating a Job on Visual Ping

Enter your email address and wait for a notification that the text has changed.

Step 4 - Notification

Dependant upon how frequently you set the tool to monitor for changes, you will get an email notifying you that a change was detected.

VisualPing Email Notification

Open up the email and VisualPing will present the two comparison snapshots.

VisualPing Email

For those of you on a mobile device, struggling to read that:

Before:

This is Google's cache of https://www.amazon.co.uk/. It is a snapshot of the page as it appeared on 3 Apr, 2017 15:38:44 GMT.

After:

This is Google's cache of https://www.amazon.co.uk/. It is a snapshot of the page as it appeared on 4 Apr, 2017 05:40:16 GMT.

We can see that the Cache of the web page was updated.

If we had made any substantial changes to the web page, these would now be indexed.

Notification When Google Updates the Cache of a Web Page

Utilising Visualping was a simple solution to the request of my client.

Although it is possible to automate something like URLProfiler to do this.

Oliver Mason also pointed out to me on Skype that it would be possible to automate Scrapbox for this too.

However, neither of those solutions are as simple as what I have outlined above.

With that being said, if you had a lot of URLs to check. URLProfiler is the best tool for the job.

The guys over at URLProfiler put together an excellent guide on checking Indexing of and the cache of URLs here.