3 Ways We Use the WayBack Machine for SEO

Whether we’re hunting down old links for our redirect list, becoming familiar with a new client, or troubleshooting various site issues, the WayBack Machine is one of our go-to tools for SEO here at Rocket Clicks.

What is the Internet Archive?

Founded in 1996, the Internet Archive is a non-profit, ever-growing digital library providing free access to more than 362 billion web pages. To use the WayBack Machine, simply go to archive.org, and enter the URL you’re interested in exploring. This doesn’t have to be the homepage; the WayBack Machine has the potential to find any webpage ever published, which brings us to the first way we use this tool for SEO.

How to Build a Redirect List Using the WayBack Machine

It’s common for companies to seek SEO support prior to the launch of a new website. If our client is working on a website redesign or planning to launch a new site, one of our main focuses is to ensure we’re setting up redirects for as many existing URLs as possible. We’ll start with a simple Screaming Frog crawl for our base list of URLs, and then continue to build it out using various tools such as Google Analytics and Ahrefs to identify any outstanding URLs Screaming Frog may have missed.

Once we’ve exhausted all our options, we’ll go through the list one by one to match up each entry with an appropriate 301 redirect on the new site. However, it’s usually not that straightforward. Oftentimes we’ll have older URLs on our list that no longer exists on the current site, which is where the Internet Archive comes into play. By plugging those URLs into the WayBack Machine, we’re able to identify the content and purpose of the page to assign a more accurate 301 redirect than simply sending the user to the new homepage.

Get to Know Your Client Using the WayBack Machine

When we take on a new client, one of our first orders of business is to become familiar with the ins and outs of the company. We want to understand details of their products and services, get to know key team members, and start to piece together how the company operates. In order to do so, it’s important to understand the company’s history, and what better way to do that than by using the WayBack Machine.

Explore a website’s history with the WayBack Machine

1. Enter the client’s URL into the WayBack Machine
2. Follow the timeline back to when the site was first archived
3. Click on the year, and sift through the results calendar to find the earliest snapshots of your client’s site
4. Select a specific date from the calendar based on the result’s color
   1. Blue means the web crawler captured a good, 2xx result
   2. Green means the web crawler got a redirect, 3xx result
   3. Orange means the web crawler got a client error, 4xx result
   4. Red means the web crawler got a server error, 5xx result

If possible, always select a blue dot from the results calendar. We’ll continue down the timeline exploring different results until we find the current version of the website. This process gives us a better understanding of where the client’s growth, as well as learning the core values that have stuck with them since the beginning.

Pro Tip – the WayBack Machine allows users to browse through a website once the archive is opened. It’s not a static image but rather a site that can be utilized and explored.

Troubleshoot Website Issues Using the WayBack Machine

Last but not least, the WayBack Machine really comes in handy when we’re working to troubleshoot website issues such as significant shifts in traffic. Here’s what we look for:

• Changes to the meta description
• New or missing parts of a page
• Differences in page structure
• Modifications to on-page content
• Changes to the Call-to-Action

The WayBack Machine also archives robots.txt files, so we’ll review these to see if there were any changes in the crawling permissions that may have affected the shift in traffic.

All in all, the WayBack Machine is an excellent tool to keep in your SEO toolbox for a variety of situations. Get to know a new client, build out a solid redirect list, and troubleshoot a wide range of technical issues with the free, easy-to-use Internet Archive.