Screaming Frog and your website's SEO

Posted by Camille Storms on Tue, Feb 02, 2016 @ 09:14 AM


Read time: 2 - 3 minutes

You’ve created your website. You’ve put up some content. You’re sharing it on social media. Now what? Will Google like your website? Will it be able to crawl your site easily? An online program known as Screaming Frog could help you answer these questions.

Screaming Frog Spider?!

It’s an unusual name, but catchy nonetheless. The Screaming Frog SEO (Search Engine Optimisation) Spider is a small desktop program you can install on your computer that allows you to quickly analyse, audit and review a site from an SEO perspective. Screaming Frog is free for indexing websites with less than 500 pages.

However, it could also be worth it for larger websites as it could be especially helpful in analysing medium-to-large sites where manually checking every page would undeniably laborious and time-consuming. Not to mention doing it manually would mean you have more chances of error – you could easily miss unnecessary redirects, meta refresh or duplicate page issues.

Some things you can do with Screaming Frog

  • Simple check-up – enter your site address and the program will do a quick overview and check factors on your site. After a bit of time, a full report for analysis will be ready for you.

  • Identify duplicate content – Google does not look kindly upon duplicate content and if found on your site will automatically affect your rank. Google's Panda update was certainly a game changer, and it caused great losses for some sites. You can easily identify and remove any duplicate content on your site by using this tool’s features.

  • Titles, Descriptions, Keywords and Main Headings (h1) – you can easily view all of these elements in one place. Are they aligned to the keyword goal of each page? If not, you can now edit the pages.

  • Missing Alt Text on images – list all missing Alt Text to go fix.

  • Links to a page – it will tell you which pages only have a few links to them, meaning that you may have forgotten to put them in your navigation. If you have no links to a page, how do you think a search engine would find the page?

  • Crawl-ability – if Screaming Frog can’t make quick work of your website, regardless of size, neither can Google; this indicates that there’s an indexing problem on your site that you need to identify and fix.

  • Create a sitemap – you can make XML sitemaps with relative ease, but it’s important that you set up the crawl correctly before you start. If you don’t limit the spider to crawl only those pages that you want in your sitemap, you can end up with a bunch of URLs that shouldn’t be in there.

  • Enhance your review using Google Analytics – you can pull your Google Analytics data directly in the crawl. For example, you could crawl your site to find the pages with the most sessions/visits from a specific segment of your audience, and see which pages are getting the most views.


Topics: Online Advice, SEO Resources, Technology

Recent Posts