And you can shift this to see if you get different results. And as you can see, if you hover over it, it'll tell you a little bit more about it. Right now, it's set to text only, so it's just rendering the text itself. Rendering, this allows you to crawl sites that maybe don't play nice with crawlers. How many redirects you want to follow, right? So if they've got a redirect chain going on and all the different URLs and how deep you want to go. So this is where you really want to set up those settings here. It takes a lot of time, but it's definitely a tool that that can handle that kind of work for you.
![screaming frog seo spider tool screaming frog seo spider tool](https://www.screamingfrog.co.uk/wp-content/uploads/2016/12/custom-robots-txt.png)
We've done crawls with over 30,000 links. So how many pages do you want to crawl? Depending on your business, you could crawl pages with 50 links, you could crawl pages through up to 10,030. So if you want to see are they using structured data, you want to have these checked. You can do page titles in metas, but you can also do JSON-LD, RDFa and. There's a number of things you can extract as well with Screaming Frog, which is really cool.
![screaming frog seo spider tool screaming frog seo spider tool](https://www.softpaz.com/screenshots/screaming-frog-seo-spider-screaming-frog-ltd/6.png)
You can even crawl the site maps if you want to as well.
![screaming frog seo spider tool screaming frog seo spider tool](https://i.pinimg.com/originals/0d/16/33/0d163374d270d12a03e2608f50eb5844.jpg)
And if you're doing an in-depth crawl and you want to get a very in-depth view of that website, I highly recommend that you have these checked.īut if you're really just looking to get a basic idea of what's going on with a website, or maybe you're doing a quick audit, or maybe someone's asked you like, "Hey, do you want to work with our site?" You don't have to go super deep in that first go around to really see whether or not this site is something that you want to get involved with.īut now, as you can see, you can crawl all of these different elements. It will take longer if you crawl the images in the CSS in a JavaScript. If you're going to do a deep crawl, you're going to want to probably have a lot of these things clicked and checked off.īut if you want to just get a quick site crawl and understand just at a very basic level how optimized the site is or what's the foundations of this site, you don't have to actually crawl and restore all of these different things. Now, I like to go in here first and do some configurations. Now in this video, we're talking specifically about crawling our own website. So this is an extremely useful tool if you want to really understand your website, maybe a list of pages, as well as the search results themselves. So you can run a list, you can actually pull data from the search results and then you can compare. You can do your whole domain here, but if you want to just run a few pages, you can do that as well. But there's a number of ways to actually run this crawl. So the first thing you're going to do with Screaming Frog is you can add your URL in here and if you just want to go, you can go ahead and hit start. And then I'm also going to share quickly just a very simple report visualization that we can do within Google Sheets. We're just going to go a quick overview of how it works, how you'd set it up and how you'd run it.
![screaming frog seo spider tool screaming frog seo spider tool](https://trafficradius.com.au/wp-content/uploads/2020/02/screaming-frog-seo-spider.jpg)
Screaming Frog SEO Spider is probably one of the most useful tools for any SEO or really any site owner that really wants to understand more about their website. All right, so in this video, I'm going to walk you really quickly into an overview of a basic site crawl using Screaming Frog. SEO Crawl with Screaming FrogĪll right, let's get into the content. This is going to allow us to see a site from a very basic level, but help give us an understanding of maybe some big or quick wins that we could use with a client or a possible prospect. And today we're talking all about a quick audit. And it helps us get both a 30,000 foot view of a website as well as a extremely granular detailed view of a website. It's a tool that we use pretty much every single day here at the agency. So as I talked about in the opener, we're going to be walking through how we can set up a quick SEO crawl using Screaming Frog SEO Spider. We create new content each week to help you get the most out of your digital marketing efforts. If you've enjoyed the videos we've created on this channel, please hit subscribe, and don't forget to turn on that notification button. In the latest episode of Hack My Growth, I'm going to walk you through a quick SEO crawl using Screaming Frog SEO Spider, as well as how we can quickly visualize that data using a Google Sheets template.