I am excited to announce the release of the Screaming Frog SEO spider for Ubuntu. I know some of you might have already been using the SEO spider on Ubuntu, but this is a properly packaged version.
Alongside the Ubuntu release we have also updated the SEO spider to version 1.90. There are not many feature updates in 1.90, but there has been a lot of work behind the scenes improving the product and fixing a few technical issues. These include –
Include follow true/false at link level in our bulk link exports. This will allow you to find use of internal ‘nofollow’ instantly.
The SEO spider now crawls .swf (flash) files.
You can now export all alt text via the bulk export.
When you’re saving files, if you rename it so that it doesn’t include the file extension, it is now automatically added. This applies to both .seospider and .csv files.
The SEO spider now has exit warnings when you have unsaved data and provides an option to file save at that point. We have also added a warning when exiting the spider when it’s still crawling.
We fixed a relative linking crawling issue.
We fixed a case sensitivity issue in our crawling.
There were some reported issues of difficulty allocating memory above 1,024mb on 64-bit machines. This update should solve these!
We now ignore newlines in anchor tags.
If you already have a licence, you will be able to use your current licence in the Ubuntu version of the tool. As always, if you spot any problems, please contact us via the support page with the details.
To download the new version 1.90 of the Screaming Frog SEO spider, simply navigate to ‘help’ and ‘check for updates’ in the interface or download it here.
Because of the current Global economic situation (basically we’re all doomed), many organisations are reviewing marketing budgets and although we would all like to be in the ‘spend as much as you want’ camp, many advertisers are being forced to looking at their numbers and are making that difficult decision to cut back during these leaner times.
One of the first things to do when this conversation rears its head is to obviously chat through the issues with the client and get a full picture as to what’s happening and then offer your suggestions as to how your SEM efforts can best help the client through what is no doubt a difficult and stressful time. One of the great things about PPC is that budgets are flexible and can change whenever they need to and as long as you have a good structure in place you can control spend quite tightly ensuring that money is only spent in those areas that provide a return for the client.
With this in mind I thought I’d share some other tips and tricks to help get as much out of your budget as possible.
1.) Prioritise The Networks – Yes, AdWords does drive the majority of traffic but YSM & Bing traffic is often cheaper and conversion rates can be just as strong if not stronger.
2.) Analyse Your Google Search & Search Partners’ Data (If you do decide to go with AdWords) – Are the search partners spending a lot of your budget without driving any conversions? If this is the case, opt out and concentrate solely on Google Search.
3.) Investigate the Google Display (content) Network – Recent changes to this network have meant that performance is better than a few years back and the network can be used to great effect. How are your display campaigns doing? Is the network is performing well for you? Hopefully you will have followed best practice and separated your search & display campaigns in which case you should be able to quickly identify if either campaign is bringing overall performance down.
4.) Ad Scheduling – If you’ve got the data use it! Are there days of the week or hours in the day letting the side down in terms of conversion? You are spending money at midnight advertising to people who aren’t going on to convert? Are you predominantly a B2B advertiser who’s spending a lot of money at the weekends when your target audience isn’t online? Cut out this spend, it might not be vast amounts of money but as Tesco keep telling us “every little helps”.
5.) Look at Device targeting – Again, as with day parting data. Take a look in your analytics package and identify the value of mobile visitors. In an ideal world, you would obviously like to target as much of the market as possible, however, during these difficult times you really need to focus on those areas that give you a return. Are mobile/ tablet visitors converting? Is your site as mobile friendly as it could be? If the answers are no, then in the short term there really is little point targeting and spending your limited budget on these areas if they aren’t bringing the customers in through the door (metaphorically speaking).
6.) Geo-Targeting – This could be classed as one of the more basic settings, but if you are only able to fulfil orders in certain regions (e.g. mainland Britain) then you should only be targeting that area. You’d be surprised how many accounts I have taken management of and found that the campaigns were targeting areas that the business didn’t cover.
7.) Keyword Match Types & Negative keywords – Use Search Term Reports. Find those search queries that are converting for you and target them, spot those occasions that Google has matched your ad against irrelevant queries and negatively match them out. If you’re running a lot of broad match terms (heaven forbid) what variations are converting? Get them in… Whilst you’re there stop using broad match and use modified broad & phrase match more ;)
8.) Be ruthless with your keyword inventory – If it doesn’t convert and spends a lot, get rid of it. Simple as that. Also make sure you take into account your current keyword Quality Scores, if it is really low on some terms – delete them! (If you’re using Google Analytics, make use of Multi Channel funnels to ensure you’re not removing something higher up the sales funnel). If the reduced budget is likely to be temporary (which you would always hope it is) and the terms could be classed as seasonal, pause them until such a time as performance kicks in.
9.) Don’t Test Too Much – This sounds counter intuitive I know but I’m not saying don’t test at all just don’t be impatient and run loads of tests at once. Run a simple A/B test with couple of ads and then when you’ve got enough data to make a statistically valid decision (and only then) find your winner… Don’t fiddle with it until then.
10.) Optimise ad copy – We know that CTR helps improve quality score which will help reduce CPCs so go back to basics and optimise the copy as much as possible. Make sure the keywords are in the ad titles and description lines; include a strong call to action as well as any other USP that you may have (a lot to fit in 95 characters I know). This will help make your ad(s) stand out against the crowd and drive visitors to your site, which then brings me nicely onto my next point…
11.) Utilise Highly Relevant/Strong Landing Pages – I don’t usually like typing in all caps but my next point needs to be as clear as possible. DON’T SEND EVERYONE TO YOUR HOMEPAGE!! If someone is looking for Nike running shoes then send them to a Nike running shoe page, they’ll be much more likely to convert.
12.) Take Advantage of Ad Extensions – OK, like previous points this is probably more tailored to those of you running on AdWords as you’re not likely going to be offered the YSM rich ads at this level of budget but Sitelinks, Product extensions & listings, seller reviews they’ve all had great effects on CTR which is going to prove very useful in your current predicament (see point 10).
13.) Don’t Go For Position 1 – Being top is great but if it’s costing you more to appear there than the money coming in, you’re not going to be around for long. Find your optimal position and CPC and run from there. As long as your revenue or profit per click is higher than your actual cost per click you’re in a good place.
14.) Set expectations – If the cut in budget has come from an advertiser used to spending big then make sure they know that their performance may change. On a smaller budget they are unlikely to get those 1m visitors a month that they may be used to and so as long as they know this at the start there shouldn’t be any surprises later down the line when they compare data to previous timeframes.
I’m not going to pretend that this list covers off all the ways that you can get creative when working with small budgets, however, I think it covers off quite a few of the big things as well as covering a lot of the points we should be looking at when optimising any campaign regardless of budget size. If there are some things that you do that I haven’t included by all means please share them in the comments so that we can all learn from your greatness…
It’s a year ago today that we released the SEO spider, so we thought we should celebrate a little! Check out @matthopson‘s birthday cake skillz below –
The last 12-months have been fantastic, both in terms of development of the SEO spider but also our growth as a search marketing agency. The tool itself feels much older to us (probably because we had it in development and used it internally before releasing it publically), but it has come a long way in a fairly short period of time. We have had 8 new versions of the SEO spider in the last 12-months, each with new updates and features as well as releasing a Mac version.
The success of the tool would not of been possible without the support of the SEO community. We have been completely overawed with how it has been received and the feedback the SEO community has provided. I have been genuinely humbled by the continued support of many of those in the industry who have helped us develop it further. You guys rock.
We have even bigger plans for the tool (as well as other projects in the pipeline) for the following 12-months, so there is a lot more to come.
I am excited to announce an update to the Screaming Frog SEO spider, to version 1.80.
The following new features are included in the update –
Crawl Speed Control – The ability to control the speed of the crawl is something we have wanted to release for a long time. We have had some requests for throttle control due to the SEO spider crawling too fast for some older servers. Hence, we have introduced a ‘speed’ feature which allows you to limit the number of threads or the number of URI requests per second. This feature also allows you to significantly increase the speed of the SEO spider at the same time, which we just ask everyone uses responsibly :-).
XML Sitemaps – We have included the ability to create XML sitemaps in the SEO spider under the top level menu ‘export’ option. Currently this feature is only for HTML pages, so it does not include images, videos etc (which we will be introducing at a later date). We conform to the standards outlined in sitemaps.org protocol. If you have over 49,999 urls the SEO spider will automatically create additional sitemap files and create a sitemap index file referencing the sitemap locations.
Exclude Is Now Regex – Much like the ‘include’ feature, the ‘exclude’ option in the SEO spider is now regex. In previously versions it was robots.txt syntax, but regex provides more control and better balance.
Bulk Export Missing Alt Text URI – We have included the ability to bulk export all images with missing alt text and the URI that reference them.
List Mode Accepts XML Sitemap Files – We added the ability in ‘list’ mode to be able to upload and crawl .xml files. Hence, if you already have an XML sitemap and wish to audit it, simply switch to ‘list’ mode, change the filetype to ‘.xml’ and upload it. We will discover all the URLs automatically and crawl them.
We also fixed up a couple of bugs –
Level – We did not always find the shortest path. Now we do.
Https – There was a bug crawling secure pages in version 1.70. This has been fixed!
As always, please just let us know if anyone has any queries or feedback about the above. I will be updating the user guide this week (so please bare with me!).
Thanks again to everyone for all their support of the tool!
Over the past few years it’s become apparent that people all over the internet love techy/ geeky pumpkin carvings, you just have to Google “geeky pumpkins” and you can see there are about 11.5 million image results!
Now I know Halloween isn’t technically until Monday but everyone loves a distraction on a Friday and with this in mind, let me please introduce you to the newest geeky pumpkin, The Screaming Frog Jack O Lantern…
Happy Halloween!
p.s. As an extra treat (see what I did there) here’s a cool video of a Halloween light show. Enjoy…
Call: +44 (0)1491 415070
