Tag: read and react report

Here’s a quick way to get Google Analytics to shine a light on problems on your site and help you sort them out. You can also use this tip to spot promising sources of new customers and other good news like that.

In the practical example you’ll learn how easy it is to set GA to alert you when new visitors are landing on a 404 (“Page Not Found”) error page and give you a one-click route to the source of the problem so you can head over there and fix it.

That’s powerful stuff. If someone comes to your site and all they get is an error they may not give you a second chance.

To adapt a well-known analytics phrase: “They came, you slapped them in the face, they left”. So a quick fix for these problems is very valuable indeed. Using this approach, GA gives you a great way of sorting things out fast.

Once you’ve tried this very practical example, you’ll probably find that you want to apply the same method to other key events on your site, including good ones. It’s and easy but powerful technique — let Google Analytics do some of the tricky work for you!

The secret is to combine micro-conversion goals with GA Custom Alerts. The hidden super-power of this combination is that it uses the cleverness of Google to give you clues as to ‘why’ something unusual has happened, not just to flag up the fact that it ‘has’ happened.

First I’d like to thank @mattycurry for really driving this point home in a @boagworld podcast I heard a while back.

Example: How to Spot and Fix Broken Links from External Web Sites

Use this simple technique to make a “Read and React” report for spotting accidentally broken links from external web sites. In the age of user-generated-content there’s a lot of this about. I often see it with links being posted in forums on voucher or coupon code sites.

Using Custom Alerts I can not only see that there’s been a sudden increase in 404 pages:

But GA also provides me with an instant analysis of the source of these visits:

And, since these visitors are being referred from another site, there’s also a handy button to take me to the referring page with the dodgy link. So I can click straight over there and try to sort things out.

In short: GA tells me something’s wrong, suggests what the cause may be and even helps me on my way to providing a fix. Nice.

How to Set Up a Custom Alert Based on a Goal

The first thing you need to do is to make sure that your 404 page is being tracked in Google Analytics. I’ve written about this already in another post, which also contains an interesting discussion in the comments about how to get the best out of error tracking if you’re starting from scratch. But this time round I’ll assume that you can identify your 404 page as either a pageview or an event in Google Analytics.

Once you know how your 404 page is being tracked, you need to set up a Goal for it in Google Analytics.

The next thing to do is to set up a Custom Alert based on that Goal. It’s true that GA will show you automatic alerts for unusual Goal data without you needing to configure anything. But I currently recommend using a Custom Alert for two reasons:

1. Custom Alerts allow you to fine tune the timing and scale of the change which will trigger the alert
2. Custom Alerts generate the oh-so-helpful ‘Major Contributors’ report

Here’s a video showing you how to do those two stages:

And that’s it.

Now all you need to do is check the Google Analytics Intelligence reports each day and watch for those alerts. When you spot one, look at the “Major Contributors” then click through to the source and try to fix the problem. If it’s something someone has posted in a forum you’ve got a golden opportunity to sort out the problem and generally demonstrate your willingness to listen and to help.

I’ve seen discussions with broken links to a promotional offer start to turn hostile because people assume that the offer has been suddenly withdrawn. This situation is a classic example of where you can turn the impression of your brand all the way from negative, through neutral, and right round to positive just by showing up and being helpful. You can rescue thousands of visits by using this technique if you move fast and handle things well.

Final point on ‘what to do’ — don’t forget to add a Google Analytics Annotation to note what happened:

Why You Shouldn’t Rely Just on Email Alerts

To repeat one point from the video: at the moment I think you need to check the reports rather than rely on the automatic email and SMS alert options.

The Intelligence processing takes time and (in my UK time-zone) those emails don’t arrive until more than a day after the trouble began. With this particular alert you need to react as quickly as possible.

In fact I also add the 404 Page Goal Conversion Rate to my morning dashboard so that I can spot anything nasty as soon as I start work. I use a desktop dashboard client which is configured to include the current day as well as the last last month so that I can spot what’s happening in almost real time. That’s good in case something really bad is happening. But then I need to do some manual work right away to track down the source because I don’t have access to that handy one-click route via the “Major Contributors” report.

Use the Custom Alerts and Goals Combination Again and Again

The example I’ve given above is a very specific and practical one which should be relevant to many sites and which can help you take action to fix something which is broken.

You can also use the same technique to tell you when something good is happening and suggest why.

For example: you may well already have a Goal set up for the “Add To Cart” micro-conversion. This approach could point you at the source of an unexpected increase in purchases.

Better still, if you have a goal which matches the start of a “new customer” checkout, the “Major Contributors” report could alert you to a promising new referral source of business. This is the kind of thing you might not normally see buried in all the campaign activity to recruit new customers.

Google Analytics Intelligence is a really powerful tool for spotting things that you may not be actively monitoring. It shows you things you weren’t looking for. You can take a look at that unexpected source and see if there’s anything you can do to get more of those people interested in you. In the age of social networking, simply showing up and showing you care can bring big rewards. Intelligence Alerts can help you take an interest in things you would not normally have noticed. Give it a try!

Resources Related to Using Google Analytics Custom Alerts and Goals

• Previous article about Why Error Messages are a Shortcut to Improving Customer Experience
• Occam’s Razor by Avinash Kaushik: huge list of great custom alert ideas (be sure to read comments)
• Official GA Video On Intelligence Alerts:

If you want to know what to do to make your site better for your visitors, you need to know what they find bad. And one of the best indicators that someone is having a bad time is if they keep seeing annoying error messages.

• Why error messages are a shortcut to improving customer experience
• Why use virtual page views to track errors
• How to choose a pageview structure to get the best from the content drilldown report
• Resources related to tracking error messages in Google Analytics
• Updates

  March 2015: how working with errors and copy reduced Mailchimp’s sign-in problems

  May 2014: Examples of custom reports, sequence segments, why events may be best — my Presentation from Superweek 2014

  November 2012: Using Custom Variables to Track Errors

Why error messages are a shortcut to improving customer experience

Most error messages are triggered when someone is trying to do something, even if it’s only to click a link to a page which isn’t actually there.

So these people are already starting to engage with your site. And then they get a slap in the face.

Many error messages tell the visitor that they have done something wrong. It’s their fault.

These people may start to feel frustrated and possibly annoyed by your site.

That’s why it’s such a good idea to track error messages. They provide a very direct source of information about where your site is tripping people up and causing friction.

I say “your site” because it is probably the site’s fault not theirs. If people are entering the wrong information, or skipping required fields it’s often because the language or signposting on the site is weak.

OK: a typing mistake is a typing mistake — that’s the visitor’s fault, isn’t it? Fair enough. But if a lot of people are having trouble there, ask yourself: “do we really need that field, have we ever actually used that information”. Look at the exit rate from that error message and judge whether the information is worth it.

So: error messages are a great source of actionable insight on the exact points of friction on your site. The action is simple: do what you can to reduce the errors. I’d actually set up goals and measure the conversion rate.

The snag?

Error messages are seldom page views, apart from the 404 “page not found” page.

Error messages often take the form of text and highlighting which indicates the ‘bad’ entry in a form (good). Others are just ugly alert boxes which pop up in the face of the visitor, with a beep, telling them they’ve screwed up, not telling them where, and waiting to be dismissed (not so good). Neither of those would normally register as a page view, so a web analytics tool like Google Analytics will know nothing about them.

What you need to do is add extra tracking code which is fired whenever the error message is shown. At the moment I favour using virtual page views for this in Google Analytics.

Why use virtual page views to track errors

There is an argument to say that errors should be tracked as ‘events’ not pageviews because they are an interaction on the site, not an actual page view. That’s debatable: the visitor certainly saw something!

Actually, when you track these you may find that some errors are repeated again and again. This is classic evidence to suggest that the visitor might not even have noticed the error message — for example a list of form errors in small red text displayed at the top of the page, way above the submit button down at the bottom of the screen. The visitor’s eyes will be looking where they just clicked. Tip: compare the pageviews and unique page views for errors to identify errors which are seen several time by the same person:

The real reason I favour virtual page views in GA is that, at the moment, only page views will appear in funnel reports as ‘abandon routes’ on the right side of the screen. Error messages in a checkout process would be a classic reason for abandoning a checkout. That’s something you definitely want to know about. So it’s best to have the information where you need it: in the checkout funnel report.

[Update March 2011: The version of Google Analytics released in March 2011 introduced the ability to use ‘Events’ as goals. This may be more appropriate for many error messages. There’s a link to an article giving instructions on using Events at the end of this post. However Events do not appear as abandon routes on the right side of GA Funnel Visualization Reports so the argument for using a virtual page view in this context remains valid.

Update May 2014: I’ve changed my advice on this over the years. I think that the case for ‘events’ is now stronger than ‘pageviews’. For example events can now be seen mixed in with pageviews in the Behaviour Flow report. The case for ‘pageviews’ depends on two things: the fact that there’s an ‘exit rate’ calculated metric available in the behaviour/content reports and because virtual pageviews will appear as abandon routes on the right side of the Funnel Visualisation report. If you want the best possible data, the solution is to use both!]

Using either events or page views means that it is easy to configure the error messages as goals so that you can easily monitor the conversion rate. For example, I use a desktop client to monitor the conversion rates on the error goals across several sites. Each morning I can see at a glance if something is going wrong.

How to choose a pageview structure to get the best from the content drilldown report

I recommend structuring the page views carefully. I currently use a path for the virtual page view like:

/errors/checkout/deliveryoptions/’the error message’
/errors/checkout/payment/’the error message’
/errors/emailsignup/’the error message’

and so on.

[Update: But see the very helpful comment from Rob Kingston below, who recommends starting all virtual pageviews with something like /virtual/errors/etc so that you can exclude them from some profiles.]

This allows me to configure goals which consolidate groups of errors in a way which makes sense by matching the first part of the page name.

This approach also means that the Google Analytics Content Drilldown Report works very well and becomes a power tool for digging into the errors.

Configuring the last part of the page view as the actual text from the message makes things easier when coding but also has the added benefit that the reports show what we actually ‘said’ to the visitor. Simply seeing a list of all the unfriendly things we threw back at people, and how often, can be a wake up call.

Once you have this information, the first thing to do straight away is to fix the error messages if they are a bit on the harsh side. And then get stuck into working out why people are seeing them in the first place.

As usual, GA will help you prioritise. I would start by looking at the ones which happen most often and which are associated with higher exit rates.

Switching to the Google Analtics ‘performance view’ of the data is a great way of comparing those exit rates. In this example, postcode problems stand out as a sore point:

Information like this is like gold dust when you’re trying to find exactly where to concentrate your efforts to give maximum benefit to your visitors.

But please apply your knowledge of the site processes when you look at the abandon rate. The error message where people are leaving may turn out to be at the end of a series of other messages. Reducing the preceding errors would be the key to happier visitors. Custom segments might be useful here. Look at people who saw (or exited from) a particular error and see what other errors they also saw.

As is always the case when working in web analytics, you need to have to be inquisitive and use your skill and experience to gain the real insights. But those error messages certainly provide a very clear starting point and possibly a wake up call.

I’ll leave the question “How come all those people had a problem with their own last name?” hanging in the air for now!

Resources related to tracking error messages in Google Analytics

Direct Performance have taken this idea even further by coming up with a solution for using Google Analytics to track Javascript errors. Their article contains report examples, detailed instructions and sample code.

This approach does involve making changes to your main tracking code, but it looks well worth the effort to me. Javascript errors can be a real problem for your visitors — the site simply does not do what you intend it to do and the visitor gets a crude message over which you have no control. So spotting those errors and fixing them would be a great contribution to user experience.

http://www.directperformance.com.br/en/javascript-debug-simples-com-google-analytics

I’ve written a separate blog post on Using Goals and GA Custom Alerts for automatic email alerts about errors.

Updates

March 2015:
Mailchimp have a great blog post about their decision to remove social sign-in buttons which includes some very interesting data. In particular, they reveal that working with errors and copy had the biggest impact on reducing sign-in problems: [Opens in new tab] http://blog.mailchimp.com/social-login-buttons-arent-worth-it/

May 2014: In January 2014 I gave a talk on this subject which included examples of some custom reports based on this data. One day I’ll add some of those here as they contain some compelling examples of lost orders from within checkouts which result from unusual errors.

For now, here are the slides. They were created for an international audience so they include a lot of text for people who could not keep up with what I was saying, which means you can probably get a good understanding of the presentation now just from the slides.

The slides also cover the ‘events’ vs ‘virtual pageviews’ question. These days I think the way to go is ‘errors’ but probably with duplicate tracking via virtual pageviews just to allow for inclusion in GA Funnel Visualisation reports and to get the oh-so-useful ‘exit rate’ calculated metric. Meanwhile the ability to create segments which track visitors across multiple sessions has opened up an interesting new area for studying the longer term effects of errors.

This version skips right to the section showing example custom reports which are so useful for identifying the errors which are costing most in lost orders. In my experience you will often find there are some surprises in the list. And they may be fixable.

You can import the checkout errors custom report shown in the slides from the GA Gallery here:
https://www.google.com/analytics/gallery/#posts/search/%3F_.viewId%3DnHpEBtURS5-v1hUGfeh8xg

November 2012: Peter O’Neill has written a great example explaining how to use GA Custom Variables for error tracking. He’s included a couple of Custom Reports to download too. Whether you use virtual pageviews, custom dimensions, or events, the main point is the same: work out where your visitors are having problems and try to make things better! http://www.l3analytics.com/2012/11/21/tracking-errors-with-web-analytics/ [Opens in new tab]