If you want to know what to do to make your site better for your visitors, you need to know what they find bad. And one of the best indicators that someone is having a bad time is if they keep seeing annoying error messages.
- Why error messages are a shortcut to improving customer experience
- Why use virtual page views to track errors
- How to choose a pageview structure to get the best from the content drilldown report
- Resources related to tracking error messages in Google Analytics
March 2015: how working with errors and copy reduced Mailchimp’s sign-in problems
May 2014: Examples of custom reports, sequence segments, why events may be best — my Presentation from Superweek 2014
November 2012: Using Custom Variables to Track Errors
Why error messages are a shortcut to improving customer experience
Most error messages are triggered when someone is trying to do something, even if it’s only to click a link to a page which isn’t actually there.
So these people are already starting to engage with your site. And then they get a slap in the face.
Many error messages tell the visitor that they have done something wrong. It’s their fault.
These people may start to feel frustrated and possibly annoyed by your site.
That’s why it’s such a good idea to track error messages. They provide a very direct source of information about where your site is tripping people up and causing friction.
I say “your site” because it is probably the site’s fault not theirs. If people are entering the wrong information, or skipping required fields it’s often because the language or signposting on the site is weak.
OK: a typing mistake is a typing mistake — that’s the visitor’s fault, isn’t it? Fair enough. But if a lot of people are having trouble there, ask yourself: “do we really need that field, have we ever actually used that information”. Look at the exit rate from that error message and judge whether the information is worth it.
So: error messages are a great source of actionable insight on the exact points of friction on your site. The action is simple: do what you can to reduce the errors. I’d actually set up goals and measure the conversion rate.
Error messages are seldom page views, apart from the 404 “page not found” page.
Error messages often take the form of text and highlighting which indicates the ‘bad’ entry in a form (good). Others are just ugly alert boxes which pop up in the face of the visitor, with a beep, telling them they’ve screwed up, not telling them where, and waiting to be dismissed (not so good). Neither of those would normally register as a page view, so a web analytics tool like Google Analytics will know nothing about them.
What you need to do is add extra tracking code which is fired whenever the error message is shown. At the moment I favour using virtual page views for this in Google Analytics.
Why use virtual page views to track errors
There is an argument to say that errors should be tracked as ‘events’ not pageviews because they are an interaction on the site, not an actual page view. That’s debatable: the visitor certainly saw something!
Actually, when you track these you may find that some errors are repeated again and again. This is classic evidence to suggest that the visitor might not even have noticed the error message — for example a list of form errors in small red text displayed at the top of the page, way above the submit button down at the bottom of the screen. The visitor’s eyes will be looking where they just clicked. Tip: compare the pageviews and unique page views for errors to identify errors which are seen several time by the same person:
The real reason I favour virtual page views in GA is that, at the moment, only page views will appear in funnel reports as ‘abandon routes’ on the right side of the screen. Error messages in a checkout process would be a classic reason for abandoning a checkout. That’s something you definitely want to know about. So it’s best to have the information where you need it: in the checkout funnel report.
[Update March 2011: The version of Google Analytics released in March 2011 introduced the ability to use ‘Events’ as goals. This may be more appropriate for many error messages. There’s a link to an article giving instructions on using Events at the end of this post. However Events do not appear as abandon routes on the right side of GA Funnel Visualization Reports so the argument for using a virtual page view in this context remains valid.
Update May 2014: I’ve changed my advice on this over the years. I think that the case for ‘events’ is now stronger than ‘pageviews’. For example events can now be seen mixed in with pageviews in the Behaviour Flow report. The case for ‘pageviews’ depends on two things: the fact that there’s an ‘exit rate’ calculated metric available in the behaviour/content reports and because virtual pageviews will appear as abandon routes on the right side of the Funnel Visualisation report. If you want the best possible data, the solution is to use both!]
Using either events or page views means that it is easy to configure the error messages as goals so that you can easily monitor the conversion rate. For example, I use a desktop client to monitor the conversion rates on the error goals across several sites. Each morning I can see at a glance if something is going wrong.
How to choose a pageview structure to get the best from the content drilldown report
I recommend structuring the page views carefully. I currently use a path for the virtual page view like:
/errors/checkout/deliveryoptions/’the error message’
/errors/checkout/payment/’the error message’
/errors/emailsignup/’the error message’
and so on.
[Update: But see the very helpful comment from Rob Kingston below, who recommends starting all virtual pageviews with something like /virtual/errors/etc so that you can exclude them from some profiles.]
This allows me to configure goals which consolidate groups of errors in a way which makes sense by matching the first part of the page name.
This approach also means that the Google Analytics Content Drilldown Report works very well and becomes a power tool for digging into the errors.
Configuring the last part of the page view as the actual text from the message makes things easier when coding but also has the added benefit that the reports show what we actually ‘said’ to the visitor. Simply seeing a list of all the unfriendly things we threw back at people, and how often, can be a wake up call.
Once you have this information, the first thing to do straight away is to fix the error messages if they are a bit on the harsh side. And then get stuck into working out why people are seeing them in the first place.
As usual, GA will help you prioritise. I would start by looking at the ones which happen most often and which are associated with higher exit rates.
Switching to the Google Analtics ‘performance view’ of the data is a great way of comparing those exit rates. In this example, postcode problems stand out as a sore point:
Information like this is like gold dust when you’re trying to find exactly where to concentrate your efforts to give maximum benefit to your visitors.
But please apply your knowledge of the site processes when you look at the abandon rate. The error message where people are leaving may turn out to be at the end of a series of other messages. Reducing the preceding errors would be the key to happier visitors. Custom segments might be useful here. Look at people who saw (or exited from) a particular error and see what other errors they also saw.
As is always the case when working in web analytics, you need to have to be inquisitive and use your skill and experience to gain the real insights. But those error messages certainly provide a very clear starting point and possibly a wake up call.
I’ll leave the question “How come all those people had a problem with their own last name?” hanging in the air for now!
Resources related to tracking error messages in Google Analytics
I’ve written a separate blog post on Using Goals and GA Custom Alerts for automatic email alerts about errors.
Mailchimp have a great blog post about their decision to remove social sign-in buttons which includes some very interesting data. In particular, they reveal that working with errors and copy had the biggest impact on reducing sign-in problems: [Opens in new tab] http://blog.mailchimp.com/social-login-buttons-arent-worth-it/
May 2014: In January 2014 I gave a talk on this subject which included examples of some custom reports based on this data. One day I’ll add some of those here as they contain some compelling examples of lost orders from within checkouts which result from unusual errors.
For now, here are the slides. They were created for an international audience so they include a lot of text for people who could not keep up with what I was saying, which means you can probably get a good understanding of the presentation now just from the slides.
The slides also cover the ‘events’ vs ‘virtual pageviews’ question. These days I think the way to go is ‘errors’ but probably with duplicate tracking via virtual pageviews just to allow for inclusion in GA Funnel Visualisation reports and to get the oh-so-useful ‘exit rate’ calculated metric. Meanwhile the ability to create segments which track visitors across multiple sessions has opened up an interesting new area for studying the longer term effects of errors.
This version skips right to the section showing example custom reports which are so useful for identifying the errors which are costing most in lost orders. In my experience you will often find there are some surprises in the list. And they may be fixable.
You can import the checkout errors custom report shown in the slides from the GA Gallery here:
November 2012: Peter O’Neill has written a great example explaining how to use GA Custom Variables for error tracking. He’s included a couple of Custom Reports to download too. Whether you use virtual pageviews, custom dimensions, or events, the main point is the same: work out where your visitors are having problems and try to make things better! http://www.l3analytics.com/2012/11/21/tracking-errors-with-web-analytics/ [Opens in new tab]