Let Google Analytics tell you when your visitors are having a bad experience

by Tim Leighton-Boyce

Screen shot of Classic Facebook error messageIf you want to know what to do to make your site better for your visitors, you need to know what they find bad. And one of the best indicators that someone is having a bad time is if they keep seeing annoying error messages.

Why error messages are a shortcut to improving customer experience

Most error messages are triggered when someone is trying to do something, even if it’s only to click a link to a page which isn’t actually there.

So these people are already starting to engage with your site. And then they get a slap in the face.

Many error messages tell the visitor that they have done something wrong. It’s their fault.

These people may start to feel frustrated and possibly annoyed by your site.

That’s why it’s such a good idea to track error messages. They provide a very direct source of information about where your site is tripping people up and causing friction.

I say “your site” because it is probably the site’s fault not theirs. If people are entering the wrong information, or skipping required fields it’s often because the language or signposting on the site is weak.

OK: a typing mistake is a typing mistake — that’s the visitor’s fault, isn’t it? Fair enough. But if a lot of people are having trouble there, ask yourself: “do we really need that field, have we ever actually used that information”. Look at the exit rate from that error message and judge whether the information is worth it.

So: error messages are a great source of actionable insight on the exact points of friction on your site. The action is simple: do what you can to reduce the errors. I’d actually set up goals and measure the conversion rate.

The snag?

Error messages are seldom page views, apart from the 404 “page not found” page.

Error messages often take the form of text and highlighting which indicates the ‘bad’ entry in a form (good). Others are just ugly alert boxes which pop up in the face of the visitor, with a beep, telling them they’ve screwed up, not telling them where, and waiting to be dismissed (not so good). Neither of those would normally register as a page view, so a web analytics tool like Google Analytics will know nothing about them.

What you need to do is add extra tracking code which is fired whenever the error message is shown. At the moment I favour using virtual page views for this in Google Analytics.

Why use virtual page views to track errors

There is an argument to say that errors should be tracked as ‘events’ not pageviews because they are an interaction on the site, not an actual page view. That’s debatable: the visitor certainly saw something!

Actually, when you track these you may find that some errors are repeated again and again. This is classic evidence to suggest that the visitor might not even have noticed the error message — for example a list of form errors in small red text displayed at the top of the page, way above the submit button down at the bottom of the screen. The visitor’s eyes will be looking where they just clicked. Tip: compare the pageviews and unique page views for errors to identify errors which are seen several time by the same person:

Screen shot from GA Content Drilldown report showing multiple views of same error message

The real reason I favour virtual page views in GA is that, at the moment, only page views will appear in funnel reports as ‘abandon routes’ on the right side of the screen. Error messages in a checkout process would be a classic reason for abandoning a checkout. That’s something you definitely want to know about. So it’s best to have the information where you need it: in the checkout funnel report.

[Update March 2011: The version of Google Analytics released in March 2011 introduced the ability to use ‘Events’ as goals. This may be more appropriate for many error messages. There’s a link to an article giving instructions on using Events at the end of this post. However Events do not appear as abandon routes on the right side of GA Funnel Visualization Reports so the argument for using a virtual page view in this context remains valid.

Update May 2014: I’ve changed my advice on this over the years. I think that the case for ‘events’ is now stronger than ‘pageviews’. For example events can now be seen mixed in with pageviews in the Behaviour Flow report. The case for ‘pageviews’ depends on two things: the fact that there’s an ‘exit rate’ calculated metric available in the behaviour/content reports and because virtual pageviews will appear as abandon routes on the right side of the Funnel Visualisation report. If you want the best possible data, the solution is to use both!]

Using either events or page views means that it is easy to configure the error messages as goals so that you can easily monitor the conversion rate. For example, I use a desktop client to monitor the conversion rates on the error goals across several sites. Each morning I can see at a glance if something is going wrong.

How to choose a pageview structure to get the best from the content drilldown report

I recommend structuring the page views carefully. I currently use a path for the virtual page view like:

/errors/checkout/deliveryoptions/’the error message’
/errors/checkout/payment/’the error message’
/errors/emailsignup/’the error message’

and so on.

[Update: But see the very helpful comment from Rob Kingston below, who recommends starting all virtual pageviews with something like /virtual/errors/etc so that you can exclude them from some profiles.]

This allows me to configure goals which consolidate groups of errors in a way which makes sense by matching the first part of the page name.

This approach also means that the Google Analytics Content Drilldown Report works very well and becomes a power tool for digging into the errors.

Configuring the last part of the page view as the actual text from the message makes things easier when coding but also has the added benefit that the reports show what we actually ‘said’ to the visitor. Simply seeing a list of all the unfriendly things we threw back at people, and how often, can be a wake up call.

Once you have this information, the first thing to do straight away is to fix the error messages if they are a bit on the harsh side. And then get stuck into working out why people are seeing them in the first place.

As usual, GA will help you prioritise. I would start by looking at the ones which happen most often and which are associated with higher exit rates.

Switching to the Google Analtics ‘performance view’ of the data is a great way of comparing those exit rates. In this example, postcode problems stand out as a sore point:

Screen shot showing Google Analytics performance view of exit rate from error

Information like this is like gold dust when you’re trying to find exactly where to concentrate your efforts to give maximum benefit to your visitors.

But please apply your knowledge of the site processes when you look at the abandon rate. The error message where people are leaving may turn out to be at the end of a series of other messages. Reducing the preceding errors would be the key to happier visitors. Custom segments might be useful here. Look at people who saw (or exited from) a particular error and see what other errors they also saw.

As is always the case when working in web analytics, you need to have to be inquisitive and use your skill and experience to gain the real insights. But those error messages certainly provide a very clear starting point and possibly a wake up call.

I’ll leave the question “How come all those people had a problem with their own last name?” hanging in the air for now!

Direct Performance have taken this idea even further by coming up with a solution for using Google Analytics to track Javascript errors. Their article contains report examples, detailed instructions and sample code.

This approach does involve making changes to your main tracking code, but it looks well worth the effort to me. Javascript errors can be a real problem for your visitors — the site simply does not do what you intend it to do and the visitor gets a crude message over which you have no control. So spotting those errors and fixing them would be a great contribution to user experience.

http://www.directperformance.com.br/en/javascript-debug-simples-com-google-analytics

I’ve written a separate blog post on Using Goals and GA Custom Alerts for automatic email alerts about errors.

Updates

[November 2012] Peter O’Neill has written a great example explaining how to use GA Custom Variables for error tracking. He’s included a couple of Custom Reports to download too. Whether you use virtual pageviews, custom dimensions, or events, the main point is the same: work out where your visitors are having problems and try to make things better! http://www.l3analytics.com/2012/11/21/tracking-errors-with-web-analytics/ [Opens in new tab]

[May 2014] In January 2014 I gave a talk on this subject which included examples of some custom reports based on this data. One day I’ll add some of those here as they contain some compelling examples of lost orders from within checkouts which result from unusual errors.

For now, here are the slides. They were created for an international audience so they include a lot of text for people who could not keep up with what I was saying, which means you can probably get a good understanding of the presentation now just from the slides.

The slides also cover the ‘events’ vs ‘virtual pageviews’ question. These days I think the way to go is ‘errors’ but probably with duplicate tracking via virtual pageviews just to allow for inclusion in GA Funnel Visualisation reports and to get the oh-so-useful ‘exit rate’ calculated metric. Meanwhile the ability to create segments which track visitors across multiple sessions has opened up an interesting new area for studying the longer term effects of errors.

This version skips right to the section showing example custom reports which are so useful for identifying the errors which are costing most in lost orders. In my experience you will often find there are some surprises in the list. And they may be fixable.

You can import the checkout errors custom report shown in the slides from the GA Gallery here:
https://www.google.com/analytics/gallery/#posts/search/%3F_.viewId%3DnHpEBtURS5-v1hUGfeh8xg

{ 13 comments… read them below or add one }

Tim Leighton-Boyce August 25, 2010 at 4:31 pm

I’ve just seen a great post over on the GetElastic ecommerce blog all about login error messages. There are loads of great screen shots which form a kind of gallery of shame.

Definitely one to read:

http://www.getelastic.com/does-your-log-in-make-them-drop-off/

Fran September 6, 2010 at 8:37 am

Very interesting, but how do you do that? Could you please give us an example with the javascript code needed?

Tim Leighton-Boyce September 6, 2010 at 11:23 am

@Fran: There’s a great example in the official help files which shows how to track the 404 page and also report on where the source of the ‘bad’ link was as well as any parameters included:

https://www.google.com/support/analytics/bin/answer.py?hl=en&answer=86927

You could apply the same approach for any error message, but the details of obtaining the dynamic information, such as the content of the error message, would be specific to your site and so I cannot provide actual code for that.

Christopher September 8, 2010 at 9:52 pm

Could this functionality be implemented on a form page that posts back to itself when an error is thrown?

Tim Leighton-Boyce September 9, 2010 at 9:00 am

@Christopher: Yes, I would expect so. If I understand you correctly, I assume that you’re describing some form of internal validation. If that’s the case, then when you display the error message to the visitor you would also fire the virtual pageview tracking call back to GA. That’s exactly the kind of thing we do on the sites I work on.

Christopher September 9, 2010 at 11:16 pm

Yep – client-side validation. Seeing these “error pageviews” as abandon routes in funnel visualization would be very helpful. And I’m assuming as long as these virtual pageviews are triggered using a bit of js in a form input, page view counts for the whole page itself aren’t affected?
Thanks Tim!

Tim Leighton-Boyce September 10, 2010 at 8:42 am

Yes, that’s right. You will record two page views: the original form page as normal and then the extra ‘error’ page view. This is the reason why some people prefer to use an event, not a pageview. Using pageviews means that you will increase the average numbers of page views per session and so on.

That’s a choice you have to make.

I tend towards pageviews mostly because they appear in funnel reports. That’s the essential information I need in order to make improvements to the site. I’m also don’t have a problem with counting these as a page view, anyway: if the visitor is stuck on the form and seeing a series of error message screens of some kind, they are seeing more/different stuff and may well feel that they have seen more (bad) content than someone who went straight through.

Rob Kingston October 19, 2010 at 11:45 pm

Hi Tim,

Great idea. I think if you’re going to use virtual pageviews in this way throughout your site, the best pageview structure to adopt would be

/virtual/errors/{location_path}/{error_name}

That way you can easily exclude all virtual pageviews from certain profiles with a simple custom filter where you just need to exclude /virtual/. That way if you’re using virtual pageviews to track form completion along with errors and other bits and pieces it makes it easy to exclude anything custom.

Just a little preference of mine…

Tim Leighton-Boyce October 20, 2010 at 8:39 am

@Rob: Thanks for the suggestion. You’re so right: that’s a much better way of doing it. The ability to have profiles in which the pageview data only shows real pages is very valuable. Even more so now that the site overlay report has been improved and turned into a powerful tool.

I’ll edit the original post to draw attention to your point.

Tim Leighton-Boyce October 23, 2010 at 9:37 am

I’ve just seen another really good post from VKI specifically on the subject of working with 404 errors in GA. There are also some very good tips on how to handle the errors themselves — what to show on the 404 page itself, which pages to 301 redirect and so on.

http://blog.vkistudios.com/index.cfm/2010/3/24/Fixing-your-404s-with-Google-Analytics

Tim Leighton-Boyce March 21, 2011 at 11:20 am

The new version of Google Analytics released in March 2011 has introduced the possibility of using ‘Events’ as goals. This means that the advice about using virtual pageviews, rather than ‘Events’ for error tracking needs to change.

The situation is now more complex. In many cases I think it would now make most sense to use an ‘Event’. However if the error is one which is part of a process which has been configured as a GA funnel, than I think it may still be more useful to stick with virtual pageviews.

I’ve edited the original post here to include some first thoughts on the subject.

Tim Leighton-Boyce April 15, 2011 at 10:38 am

I’ve just realised that this post doesn’t mention one of the reasons for configuring critical error pages as goals.

The Google Analytics Reverse Goal Path Report, which is now much easier to read in the new v5 release, works really well with goals for which you are unable to predict the intended path to the goal.

Screenshot showing Google Analytics Reverse Goal Path Report for 404 page

Some error messages will only ever be seen in one location, such as form validation. But even with those I know from bitter experience that it can be hard to nail down something like an error which can never be replicated, unless you have first done this, then done that, and then seen this other error message.

Many other errors can pop up all over the place, so being able to see the three pages viewed before the error can be extremely helpful.

Tim Leighton-Boyce May 19, 2011 at 6:52 am

I’ve just seen another great post on this theme by Daniel Waisberg. It contains great suggestions about creating better error pages as well as detailed instructions on reporting on the errors.

http://www.smashingmagazine.com/2011/05/18/optimizing-error-pages-creating-opportunities-out-of-mistakes/

Leave a Comment

Comment Spam Protection by WP-SpamFree

{ 4 trackbacks }

Previous post:

Next post: