Using Google Analytics to Find Browser Bugs

In June, one of our SEO clients had some development work done by a third party vendor. Once the site update was launched, it was our job to monitor the changes in indexing and make sure that everything was running smooth in the engines.  The work that had been done was on an e-commerce site, so you can imagine how important it is that things go smoothly.

After a couple weeks of collecting data we realized that our sales numbers were low. Given our recent increase in traffic we knew something was wrong. So, time to dig into Google Analytics. 

Diving into Google Analytics

First we ran down all of the steps in our conversion funnel to try and identify were the hitch was.  Our bounce rate was fine, in fact it had improved over the past weeks.  It turned out that we had no major drop in page to page conversion when looking at our whole userbase.

Since we were not able to identify any major problems in our gross user base, the next step was to break down the users into groups that would have specific bugs. We decided to split groups into browsers because that would address any CSS or Javascript errors.

At first glance all browsers appeared to be fine at the top of the funnel, however on closer inspection it turned out that IE6 was not converting compared to pre-launch.  Upon closer inspection it turned out IE6 was having issues during the checkout phase.

Needless to say, once the problem was pointed out to the vender, it was fixed right away.

Lessons Learned

When working with a third party, always do independent testing.

It is important that when you work with a third party vendor that you make sure there testing methods live up to your standards.  It is natural to assume that a company will catch there own bugs; however, the truth is that bugs can pop out of nowhere, which is why it is important to constantly test for them.

Continually test and retest

Just because something is live and has passed internal testing doesn’t mean that it is foolproof.  Make it a habit to constantly check your sites in all browser, as sometimes a bug can slip through.

Always browser test, even small changes.

Just because the layout looks okay on IE6 doesn’t mean that all functions work right.  Justin can attest to the fact that browsers can be just as much trouble for JavaScript as CSS.  Even if you only are changing HTML, adding new content, etc  check the page in various browser.  It is better to spend extra before the bug happens than after it goes live.

Dig DEEP into Analytics at least once a month.

Many SEO’s get stuck in the habit of checking Analytics at least daily. This is a good practice for catching major changes when they first show up.  However, if all you do is check bounce, traffic and referrals, then you can miss out on some major chances to increase conversion. Narrow down your user sets starting with large groups and continue trimming until you are as specific as possible.  It is important, however, that your user groups are large enough to account for margin of error and randomness.

A quick comment on bugs and testing

It is important to remember that bugs do happen.  Yes, as developers, it is our job to polish a product as much as possible before it goes live.  But it is equally as important to keep tabs on a project post-launch so that any minor bugs can be fixed the moment they pop up.  One of the things about designing a website, as compared to print media, is that we create in a world with changing context and parameters.  It is important as designers and developers that we accept this and learn to roll with the punches instead of getting frustrated with old browsers.  That said, if you are still using Internet Explorer 6 or 7, please upgrade.

In what ways has deep site analysis helped you improve your site?