Newfangled works with independent agencies to create lead development web platforms for their clients.

Newfangled works with independent agencies to create lead development web platforms for their clients.

Using Advanced Measurement Techniques to Tell the Real Story



By advanced, I don't mean anything that requires the help of a web developer, or any other special configuration beyond simply placing Google Analytic's script on your site. There's plenty of information on our site (and the rest of the web) on various tracking techniques, goal funneling, custom reports, and the like, but this article is about taking your measurement of the basic metrics—visits, bounce rate, referrers, and conversions—to a more advanced level...

Gathering Data

Google Analytics does a great job of presenting a vast array of data in many meaningful reports, but those reports can also be confusing or not quite configured to tell the particular story you're looking for. With that in mind, I'm often inclined to pull the data out of Google Analytics and into another context in order to make the connections I need to see in order to discern what's actually going on with our site's traffic. Fortunately, Google Analytics offers the ability to export any report—in PDF, XML, CSV, TSV, or Excel format—a function you can access at the top of the page, as shown below.

export.jpg

For the purposes of this article, I wanted to look at a large sample of data. So, after exporting the data I needed from Google Analyics, which came from applying an advanced segment for goal conversions to three different reports, the Top Content, Top Landing Pages, and Top Traffic Sources, I put together my own spreadsheet to see how this information might enable me to draw conclusions about the effectiveness of our site in converting traffic over the past two years. You can view this data in its raw form below:

source_material.jpg
Click image to enlarge.

I can't emphasize enough the benefit of exporting Google Analytics data. There are connections you just won't be able to make, particularly if you've been doing basic website measurement for a few years now and are ready to go deeper, by only viewing isolated reports. Sometimes, in order to answer the more meaningful questions you have about your site, you'll need to create your own reports that draw from multiple analytics sources. It was by doing this that I came to an interesting conclusion about how referral traffic converts, which I'll share with you at the end of this article. But first, I'd like to clear a few things up about bounce rates and show how the principle of properly gathering your data will help you to better understand how your pages are actually performing.

Understanding Bounce Rate

The bounce rate measures the percentage of visitors to a web page that leave it without following links to any other pages on the site. This means that a bounce can be registered if a user remains idle on a page long enough for his or her session to be terminated (30 minutes or more), or if they click a link to a different URL that is not configured to open in a new window, or if they close the browser window outright. But the most important principle to remember is this: the bounce rate only applies to traffic that landed on a page. This means that in order to determine the exact number of bounces from a particular page, you must combine data from two different reports in Google Analytics—the Top Content Report (or Content Detail Report) and the Top Landing Pages report.

Google Analytics' Top Content and Content Detail reports list the bounce rate along with the total number of unique visits but exclude landing data, creating the false impression that the bounce rate should be applied to the total number of visits. However, the bounce rate should only be applied to the number of visitors that landed on the page—often a much lower number than the total visits, which means that the bounce rate is likely lower than you might initially conclude.

planning.jpg

The page I've decided to use as an example is one of the most important pages on our website —one that describes our prototyping process1. It's one of only 12 pages in our main navigation and is the first in a section specifically identifying key aspects of our services and approach to web development. Because it describes the unique way we approach web development protoyping, its performance is a good indicator of how successful our website is in communicating what we do to potential clients. Therefore, I put it in a category of pages I call "positioning pages" — pages I pay very close attention to. Below are the details:

top_content_top_landing.jpg

By merging data from both the Top Content and Top Landing reports (shown above), I can create a new, custom report of my own that will help me to better understand the traffic to this page (shown below):

conclusions.jpg

The Top Content report tells me that a total of 2,541 unique visitors have come to this page since we launched the redesign of our website on January 12, 2010. I can assume that most of the 2,541 visitors arrived here after viewing another page on our site, but without checking the Top Landing Pages report, I won't know the exact number of them that arrived to our site on this page—893 visitors. Now, between these two reports, I want to be able to reconstruct the story of all the visitors to this page. I can do this by focusing in on the visit numbers, the bounce rate (47%), and also the percentage of visitors that exited the site after viewing this page (28.56%, which I'll round up to 29%).

My first step is to subtract the number of visitors who landed on this page from the total number of unique views of the page, which leaves me with 1,648. I apply the exit percentage to this number, which gives me 478—the number of visitors to this page who came to it from another page on our website, but then decided to leave. This is different from the bounce rate, which, as I mentioned before, identifies the portion of landing traffic that left the site without viewing any other pages on it. To determine the number of bounces, I apply the rate (47%) to the number of users who landed on this page (893), which gives me 420 bounces. By doing this bit of math, I now know that out of the total 2,541 visitors to this page since we relaunched our site, 893 entered our site for the first time on it (landed), 420 left right away (bounced), 478 left our site from this page after having viewed other pages (exited), and 1,170 continued from it on to other pages. Now 420 is only 17% of the total traffic to this page, leaving 2,121 others that at least saw additional content on our site (83%). That's a much better reality than a quick glance at the reports indicates!

What Does this Mean for Conversions?
If I apply my "sessions with goals" segment to both the Top Landing Pages report and the Content Detail report, I can get an even clearer picture of how traffic to this page performed. In total, 897 visitors of this page (35%) also completed goals. The Top Landing pages report tells me that 123 of these were completed by visitors that entered the site on this page. That leaves the remaining goals (774) completed by those non-landing visitors (1,648). If I parse these two amounts, I can conclude that 14% of landing traffic to this page converts, while 47% of visitors that viewed this page among others during their session converted. That's a massive difference, and a much more meaningful number than just looking at the overall conversion rate applied to the total visits. From these numbers, I surmise that the message of this page is much more compelling when additional pages about what we do are viewed. The more a user reads, the more likely they are to convert.

The Value of Referrals

full_graph.jpg

Looking at what visitors do once they've arrived at your site, which we've done by focusing in on the traffic data for an example page above, tells you a lot about how your content enables certain user behavior—particularly conversions. But that's only one piece of the puzzle. To understand even more about why some visitors convert, we need to look at where they came from.

In the graph above, I've plotted the top 10 referrers to our website over the past two years. (Again, I'm working from the data I assembled from various reports in Google Analytics and reconstructed in my own spreadsheet, which I included in the "Gathering Data" section above.) I think the problem with this kind of visualization is pretty clear: because the top contributing referrer (organic search) brings almost 5 times the visitors as the next one (direct visits), the graph portrays the remaining 8 referrers as contributing relatively the same amount of traffic. Consequently, it could be simplified down to direct traffic versus everything else. But that isn't a very helpful story, especially in light of the fact that despite bringing huge numbers of traffic to our site, organic search performs dismally when it comes to conversions.

The Familiarity Spectrum

Here's where things get interesting. For a site like ours, which receives quite a lot of referral traffic from search engines, simply plotting out the top ten referrers is pretty much a waste of time. We've got thousands of pages on our site, which bring in visitors with all kinds of interests—a very small sample of which are actually looking to learn more about a web development company. So it's no surprise that the percentage of visitors referred by search engines that convert would be low compared to other sources. Last October, I wrote an article on this very subject and included a graph of six months of referral traffic to our site that really emphasized the point that more conversions come from human referrals than from search engines2. The principle makes perfect sense when you think about it—and remember, we're talking about organic search engine traffic here—people trust people and people act upon trust.

traffic_vs_conversions_2.jpg

But I think there's an important nuance to that principle that's worth considering. When I plot out the top referrers to our site, the general point still stands: Organic search traffic accounts for the largest portion of incoming traffic, but a comparatively small amount of conversions (.3% in 2008/2009 and .7% in 2009/2010), while LinkedIn brings far fewer visitors but performs much better in terms of conversions (2% last year, 5% this year). But then I noticed something else. Smashing Magazine has referred to us a few times over the past two years—not enough to make the top 15 I pulled in my report last year, but plenty this year—and is proportionately one of our most frequently converted referrers (22%). But our own newsletter blasts, which offer conversion opportunities by linking to upcoming webinars or being forwarded to non-subscribers, fall below Smashing when it comes to conversions (9%). In the grand scheme of things, its performance is still great. I consider anything above 1% as worthwhile. But, it's less than Smashing despite targeting an audience that is far more familiar with us and what we do than Smashing Magazine's diverse audience of over 200,000 readers. Why is this? Here's where the familiarity spectrum comes in.

scale.jpg

At the top of the familiarity spectrum are those referrers that are the most unfamiliar with their own audience, like search engines. Search engines are robots and, though they're used by millions to find information every day, their "recommendations" drop in value compared to those users receive from other human beings. LinkedIn, on the other hand, is a social network that is shaped by its users—where proximity in connection tends to increase familiarity. We have a few thousand participants in our Web Development for Advertising Agencies group, and can reach a bit further in our "network" by asking and answering questions and polls, so the familiarity here between participants and others that recommend our content is obviously higher than search engines. Smashing Magazine, on the other hand, tends to have a more brand-loyalty connection with its readers. Like any magazine, Smashing's readers trust them to give valuable information and recommendations and are likely to act in response. This is evident from the participation level of the readers on Smashing's own site, and from the value of the traffic we've received from them—the familiarity here is what keeps the magazine afloat. At the bottom of the spectrum is our own subscriber base, who are as familiar with us as anyone could be without being a client or employee! They already converted in the past—when they signed up to receive our newsletter—and it's going to take a bit more for them to convert again. Think about it this way: given the choice of professional recommendations from a robot, a stranger, the president, or your grandmother, which one are you going to act upon?

What to Do?

Look at your own referrers. You've probably got some robot-referred activity (organic search), some semi-anonymous social network stuff (LinkedIn, Twitter, Facebook, etc.), and probably even some from your own marketing initiatives, like our newsletter blasts. But what about more influential recommendations? You may not be able to get a nod from Barack Obama, but I'm sure there are influencers in your industry or network that would love to point people your way. If you haven't already, seek them out. Don't ask for a link (that's so 1998), but build a relationship with them so that the referral is implicit in all kinds of activity between you, whether that be social network activity, participation in blog conversations, or something else. You may even be able to deepen that relationship by writing articles or creating other content for them, like we've done with Smashing Magazine3.

Put simply, don't expect to re-convert the already converted. Focus your efforts on spreading new seed, in the most fertile ground you can find. The "sweet spot" of the familiarity spectrum is going to be just shy of the most familiar audience you have, probably among those connected strongly to an influencer with whom you could build a strong, mutually-beneficial relationship. Looking closely at your conversion data, using reports like those I've reviewed in this article, should help you to pinpoint who that might be.

Footnotes

  1. Our page on the Web Development Prototyping Process.
  2. This comes from a page of our October, 2009 newsletter, Who Are You Speaking To? discussing measuring the conversion rate of referral traffic.
  3. Last August I noticed that a Smashing Magazine article recommending our newsletter was generating great referral traffic to our site. Shortly afterward, I began speaking with their editor about writing some articles, the first of which, Holistic Web Browsing: Trends of the Future, was published in April.




Comments

Ed | June 1, 2010 7:34 PM
"...which means that the bounce rate is likely lower than you might initially conclude."

The rate doesn't ever change, it's a percentage, so it's always going to be that percentage. Google's conversion university people say that a bounce rate over 40% is problemmatic, probably why they put that metric front and center, right? It doesn't really matter how many people come to the page. If 40% of them are leaving, that's a problem. I like the analysis, but aren't you just creating a handicap?
Chris Butler | June 1, 2010 7:52 PM
Ed,

What I probably should have written is that the number of bounces is likely lower than you might initially conclude by seeing the percentage displayed next to the total number of visits. This is an important fact. Let's say the bounce rate is 50% for a page that received 2000 total unique visits. You would be wrong to conclude that 1000 visitors bounced. You would need to look at the landing page report to see how many visitors during the same time frame came in to your website by landing on that page. That is a different number, which will be less than the total number of visits. Let's say, again, hypothetically, that the number of landings was 300. This means that the number of bounces was actually 150, not 1000. A massive difference! That is my point.

So, no, I'm not trying to prescribe a handicap. In my opinion, bounce rate is only relevant when translated to a real number of visitors. Aiming for a percentage, like 40%, simply because other people say it's a good number is mistaken. There are too many different kinds of websites—of vastly different sizes—for that to be a realistic goal. A site like ours receives far too much traffic to a much-too-large number of unique pages for a site-wide bounce rate of 40% to be realistic. In fact, I might argue that the more content you create, no matter how focused you believe that content to be, the higher the site-wide bounce rate. There's far too much search engine traffic across the web for that not to be true. But we can debate that point...

In any case, you ended by saying that "it doesn't really matter how many people come to the page." I completely disagree. Concrete numbers do matter. Percentages don't. I want to know which visitors bounce and which convert, so the hard numbers matter. Looking simply at a percentage and saying, well, it's higher than 40%, therefore it's bad, is exactly the kind of "measurement" that I'm trying to steer our clients away from by writing this article.

Chris
edaviesc | June 2, 2010 10:49 AM
Interesting article, seems you are losing visitors
Chris Butler | June 2, 2010 11:07 AM
edaviesc: We definitely are, and that's a good thing as far as I'm concerned. A couple of years ago, we began pursuing a goal to serve fewer clients at a higher level. This meant focusing our positioning and making sure that our marketing took its cues from that, rather than everything we could potentially talk about via various forms of content. This newly focused content strategy had an impact on the search engine optimization of our site, particularly as we focused on matching meta data to search intent. The end result, ideally, would be content that is more compelling to our prospects at the expense of some overall traffic attrition. After all, those higher visit numbers were supported by visitors that weren't prospects anyway.

Incidentally, Alan Beberwyck of Wanderlust, one of our agency partners, hit on this concept in an article he released just this morning. He's writing to the tourism industry, but his point about increasing the effectiveness of marketing by limiting the audience is indicative of this idea propagating across industries:
Today we’re creating more content for fewer people, and spending less to deliver each piece. This is what some insiders are calling ‘narrowcasting.’ Instead of dedicating the bulk of marketing spend to mass media channels such as network television and large circulation print vehicles, we’re diverting increasing dollars by investing in the brand experience, creating our own channels and focusing on tactical executions in a variety of media.
I'm glad to see our industry, and the niches attached to it, moving toward a "doing more with less" mentality. And as I mentioned to Ed above, that is the kind of measurement we are encouraging our clients to be about: less concerned with traffic numbers as and end in themselves and more about the value of specific user engagement.
MaggieB | June 2, 2010 11:33 AM
Thanks for this tutorial. What a great resource!
Alan Wynn | June 2, 2010 2:32 PM
Truly fascinating stuff. But realistically, I'm left frustrated. If you can't get a clear picture from GA, then what good is it? I've always thought that the benefit of analytics tools like it was in saving people the time they'd otherwise spend rooting through piles of data by hand and filtering out conclusions. I don't have the time to do more than monitor our GA acct with the assumption that they're reports are the most important ones to pay attention to. I need to be able to quickly log in and just see whether or not the site is performing.
Jesse | June 2, 2010 4:22 PM
I'm putting this all together. My only question is about the point you made in the bounce rate section: "The more a user reads, the more likely they are to convert." If conversions come from deeper sessions, how do we get users to read more pages per visit?
Alexandra | June 2, 2010 4:50 PM
@Ed, @Chris, @Jesse, I think one area of confusion with the whole bounce rate thread is that sitewide site bounce rate and individual page bounce rate are two very different things. (I'm linking to a marketing jive post that I found helpful), but for a smaller site like mine, my overall bounce rate matters more. For a larger site like this one, it's gonna be be higher because the overall rate is just an average of all the other pages' bounce rates.

The marketing jive post points out that a page's bounce rate is good or bad in terms of what the page's purpose is. A campaign landing page with a bounce rate over 35% is probably not good, but a blog post or something with the same rate would be doing just fine.

But the big point is you can't just look for a number and say good or bad. Page performance scales aren't binary systems, like on or off, 1 or 0. There has to be a deeper kind of thinking here, otherwise we're doing the same thing as teaching to the test.
Mark O'Brien | June 2, 2010 5:32 PM
Chris,

The part about this newsletter that I find to be most interesting is the idea you raise about the "Familiarity Spectrum." I think the idea of focusing one's online marketing efforts on the removed sources of trusted authority is a really brilliant one that is going to prove to be a pretty important principle of online marketing in the coming years. I look forward to hearing more about that in the future, and to Newfangled learning more about this directly through our marketing efforts.

In regard to the bounce rate thread, we've learned that quality is better than quantity. There has been a lot of content on our site that people found interesting and that did great with search engines but didn't really bring in people who were necessarily looking to hire a web developer. Chris's efforts have been very focused on writing content that is likely to bring more accurate prospects to our site, although we still love being an educational resource for anyone that might be interested. Chris has also removed a lot of the content on our site that brought in a ton of traffic, but was mostly one-off traffic.

@Jesse, one big way we've tried to get people to view more pages on our site is the "Related Content" widget you see on this page's sidebar. This has been successful for us, although we're always looking for more ways to further improve the site.

Great job, Chris, I really enjoyed reading this newsletter.
Chris Butler | June 2, 2010 5:44 PM
MaggieB: I'm glad it was helpful. Thanks, as always, for the compliment!

Alan Wynn: I understand. I think that the reality is that measurement worth doing is just more work than most people (myself included) realize. In fact, doing this assessment took much longer than I anticipated. But isn't that true of most things? Measurement is partly a methodical practice that you can allocate set amounts of time toward—particularly the gathering of data portion. But it's also a more research-like task that can be very difficult to estimate in terms of how much time you'll need to do it properly. The other thing to keep in mind is that a "deeper" assessment like the one I'm describing in this article wouldn't be necessary to do that frequently. It's more of a quarterly-level routine than a weekly or daily one. Your overall point though, is indicative of what I've observed among my own network: that measurement requires more time than most allocate. The larger the company/site, the more likely it is to be a full time role for an analyst. In light of your feelings though, I'd like to invoke some WWII-era British propaganda: "Keep Calm and Carry On."

Jesse: That's a very good question, and the answer is not a simple one. Sure, you've probably read things like "write quality content," but what does that mean? I think the more relevant question to ask (that I think will get you to where you want to be) is "Who are you trying to speak to?" The kind of content you need to create will naturally flow from that—actually identifying, based upon your company's positioning, who your prospects are and what they're looking to learn or hear from you. This process of persona development is something we've dealt with in past articles (How to Create Valuable Content and The Web Development Planning Process," as well as a recent webinar, "Using Personas to Build Better Websites"), so I'd encourage you to check those out. The functionality that Mark mentioned only works well because of a taxonomical structure behind the scenes that links this content based upon tags we apply to it, so theoretically, that related content should actually be directly related to what you're reading. It's not something that we manually override so that we can ensure that certain pages are seen more. That aside, I think that without doing the right planning work in terms of identifying your audience, there isn't really any mechanical way to increase the depth of your visitors' sessions that isn't manipulative in some way.

Alexandra: Yes, the distinction you raise is critical. For the most part, I'm talking about page-specific bounce rate in this article. Site-wide bounce rate, especially for a larger site with a widespread audience, is likely to be higher than 35-40%.

Mark: Glad you liked it—I aim to please ;-) I'm also interested in keeping an eye on the familiarity spectrum principle. This is something that I hit upon pretty recently, that seems to be pretty consistently true across various date ranges for which we have available data. However, I can't say for sure that it's true for everyone else, though I'm leaning toward thinking it is. We'll have to see. On that note, though, I'd be very interested in reader feedback on that point. Do you see the same trend or not?
Russ | June 3, 2010 9:36 AM
Bounce Rates confuse me. Isnt the goal of every website for the visitor to take action - no matter what it is - pick up the phone, fill out a form, download a PDF, read the text provided...

If I have 10 visitors who all pick up the phone and call me and all 10 people exit my landing page so I have a 100% bounce rate - then why is this a bad thing ?? I think my job of providing quality content has succeeded and the visitor has taken my wanted action...

I find it hard to gauge too much importance into bounce rates if I continue to see user activity...
L.K. | June 3, 2010 8:13 PM
Really good post; a much-needed reality check for those who are stats-obsessed. Thanks for taking the time to explain this, because its a needed tutorial and admonition to get our priorities straight.
Chris Butler | June 4, 2010 8:01 AM
Russ: Exactly! All of this should be measured on the basis of conversions. As you say, the bounce rate becomes less and less important as the page's purpose remains fulfilled. You know, the working title for this article (until the day before I published it) was "Conversion-Focused Measurement"...

L.K.: I'm glad you appreciated the article. Thanks for stopping by!
Paul | June 16, 2010 7:25 PM
Thanks for a very relevant post. As marketers we are faced with an often unreasonable demand for performance evidenced in stats. But since those holding us accountable for the evidence seem to have the wrong idea about what is actually worth measuring or considering relevant, we end up wasting our report opportunities qualifying our findings. This procedure is right on target with looking at the actually relevant data and not worrying about constantly increasing traffic.

One question remains. You've got me sold, but how would you go about changing the culture of expectation from the higher-ups?
Chris Butler | June 17, 2010 10:44 AM
Paul: Right on. I'm glad this article resonated with you. Your question is a good one, though I'm not sure I have a great answer for you. I think the best thing you can do is find ways to educate those who set the expectations, rather than try to manage them. Reframe the conversation by leading with data that is both pertinent and indicative of success. This way, you use positive feedback to encourage those around you to see the entire scheme of goals and measurement more realisticaly without any battles.

The other advice I have is to take that approach in small increments. Begin by pointing out conversions that are indicative of traffic sources or entire threads of activity that have so far been "invisible" to your superiors. Take a moment to do a quick demo of how applying an advanced segment can show that the value of smaller traffic sources can exceed larger ones. Congratulate those who have generated and nurtured leads that are most visible within your analytics.

I think you get the idea. It's not so much about flattery as it is about using positive affirmation to reframe the discussion and refocus the expectations. Otherwise, it's sure to become contentious. Good luck!

↑ top