by Christopher Butler on in measurement, seo

Months ago, while planning the topics I’d cover in this newsletter through the conclusion of 2011, I had in mind to write something about search. It had been a while—almost two years at that point—since I last wrote anything about search specifically, though I had covered all sorts of things related to search many times since then, like search engine optimization and measurement. So, as is my habit, I created a text document called “search article” and began adding notes as ideas or reference material came up.

As I reviewed those notes, I realized something. Most of them were really about measurement. That’s when it hit me: Search as a topic is interesting—there’s certainly academic value in exploring how search engines work and how we use them—but for all practical purposes, there’s very little perceptual difference between search and measurement. After all, we’re not that interested in what people are searching for in general; we’re interested in what queries people use when they are searching for the kinds of products and services we offer, and especially in how they get from their search to our websites. In other words, what we’re really looking to understand is the feedback loop that exists between search engines and websites, and the key to doing that is in measurement.

In the past year, however, there has been at least one major change to how Google participates in that feedback loop—one you’ve probably noticed and have urgent questions about. I’m going to get to that. In fact, discussing that single change will be the bulk of this article. But before I get there, let me offer a prediction for the coming year that is, for better or worse, largely the result of decisions Google made in the last few months: 2012 will be the year that many of us start paying for analytics. Whether for specific web analytics applications, API integration, or AdWords, we are going to start discovering that consistent, reliable access to data and analysis is well worth budgeting for.

If you haven’t already come to that conclusion yourself, let me try to convince you…

Google has historically provided a very wide range of perspectives when it comes to search analysis. On the farthest end of the spectrum is Google Zeitgeist, a yearly anthropological analysis of the very big picture as far as search is concerned. Here you’ll see how our search queries form an index of culture, describing who we are and what we care about. Not surprisingly, you won’t see many of the keywords you care about professionally, that is unless you are a celebrity publicist. Google Trends zooms in just a bit more and evaluates individual search queries based upon volume. Again, getting too specific probably won’t be very helpful. For instance, there’s not enough volume to provide data for “website prototyping,” but there is for “web design” (trending down, by the way). Next, Google Insight can help you get a little closer to the terms you care about and see how certain search queries trend over time and geographically. For example, searches for “website prototyping” became far more relevant globally starting only in 2008—surprising, yes, but that insight helps me to get a better sense of the personas I consider when writing articles like this one. And yet, even though we’re getting closer to our daily, practical needs when it comes to understanding how search traffic corresponds to website traffic, none of these tools offers the value that Google Analytics has.

Three years ago, I wrote an article on how to use Google Analytics that has consistently been one of the most popular pieces of content on our website. That says nothing about the article’s quality, really, but everything about the demand among readers for information that might help them use this powerful tool more effectively. And powerful is really an understatement. Back then, I put it this way: “Google Analytics is, in my opinion, the most valuable application that Google has created so far.” Until very recently, I’ve stood by that statement. But Google changed something so central to the value of Analytics that I have to downgrade it just a bit. It remains a great tool, certainly, but it can no longer stand alone. I now see it as just one part of a larger, diversified analytics suite—one you’ll need to start thinking about more deeply as well as financially.


The Search to Stats Feedback Loop is Broken

As of October, 2011, users logging in to their Google accounts were redirected, by default, to the secure (indicated by the https:// in the URL) version of Google’s services. Announced as an initiative to make search more secure (i.e. private) for users, Google began encrypting search queries submitted by users in the secure environment, which means that the keywords they searched for were no longer recorded as part of their referrer information if their search led them to your website. To be clear, this means that anyone signed in to their Google account will always be searching in a secure environment. If they open another browser and don’t sign in to their Google account but search for something using Google, that search will not be encrypted. But as you can imagine, that group is an increasingly unlikely number. There’s no way to soften the blow here. Losing access to this information is a big deal. You’ve probably already seen the results. If you log in to your Google Analytics account and take a look at the Traffic Sources report, “(not provided)”—the aggregate entry of all encrypted keywords—will be a rising star on your keywords list. For Newfangled.com, “(not provided)” is now accounting for over 20% of our incoming organic search traffic—the largest single entry on the list. Like I said, a big deal.

So why is Google doing this? They say it’s about respecting user privacy, which I’d like to take at their word, but there are definitely many benefits to Google that may or may not be the driving force behind the decision. For instance, tying all user searches to unique accounts will enable Google to create a more personalized search experience, which should lead to more committed users. Google wants you to stay, you know. And for those in marketing that can’t stand losing access to the keyword data, routing more budget to AdWords, which still offers keyword and session data, is a viable option for now. That means more revenue for Google and a deeper level of commitment from those users.

By the way, Google Webmaster Tools does still aggregate the top 1000 search queries responsible for traffic to your site over the last 30 days only, including those from users in the secure environment. So, working with that tool could be attractive to you, but my opinion is that it won’t be worth the trouble to set up and begin managing—not to preserve 30 days worth of data, anyway.

In response to all of the concern about the “(not provided)” data set, Google’s resident Analytics expert Avinash Kaushik provided a few custom reports that you can import into your analytics account to analyze it—sort of—and concludes that it’s reasonable to assume that “(not provided)” is a cross-section of your website’s overall traffic. Kaushik believes that this just stands to reason, given how secure search has been implemented across all Google Accounts. Maybe. But one could also assume a basic qualitative difference between those who use Google services and those who don’t. Why not? Or, a difference between those who bookmarked the secure version of Google over a year ago because of privacy concerns or literacy of that issue and those who are now searching secure whether they know it or not. In either case, big assumptions are being made. This approach isn’t much of a consolation.

What people really want to know is what words are initiating sessions on their websites. But that’s exactly what’s being withheld by Google and exactly what defines this new set. This new set is a black box full of words you want to know. As more and more users become part of the secure search set—which will happen—this sorta-kinda general comparative analysis (i.e. “(not provided)” vs. everyone else) will be relatively meaningless. Honestly, it already is. So, if you were hoping for a workaround, you’re out of luck.

Here’s a light hunch: Google has explained that withholding search terms was a decision they came to out of respect for user privacy. If that’s true, I can respect that, though it gets me theorizing there is certainly some kind of gradation to the nature of search queries and how private they are, and that, just like any other matter of ethics, we should continue to carefully refine our approach rather than be reactive. In the meantime, though, I have a bit of a default skepticism as far as Google’s concern for privacy goes. Sincere or not, shutting down access to keyword data definitely makes paying for AdWords more attractive, even if only on a petty cash level, which of course benefits Google substantially in the aggregate.

My bigger-and-more-prone-to-be-wrong hunch: that Google will release an enterprise-level analytics tool through which this data will be provided, though in keeping with their privacy angle, not tied to specific sessions or page views. You’ll be able to see which terms are bringing traffic to your site and score those terms on the basis of time on site, page views, bounce rate, and the like, but not necessarily see that term “X” led to a session that landed on page “X,” followed by page “X,” etc. Just a hunch.


There is No Next Google Analytics

If Google Analytics were to be discontinued completely—something I definitely don’t expect to happen—it would sting, but it wouldn’t really incapacitate anyone. In fact, there are more analytics tools today than you could possibly use. Some relatively new examples include Clicky, Woopra, Chartbeat, Mixpanel, and ShinyStat. I’ll be honest: I haven’t purchased accounts for any of these services, but on the basis of the bit of looking at them and reviews for them I did to prepare for this article, none would be a waste of your time or money at this point, especially if you’re willing to pay for the freedom to experiment with a variety of sources of analysis.

Given the availability of tools and the economic factors at play, a wise marketer would assemble a suite that includes a general-purpose analysis tool, a CMS-integrated solution using social media and keyword APIs, as well as several free tools that look at web data from different angles. As far as API’s are concerned, we made the decision this year to integrate our CMS with two in order to bolster its reporting capabilities. To preserve keyword ranking as a datapoint in our CMS dashboard, we selected an API that queries keyword ranking on the fly, which in turn lets our users enter keywords into their dashboard and track their Google ranking over time. This is something we used to be able to do using the basic Google Analytics API until last summer, when Google restricted queries for that data. Rather than give up on keyword ranking as a useful datapoint, we tracked down the best solution we could find and, yes, paid for it. To get more information on the leads generated by our content, we chose a social intelligence API that will look up email addresses and retrieve active social media profile info for those accounts, which provides us with really helpful information as we score the leads generated by our site (shown in the screengrab below). We pay for this one, too. But the value to us is clear: This kind of integration has helped us create what I sincerely believe to be one of the most forward-leaning content management systems out there. Now is the time to be creative and experimental with measurement.

And yes, there’s free stuff still worth exploring, too. Here’s just one tiny example of a free but very unique measurement tool: Plenty of people use bit.ly to shorten URLs, especially since it integrates nicely with Tweetdeck, but many don’t know that you can get analytics on your bit.ly links simply by adding a + to the end of the URL.

With the bit.ly stats, I was able to see how many people clicked the links I created for the content included in a pet project of mine called A Year of Ideas. In a nutshell, A Year of Ideas is a yearly compilation of web content that I gather, print, and give to friends and colleagues. But online, I list out the chapters and provide links—using bit.ly—to the original content. By extracting data from the bit.ly stats too, I was able to discover that—so far—the average number of clicks has been a modest 13, with most of the clicks skewed, predictably, toward the top of the list. (In fact, the average of the top half of the list is 16, while the average of the bottom half of the list is 9.) Anyway, bit.ly’s stats are a powerful way to measure how your content is shared across the entire web, too, especially if you’re doing lots of social media promotion. For every link you create, bit.ly also connects it to an aggregate bit.ly version to show how often the content you are linking to is shared using bit.ly. That includes anyone on the web who used bit.ly to link to your content. Very cool. So, for example, I used bit.ly to link to a wonderful piece by Adam Gopnik in the New Yorker called How the Internet Gets Inside Us. Here’s the stats report for that link. My bit.ly link received 20 clicks, but the aggregate received 8,326. Of course, site-side analytics should be able to tell you how much traffic is incoming to any particular page, but the bit.ly side can help you to dig further in to the detail: when the link was created, when it was clicked, on which social media platforms the link was shared, where the users were located, and even a timeline of the shortened URL’s life so far.

To sum up, what you’ll do next will be to find a blended measurement solution, not an all-in-one.


The BIG Picture

I think the bottom line here is that all of this—Google’s moves and the proliferation of other measurement tools—amounts to an indicator that our entire notion of the value of index search data will change in the not-too-distant future. In fact, it probably already has, though we’ve not cognitively caught up with our own behavior. In fact, Roger McNamee, founding partner of Elevation Partners, goes so far as to say that index search has peaked and I think his analysis is sound.

If you don’t have time to watch his entire TED talk about disruption and engagement, take it from me, he lays out a pretty solid case: First of all, Google went from powering 90% of all internet search volume 4 years ago to just under 50% today. This is an inflection point, where “other” tools like Wikipedia, Facebook, Twitter, TripAdvisor, Yelp, etc. crosses over and collectively outrank Google. This is what is depicted in the screengrab I took from McNamee’s video, above. Many of these tools have gained momentum due to the content associated with their platforms, but also because they have unique, searchable mobile applications—an important thing to note especially as only a small fraction of the overall index search activity happens on mobile devices.

This isn’t a such-and-such-is-dead announcement. It’s not that index search is going away, it’s just that its overall dominance is being eroded by a variety of other options. Just like measurement. See the connection? Our job will be to maintain flexibility and remain on the lookout for the trends behind the data we measure, as well as the best tools available to do the measuring.

  • Jake

    Great read and thorough explanation of what has been puzzling me in my GA acct for a while now. But now that I know, I’m ticked off.

    It’s not that I’m against Google making changes, or even making changes that might push account holders to become paying customers, it’s just that it would have been classy if they’d let people know in advance what the outcomes might be. The announcement is one thing, but a “what this means for you” explanation was needed. A missed opportunity for Google to show that they actually gave people a passing thought.

    I guess you’re right to be skeptical of Google by default. So, yeah, it looks like you’re right about paying this year, but for most of us, that’s not just an unattractive option, it’s not really an option at all. :-(

  • http://gk.com gk

    good analysis as usual. i think you’re hunches sound about right to me!

  • MaggieB

    Losing the keyword data is disappointing, yes, but I don’t completely get why you’re downgrading Google analytics overall. It still is far and away the best tool of its kind out there for most people, isn’t it? Most of the other tools you mentioned cost something, but Google is still giving theirs away. Not to harp, but disowning them seems a little premature. :-)

    I actually don’t really have that much of a bone to pick about the encrypted data, anyway, because in the end I completely agree with your analysis and hunches. Why wouldn’t Google want to make some money off of analytics at this point? If they do release something bigger that requires payment, it’s probably going to be worthwhile.

  • https://www.newfangled.com/chris_butler_blog Christopher Butler

    Jake: Agreed, a more qualitative warning would have been considerate. Maybe there was one and I didn’t hear about it?

    gk: Thanks, and thanks for reading.

    Maggie: I’m not trying to be overly harsh on Google. It’s certainly their prerogative to add or remove functionality from their software as they see fit; especially when they’ve not asked for anything in return for the use of that software to date. That’s not why I downgraded them. And by the way, I should say that I was trying to be a bit tongue-in-cheek when I wrote “downgrade,” knowing full well that my opinion on the matter should have little to no influence on the value of the tool or their company. It would be absurd for me to think otherwise!

    Anyway, to me, the keyword data were what set Google Analytics apart from other tools precisely because the keywords maintained that search-to-stats feedback loop I was talking about. That’s why an analytics tool offered by a search engine is such a brilliant idea! Billions of search queries pass through Google every day, and their analytics tool helps you to understand how your tiny piece of that activity corresponds to the traffic your website gets. That’s been a game-changer for marketing analysis. Without that connection, it’s not as if we have nothing. But we don’t have much that another analytics tool can’t offer us. This is why I don’t find Google Analytics as essential as it was, and frankly, why I spend far less time in it today now that our CMS has been updated than I did a year ago.

    As for my hunches, we’ll just have to see. But I appreciate your vote of confidence ;-)

  • Amy

    Fantastic article! Thanks for your thorough research and thoughtful insights. I’ve noticed an uptick in “(not provided)” keyword results and was curious about it. If Google keeps so many keywords in the secured space, would I see a change in results if I paid another company for analytics?

  • http://www.highrankings.com/ Jill Whalen

    But, Chris, because Google has shut off access to the keyword data for any logged in user, that information is not available to ANY analytics tool, not just Google Analytics.

    So there’s no need to stop using Google Analytics in favor of another analytics program since you still won’t get the missing information.

    Your point about using other types of analytics tools that provide a different sort of data/information, is well taken, however.

  • http://www.creatingITfutures.org Eric Larson

    What do you think of services like Hubspot?

  • https://www.newfangled.com/chris_butler_blog Christopher Butler

    Amy: Thanks, I’m glad you enjoyed it. Although on the thorough research point, no matter how thorough I think I am, I always miss something. In fact, just five minutes ago, Mark found a link to a service neither of us knew about: Google Analytics Premium. As far as I can tell, the advantages it offers are heavily in the guarantee area—good SLA, uptime, data ownership, etc.—as well as a much higher level of customization. But there’s no indication of whether keyword data is handled any differently within this tool than the free version. My assumption is that most of these distinctions were made before the encryption change went public last October. In any case, we’ll still have to wait and see if my big hunch is right—that a tool like this Premium version might offer keyword analytics at some point in the future.

    The short answer to your question is no. But let me explain in a way that will answer Jill’s comment, too.

    Since Google has encrypted the keyword data, no other analytics tool would have access to it either. That’s why the API we were using also stopped allowing keyword queries. It’s not just Google Analytics, it’s the data; Google is keeping it all close to the chest.

    Or, in other words, what Jill said.

    But just to clarify, I wasn’t saying that anyone should switch from Google Analytics to another tool because they might be able to regain access to keyword data. I was saying that it was the keyword data which made Google Analytics such an incredibly powerful analytics tool and distinguished it from many of the others. Google was providing the essential piece of that search-to-stats feedback loop: being able to help website administrators understand what people are searching for and how that directed them to their sites. Without that key piece, I’m not sure that Google Analytics is substantially better than another tool. So my advice is to start experimenting with building a “suite” of tools that are both generalist, as well as unique—as many of them are (i.e. some offer real time, some offer unique social measurement, etc.).

    Eric: I have mixed feelings about Hubspot. But I also have limited experience with their services, which I know have grown and changed significantly over the last few years. Mark recently bought a short-term consulting package with them—within the last year—so I’m going to ask him to share his thoughts about that.

    The one thing that I haven’t been a fan of—and again, this is information I only know about from clients who have used Hubspot in the past so I’d love to hear if this is incorrect or being handled differently now—is that Hubspot customers that use their blogging tool are not given the ability to export that content when/if they decide to end their relationship with Hubspot. Because blog content is such an important piece of a company’s web property and marketing strategy, keeping it offsite, and in particular, with a service that restricts access to that content, is a mistake. Mark?

  • http://newfangled.com/mark_o_brien_blog Mark O’Brien

    Eric: I’ve always had good experiences with Hubspot, particularly with one consultant there by the name of Mark Greco. The way I read their pricing, most mid-sized businesses with a decent book of leads/subscribers would end up paying around $1,000/mo. or so to use their tools, so it is in the same basic ballpark as other high-end LMA (Lead Management Automation) tools such as Eloqua, Marketo, Genius, etc. Most mid-sized businesses wouldn’t build their sites with HubSpot, so that ~$1k/mo. is in addition to whatever hosting and maintenance costs they’re already paying.

    None of these tools would be used in place of Google Analytics, but rather are used in conjunction with it.

  • Tyroga

    I gotta say I’m not that bummed about it all, and am pretty sure there was indeed a “what this means to you” type email that went around about it.

    Ultimately what it may mean is less “click bait” and “keyword loading” you won’t have shady website owners going “awesome we put that keyword throughout our site and got this many clicks from it, even though it doesn’t have a lot to do with what we really offer”.

    The power in Google Analytics is how the user moves through your site, your funnels and filters. They get there because you created good content or connected with them in some way. The “how they get there” should be less important than what they are doing why they are there and what you can do to keep them.

    Too much emphasis is placed on the numbers. If you’re a content provider, create good content. In the words of that faithful movie “Field of Dreams”: “if you build it, they will come”.

    Granted I’m not in the business of making money directly from my own site (but I have). And maybe my opinion would be different if I were, but word of mouth is an always will be the best we can get, and Google is really just another form of that. And just like any other customer who provides a word of mouth referral they do so on the basis of our products and the overall service we provide.

    And I think the other thing with people logged in: users are getting a more curated search these days. Results are based on past searches and all round general belief in what Google thinks you’re interested in from web usage while logged in. It’s no longer like it used to be, the results you see are not the same as the ones I will see.

    For me the “keywords” section of analytics has always been a curiosity thing, I love it when I see something weird pop up in the list, but often when I try and replicate the search it doesn’t result in a link to my site appearing on the first however many pages I click through. So it’s no longer relevant to me.

  • https://www.newfangled.com/chris_butler_blog Christopher Butler

    Tyroga: I get regular email updates from Google Analytics; several came before, around the time of the encryption announcement, and after, but none pertaining to the encryption or what it would mean. That said, if you got one or know of one I’d love to see or be directed to it. Again, I’m not trying to pick on Google about this. As I said, it’s their prerogative to make these kinds of decisions. Or, like you said, I’m not that bummed about it. ;-)

    As for what it may mean, your idea may be right to some degree, although I think that’s already happening as a result of Google’s crackdown on content farms and the tweak they made to their algorithm last summer (or thereabouts). In general, though, I agree with you that creating good content should be prioritized over driving traffic. But that doesn’t mean that measurement of the traffic you do get isn’t important. That’s what I’m talking about here, and what I tend to focus on in all my articles on measurement. I’m certainly not a lots-of-traffic-is-a-good-thing kind of guy.

    Thanks for commenting!

  • http://www.broadbandall.com Russ

    Chris, great commentary – it does stink that Google is encrypting search queries – it is one of my favorite features.

    Google Analytics has made me a believer. I switched over from webtrends and am absolutely blown away by how they progress this “free” tool – What a gift. The amount of info and how far you can drill down is amazing to me. I get lost in it for hours.

    I always thought that they should combine Google Webmaster Tools and Analytics together. There is overlap there and sometimes I like to look at info from both at the same time.

    I have given up trying to figure out why Google does what it does – but Analytics still remains the best free package on the market. Nice read….

  • http://www.html5canvastutorials.com Eric

    Great article, but don’t forget that Google Analytics, although free, is already EXTREMELY valuable to Google because it serves as yet another mechanism for them to mine data about our websites. I think this data is actually more valuable than the potential revenues they would make from a paid service.

  • Richard

    You already have to pay for Google Analytics. Apparently if you send more than 10M page views they will make you sample on your own or you have to pay for Premium.

  • http://piwik.org Peter

    Piwik (http://piwik.org) is a great alternative you should take a look. It provides a feature rich API with segmentation and stuff. It is free but you should be aware that you are hosting it on your own servers. Hosting strong servers can be expensive, so it is not all in all free in the end.

  • https://www.newfangled.com/chris_butler_blog Christopher Butler

    Russ: I hear you. Google has definitely been accelerating in various ways lately. Just posted something on that today, by the way. All we can really do is watch and see…

    Eric: Great point!

    Richard: Interesting. I didn’t realize there was a traffic limit for sites using GA, but it makes sense given how often Google’s servers would be hammered in those cases.

    Peter: Thanks for the link. I’ll check it out!

  • http://www.a1webstats.com Andy Harris

    Great article Chris – refreshing to see well-thought out content to keep us analytics-heads interested.

    When we created our analytics software product (which, like the others, is hit slightly by the lack of keywords being visible, and will increasingly be so over time), it was (as I think other analytics systems are) created as complementary to GA. There are many great things that GA do that other tools wouldn’t attempt to replicate, but other tools can add additional insights that are useful to certain types of users.

    In our case we decided that, amongst other things, people within companies would be most interested in which other companies had been to their websites, and each page they viewed, how long for, etc. (and keywords they typed of course, when possible to track that!). For our target market, such companies utilise that information in various ways and so get their ROI.

    For other analytics systems there will be gems of features that we don’t feature and GA doesn’t feature. It all depends on what the buyer is most interested in I suppose.

    However, I fully agree that 2012 will see a significant increase in people interested in getting deeper into analytics. We’ve seen a noticeable increase over time and once that hurdle (people ‘getting’ what the data can tell them) is successfully leaped, companies start to benefit.

    A comment about Google themselves and why they hold back on keywords. I’m happy to be a sceptic and my viewpoint is that it comes down to one thing: an awareness that analytics tools (and knowledge) are making people increasingly able to dig deep into their website traffic, makes them aware of where they’re wasting money. If Google shut off that ability within organic search but effectively make people pay for it (via PPC) then it’s just another way of pushing people into the PPC direction. And of course, it’s no secret that Google would love the whole SEO community to just wither up and die so that they can make as much money as possible.

    For those of us in the analytics industry who care about the ROI that companies get from their expenditure, all the time that there appear to be unfair practices going on (for example, the content network being switched on by default within Adwords accounts when set up), then our work is never done.

    Forecast for 2013-14: consolidation of analytics providers – analytics companies buying other analytics companies purely to get their customer bases who are willing to pay subscription fees.

  • http://bits.blogs.nytimes.com/2012/01/16/i-b-m-makes-its-social-computing-strategy-smarter/ Jessie

    what do you think about IBM’s analytics stuff? just saw this piece in the NYT where they say that IBM “has spent more than $14 billion on 25 companies that specialize in data mining and analytics.”

  • http://www.qinsen.com tony

    Apparel Packaging

  • http://www.tutorialspark.com/ Andrew Symonds

    Future of Internet will not be decided by google, They are not the company which can successfully organise all the global data in a free and fair way. Other companies must venture in this field now. Even the comsumer must be aware


By Christopher Butler

Chris Butler is the COO of Newfangled. He writes and speaks often on design and the web. You can follow him @chrbutler. More by Christopher Butler