Search is a Work in Progress

We are all librarians now. I'll back that statement up over the course of this article, but if you're already picturing a stern, bespectacled figure who might shush you at any moment, think again. Today, our lives represent a fusion of roles hitherto segregated to a minor piece of society- the author, the producer, the librarian. The libraries I'm talking about are networks of content on the web, and they're run by ordinary users just like you and me. We create the content, using all kinds of simple yet powerful tools, with a prolificacy unheard of ever before. With those same tools, we can immediately package our content for distribution. And any content we find valuable--whether our own or someone else's--we make sure to quickly share with others. Like librarians, we guide the reading of our contacts, and they perform the same function for us. We are personally doing the creating, the organizing, and the connecting- and this changes how the content is found and received.

In past newsletters, we've focused on creative strategies and helpful tools to enable you to plan effectively and execute professionally on the web. But we haven't taken a close look at how we as users organize all this content and connect other users to it. This effort is much bigger than individual applications and has become such a natural part of our daily lives that it just hasn't stood out much. But it is now clear that the way we organize online is having a profound effect upon the world of search.

Search has changed significantly, and just as we adapted to the algorithmic approach of search engines, we must also adapt to today's crowded social environment. This month, I'd like to look at the ways that our use of social tools affects how we connect with content, and the practical ways we can apply this knowledge to improve websites.

In short, search is going social. But in order to understand how we got here, let's take a quick look at how search has grown along with the web.

In the early days of the web, few had a realistic expectation for just how quickly it would grow, nor how incredibly big it would become. So at the time, creating portals seemed like a sensible way to enable web users to find content. A portal (think Yahoo, circa 1998) is/was a directory of web pages gathered and manually categorized by human curators. The limitation here was that any content missed by the individuals in charge was virtually non-existent. As soon as it became apparent that this system wouldn't come close to keeping up with the rapid proliferation of online content, portals gave up on directories and started offering other tools like news feeds, stock tickers, and email addresses.

Eventually, we moved on to an algorithmic approach, building search engines that deployed robots to crawl the web looking for new content to add their master index. Rather than relying upon an individual's ability to find and list content, which (optimistically) could have lagged months behind, we could depend upon the algorithm to almost instantly match our queries with current content. This was a massive innovation, one that led a tiny startup in a garage to quickly become one of the largest and wealthiest companies in the world, and to have their name, Google, become synonymous with search. But with robots in charge, any content they missed was virtually non-existent, too, which is why search engine optimization has become a major industry in and of itself. And you know, SEOs have had a great run...

Now we're returning to a more human process, but unlike the portals of fifteen years ago, today's social approach to search depends upon everyone. With such a large number of people creating and sharing content across many different online channels, just about anything can find its way to you before the robots even notice it. No curatorial elite needed, no tireless robots needed. Just people.

Like any other subtle change in culture, this shift back toward a more human-oriented social structure has not happened abruptly, nor as a result of intentional action on the part of any one individual or organization. As far as I can tell, this process is the result of four distinct factors.

Factor 1: Rapid Fractalization
In an article written this past July, I observed that "the connections across the web, whether personal, professional, or other, are growing in number and granularity, making the passage of information faster than we can even begin to comprehend." I called this trend the fractalization of the web, because the image of a fractal, a geometric pattern that is repeated at every scale, seemed most appropriate to illustrate the speed and complexity of our growing interconnectedness. This fractalization is resulting in a massive cluster of web users and content within protected networks that are, for now, outside the reach of search engine robots. The more that fractalized networks grow, the weaker the search engine model will be in both representing the available content on the web and connecting users with that content. Indeed, the same social networking technology that enables this fractalization is what also enables human users to more quickly create and share content with one another. As a result, the number of content creators is growing to historically unprecedented levels. Consider the following quote, from Denis G. Pelli and Charles Bigelow, who recently wrote of what they consider a "writing revolution" for Seed Magazine:

"Since 1400, book authorship has grown nearly tenfold in each century. Currently, authorship, including books and new media, is growing nearly tenfold each year. That’s 100 times faster. Authors, once a select minority, will soon be a majority."

William Drentel, in a post for Design Observer on mapping the web, also put it nicely:

"The interconnectivity of the internet is moving at a speed that defies our imagination. Blogs, a term unheard of even two years ago, are now a ubiquitous part of the online experience. Technorati, as of today, tracks 10,552,890 weblogs and 1,164,251,940 links. A month ago this number was only in the 9 millions. We are counting sand on the beach, the numbers almost beyond our comprehension."

Between the exponential growth of social network users and the equally mind-boggling growth of content creators enabled by these networks, both of which reside in "walled gardens" outside the purview of search engines, the fractalization of the web is the most obvious and important factor driving the current socialization of search.

Factor 2: Increasing Human Efficiency
Employing robots to index the web was obviously more efficient than relying upon human curators, which is why the passage from portals to algorithm-based search engines was so quick and welcome. At the time, there were no simple tools available that would allow the people creating new web content to extend their reach across their existing social networks and beyond. Search engines eventually provided the reach, but did so without leveraging the social connections already existing between people; they did this on the basis of simplicity and efficiency alone. Now, with the power of social technologies, people can communicate across extremely flexible networks immediately. If you want to alert and engage people around your content, you can find the most appropriate channel based upon who you are speaking to with that content using any number of formats (i.e. status updates, tweets, blog posts, comments, images, videos, etc.). As content sources increase, the ability for people to connect one another to new content will exceed that of search engine robots to find and index it. If you're at all skeptical about this, consider how many recent newsworthy events were "scooped" by Twitter users long before the major networks were able to broadcast them. When you begin to notice Twitter posts containing links to content that has yet to appear in search engine results, you'll naturally switch your focus from using search engines for finding new content to depending upon social networks to bring it to you. This is already happening; human efficiency is exceeding robot efficiency.

Factor 3: Acting Upon Trust
Social networks are voluntary, opt-in communities. Because they attract willing participants to a virtual setting, social networks coalesce largely around common interests rather than particular factors, such as location, gender, age, ethnicity, or occupation. When users encounter content within virtual social networks, they are "pre-qualified" for it and more likely to trust that it is more relevant to their interests than any content that appeared in their most recent search engine query. If a friend sends me a link or simply posts one to a social network profile, I am much more likely to consider it a recommendation worth following up with than one of the millions that return in search engine results. While this may not affect the fact-checking or "look up" power of search engines at the present time, top engines like Google know their days of search preeminence are numbered. Their Social Search experiment, which delivers results pulled from your specific social network within their algorithm-generated results, is indicative of their recognition that trust is central to the way people find and engage with content. People trust people and people act upon trust.

Factor 4: True Equalization
Finally, one of the most exciting results of the social focus of the web has been the potential for true equalization among people. Because of the ease of access to powerful content creation tools, just about anyone can make themselves heard on the web today. Whether through blogging, YouTube, Twitter, or some other network, regular people have quickly accumulated influence online without needing to conform to the traditional celebrity-making systems that create music and movie stars. While this is certainly encouraging from a purely social point of view, equalization is probably the most troubling factor to businesses- particularly, but definitely not limited to, consumer brands. Again, I'll quote from Seed authors Pelli and Bigelow on the impact of equalization. Notice how everyday people are empowered to force change upon much larger entities in a way unheard of before now:

"Today, at 0.1 percent authorship, many people are trading privacy for influence. What will it mean when we hit nearly 1 percent next year and nearly 10 percent the year after as the current growth predicts? Governments, businesses, and organizations must adapt to a population that wields increasing individual power. Protestors used Twitter to discredit the election in Iran. When United Airlines refused to reimburse a musician for damaging his guitar, the offended customer posted a song online—“United Breaks Guitars”—and United’s stock dropped 10 percent."

This last factor is certainly much more difficult to swallow than any of the previous three. The vulnerability it represents to any business is quite intimidating. But fractalization, efficiency, and trust will also be game changers to all businesses, not just the major search engines. Considering these four factors, and the ways in which they are shaping how we create, find, and share content over the web, is essential for any company hoping to continue to effectively navigate the complexly interwoven real and virtual worlds.

Because search has evolved to conform to the more intuitively social way we use the web in general, we are going to need to take on a few new habits in the business context, too.

It sounds simple, but most of the time we act upon what we think people want, not what they really want. But for any company offering a product or service, knowing what people actually want is essential to keeping existing customers happy and gaining new ones. In examining the converging factors that have shaped search as we know it today, I noted that social networks and tools have created an equalization among web users, enabling anyone to broadcast their opinion to as many people as they like. This puts a lot of power in the hands of the consumer, power they are just as likely to use to trash your company when they are unhappy with your product or service as they are to praise it. In fact, most consumers are more likely to speak out when they are unhappy than when they are happy. Fortunately, there are plenty of ways for you to listen to your clients and potential clients.

Twitter Search

Twitter search is a very powerful application, one I suggest you make part of your everyday toolset. In the screenshot above, you can see that I've searched Twitter for "healthcare marketing" just to see what kind of activity those words are getting. If I worked for a marketing agency that primarily served the healthcare industry, this might be a valuable search for me to run. In fact, I might subscribe to the RSS feed for this query so that I can get updates whenever new messages on Twitter mention the words "healthcare marketing." Doing that might clue me in to the larger "conversation" around that topic, including what my competitors have to say about it. In the screenshot below, I'm using the advanced search to limit my results to only those with an indicator of unhappiness. Notice that when I do that, Twitter puts a sad emoticon in my search- you can use that modifier yourself right in the search bar if you like.

Google Alerts

Twitter is not the only place you should be listening. In addition to using Twitter's search tool, you should also be setting up Google alerts for mentions of your company or any other relevant topic in blogs. You can do this by using Google's Blog Search In the screenshot above, I'm searching for "healthcare marketing" again. Notice that I can also subscribe to an RSS feed for this query, making the search automatic. Any future matches to my query will show up in my feed reader, which saves me from having to remember to routinely search blogs for this information.

Both of these tools offer you significant value for very little effort. If you go the extra step of setting up an RSS feed for each search you might want to run routinely, you won't have to even think about this again unless you want to add a new one. I'd strongly recommend that you set at least a few of these up, including your name, your company's name, specific product or service names, etc. For more on this sort of thing, check out our newsletter on managing your online reputation.

Last month, I shared the conclusions we've come to about the right way to conceive and execute a web content strategy. Put simply, it comes down to knowing who you are speaking to. Only by clearly identifying your prospects can you go about creating content that is truly valuable to them- material that engages their need and brings them into relationship with you. Unfortunately, some of the ways that we've been handling search engine optimization have become a barrier to creating valuable content. Often, we've been writing for the sole purpose of adding indexable content to our websites. But the fact is that more web conversions (i.e. completions of calls to action) come from human referrals than from search engine traffic, particularly for B2B service firms. This means that content we write to attract search engine indexing bots may not be worth all the effort we put into creating it. Bearing this in mind, we recommend regrouping with a four-step value-oriented content strategy that includes planning with personas, writing for humans, engaging with your audience offsite, and routine measurement. If you only remember one thing about speaking to people, make it this: People trust people and people act upon trust. This principle alone should enable you to re-calibrate your content strategy so that your website speaks clearly to people.

Using third party applications is not the only way that you can better adjust to social search trends. If the principles of simplicity and conversational operations are important to the web at large, we should be applying them to how we make information available within our own websites as well. One powerful way to do this is to broaden the scope of your site's internal search tools while making them even more simple to use.

In the screenshot above, you can see how simple the site search looks on Brahmin.com. But behind the scenes, we've built a database with the user in mind- particularly how a shopper is inclined to search products. Rather than creating an advanced search with lots of category checkboxes and pulldown menus to filter search results, we've created one that dynamically responds to queries by showing results as the user types. In this image, I've searched for the word "black," which generates a list of all the black handbags in the database. If I add a modifier like "tote" or "clutch," the list will immediately shrink to only those bags that are black totes or black clutches. Each product in the database has numerous descriptors as part of their content definition that enable this kind of user interaction. The emphasis is on doing much more for the user and requiring much less of them. In terms of the social trajectory of the web in general, this type of search is an incredible emulator of human experience; you're creating a tool that is much closer to dealing directly with an informed salesperson than filling out a mechanical form.

You can learn more about this type of functionality upgrade in my newsletter on doing more with less, or Sarah Dooley's specific writeup of Brahmin's advanced product search solution I showed in the screenshot above.

As my title suggests, this is not the end of the story of search. Reviewing the short history of the web shows how quickly things change, and remarkably, how quickly we adapt to them. It would be foolish to think that the algorithmic approach of the search engines is entirely dead, as well. Google's experiment with social search doesn't necessarily represent a white flag of surrender, but more likely a coming adaptation of their own to the current environment. And that, really, is the point: even the largest, most dominant entities on the web must also submit to the inexorable progress of the web. After all, a company of 20,000, though comparatively massive as far as businesses are concerned, is nothing amidst the billions of individuals who make up the web. In the meantime, understanding the factors enabling the social thrust of the web will empower the rest of us little guys with a strong business strategy focused on to whom we really want to speak- people.