Limitations of Search Engines for Discovery

I've written about the problems of global information overload in past newsletters like Wikis, Swikis and Blogs. But let's review. Having access to all the world's information and stuff is a good thing. When choices are constrained we adapt our tastes to what is available--not necessarily what we might actually like best, if we only knew. I like Coke, but I've learned that I like Diet Stewart's Fountain Classics Orange 'N Cream better. But without access to all choices I might not even know that Stewart's exists. I discovered Stewart's when I walked into Soda Pop Stop, a specialty soda store in LA to shoot one of our videos, The Simple Secret to Search. When I first walked the aisles, I was overwhelmed with the variety of soda brands and flavors. It took me awhile to take it all in. Finally, John Nese, the owner of the store, helped me out by explaining the difference between a truly brewed soda and one that's mass produced using shortcuts. When I told him I was particularly interested in diet orange sodas he helped me identify a few options that I could reasonably sample. Imagine though, if a typical grocery store stocked all of the choices that Soda Pop Stop stocked. And imagine if they stocked the same breadth of brands and varieties for all their products. It would be impossible to shop. It's already a bit overwhelming to walk into today's super stores--and they carry a very short list of options for each of their product lines.

In this example we're just talking consumer food brands. Multiply that out to all products, all content, all music, all everything and the ability to categorize, rank, filter, sort, and process it all becomes almost impossible.

One way we've learned to manage so much stuff is to use search engines like Google. If I want to find a shoe rack for my closet I can type "shoe rack" into Google. Google returns and ranks all of the 1,270,000 pages containing the phrase "shoe rack." I can count on its results to list the most relevant shoe rack pages at the top. Google and other search engines are one way we contend with billions of web pages. But it's not the only way, and sometimes not the best way.

For example, what if I'm not looking for a shoe rack in particular, but instead I want to organize my closet and I'm not sure exactly how. Now Google's not as much help. The broader, or more open ended my search, the harder it is to get a good answer from Google. Another example: I can type in "diet orange soda" into Google and get 5,200 pages. But it can't tell me which diet orange soda I'm most likely to enjoy. And the more stuff out there, the harder it gets. Google is great at finding the things we know are out there. But suppose you were to ask Google to find an interesting website that you don't already know about. Google would not be a very helpful tool--StumbleUpon.com is--but I get ahead of myself.

There are other times when even a super-accurate search engine is not much help finding what we need. For example, if you move into a new state and want to find a good doctor, what do you do? You can search for doctors on Google, or you can open the Yellow Pages. But on what basis do you choose from among the scores of doctors? Do you choose the doctor with the biggest ad on the page? Do you call a few and ask them some tough medical questions to test their expertise? No. Usually, you ask people you meet who they go to. You ask for specific recommendations like "who's the best doctor for allergies?" Friends are better at recommending certain things than directories and search engines ever will be.   next >