Newfangled works with independent agencies to create lead development web platforms for their clients.

Newfangled works with independent agencies to create lead development web platforms for their clients.

How SEO Works



Arthur C. Clarke once wrote that "any sufficiently advanced technology is indistinguishable from magic" an insight that sheds a great deal of light on why our historical predecessors, without access to much of the knowledge we take for granted today, believed some of what they did. But it also applies to contemporary technologies, some of which we depend upon greatly yet understand only in part (or perhaps not at all).

The evolution of the meaning and use of the word "Google"—from proper noun to verb—corresponds with the increasing disconnect between web users and search technology. Ten years ago, searching for content on the web was a difficult process, but today one has only to enter a few words into Google's search bar, and Presto! (magical incantation intended) instant and accurate results. As much as this might seem like magic, it's a thoroughly mundane—albeit ingenious—technology at work. But if search engine technology is indistinguishable from magic, the process of optimizing web content for search engines will seem just as mysterious. Unfortunately, it's difficult to trust what we don't understand, and mistrust breeds the very kind of problems that are rampant in the search engine optimization industry: myths, abuses, and profit for those that would rather be seen as magicians than marketers.

Fortunately, we know enough about how search engines work to optimize our content with words, not wands. While there is some value in examining the myths and abuses of SEO, I think it makes sense to first explore how it works. I'll start with a brief explanation of how search engines (I'll focus on Google) work, then explain how web content can be optimized for them. Knowing how search engine optimization, in it's most basic form, really works will shed some light on the misunderstandings that often get in the way of doing it well...







Comments

jackWeb | January 3, 2011 10:27 PM
thanks for a very helpful treatment. i have a long list now of to-dos for my website. one question: how important is it to have friendly URLs? my site doesn't have the rewrite engine you wrote about, so i'm not sure how to do this part. also, how do you find out what your page rank is?
Jenn | January 4, 2011 11:32 AM
This article on Google's blog, which is actually 2 years old, states that the search engine does not recommend rewriting URLs:

http://googlewebmastercentral.blogspot.com/2008/09/dynamic-urls-vs-static-urls.html

Would you disagree with their assessment?
Chris Butler | January 4, 2011 12:06 PM
jackWeb: In my opinion, I don't think it's essential. A website with well optimized title tags, headings, meta descriptions, and good, recurring content that does not use "friendly" URLs could do far better with organic search traffic than a website that has them but falls short in other areas. That's why I mentioned it last, even after the idea of optimized link text.

To check the PageRank of any website, you can use www.prchecker.info.

Jenn: Good question. In short, no, I don't disagree with their assessment, but that is because I don't think their assessment is that all URL rewriting is bad. Let me explain.

At the beginning of the article, the author, John Mueller, writes:
"While static URLs might have a slight advantage in terms of clickthrough rates because users can easily read the urls, the decision to use database-driven websites does not imply a significant disadvantage in terms of indexing and ranking. Providing search engines with dynamic URLs should be favored over hiding parameters to make them look static."
What he's trying to say is that while webmasters are motivated to create more "friendly" URLs for the purpose of SEO (those that contain relevant keywords), it can be done wrong and create more problems than benefits, especially if parameters indicative of important processing functions are hidden in the process. If you read through the long string of comments, you probably noticed that the article created quite a bit of controversy, mostly because it seems that it was widely misread. But within that string, I think some clarity emerged. An early comment from Ryan Williams made some sense of it:
"This article is just meant to highlight some potential problems that can arise from developers turning dynamic URLs into static ones incorrectly, rather than be some general guideline for any and all purposes. Or to put it another way, it's saying that Google bot can potentially screw up if a developer isn't savvy with how he generates his rewritten URLs, whereas if he just leaves them as their default dynamic selves Google bot will generally get it right every time."
Then, a bit later, this exchange between the author and a user going by "lordscarlet" helps to clarify his perspective:
lordscarlet: "Your replies are very different form the blog post. You are saying that responsible URL Rewriting is good. The post, however, tells you to absolutely avoid using URL rewriting."

John Mueller: "The devil is in the details :). If you look at the web on a whole, you would have to say that most sites do a bad job at rewriting URLs and that it would be easier for search engines in general to see the real URLs. However, there are exceptions and situations where a properly rewritten URL, that does not contain any irrelevant elements, can be an advantage.



If a webmaster is unsure whether or not a chosen URL scheme is perfect (and implemented perfectly), then I would recommend leaving the dynamic URLs instead of providing something which can make it hard for us to crawl a site properly and completely.



We did not significantly change our crawling and indexing system for this blog post. We have only noticed that the myth that "any rewritten URL is better than a dynamic one" is very wide-spread and is just plain wrong."
So, it seems that Google's point of view on the matter is primarily to exercise caution. I completely agree with that.

You'll notice that I placed this last in the list of on-site factors. I did this intentionally because I believe it's the least important among the others (as I mentioned to the commenter above), or, in other words, has the least impact on a website's search engine optimization compared with proper optimization of the others.

That said, we do have a re-write engine built into our CMS that handles the action properly and causes no issues with Google. The way it works is that when a "friendly" URL is specified by the content author, the CMS creates a 301 Redirect (you can read more about URL Redirection in a helpful Wikipedia entry), which tells Google to index the static URL rather than the dynamic one. A 301 Redirect is the proper method for informing Google of permanent URL changes, so no variables are hidden in this case.
Desmond Williams | January 4, 2011 1:15 PM
Great article. SEO optimization can seem like a daunting task, but your list of simple steps really puts Google indexing in perspective. Informative as always.
Jonathan Hinshaw | January 4, 2011 5:13 PM
Great Article and Fantastic information. SEO is such a tough subject as there is probably more miss-information our there than any other website related topic. Thanks for keeping it simple and making the article usable in the real world - keep up the good work!
pj | January 4, 2011 6:20 PM
i saw this on twitter and was surprised after reading that there's no mention of social media. i wonder what the relevance of seo is today in the age of facebook and twitter?
Alex | January 4, 2011 9:08 PM
A nicely written article as usual, Chris, but I'm with @pj. At this point, I see very little SEO imperative given what social media has offered the average person. What's the point of searching with Google when most of the information I want or need I can either find or is brought to me other ways?

The eyeglass scoundrel you mention from the NYTimes piece is probably not such an anomaly. I'd find it hard to believe that those who know enough about how SEO works *aren't* tempted to be dangerous, if you know what I mean. Whereas social media seems to flow on the goodwill of personal relationships, this whole SEO bit seems like a casino to me.
Chris Butler | January 5, 2011 7:28 AM
Desmond: You're right, it really can seem daunting at times, especially if you have a large amount of content that needs to be optimized. But once it becomes part of the content creation process, doing SEO well will be far less of a chore.

Jonathan: I'm glad the article met you where you are. This is a lesson I'm still learning: how to take topics that we talk about often (like this one) and continue to produce resources that are engaging and actionable to our readers. Our approach to SEO hasn't changed radically in years, but it's still a very meaningful issue that I want to make sure we continue to discuss regularly.

pj and Alex: I intentionally left social media out of this piece, mostly because I wanted to focus on how search engines work and basic on-site content optimization. However, I agree with both of you that social media is quite relevant to SEO as it contributes heavily to off-site link and awareness building.

The other issue—which was implicit in pj's comment and the main thrust of Alex's—is whether SEO is relevant today given the ubiquity of social media. In short, I would emphatically say yes; as long as people continue to search for information, I believe search engine optimization will remain very relevant to what we do. But your points are well taken that social media have greatly impacted what SEO means. Certainly for the average person, there's no shortage of content available, and given the socialization of search, the content you're likely to find interesting is often just as likely to find its way to you via your social ties. With a system like that in place, there's little need for proactive pursuance of content. On the other hand, the availability of an individual's content to family and friends is going to be much more important to them than whether its findable by a stranger. But for business, organic search traffic is critical. One thing we often remind our clients is that they need to frame their on-page factors for the customers that are looking for them but just don't know their name yet, because those people—certainly thousands of them—are out there.

Alex, your other comment about social media carrying more inherent goodwill versus the feeling that only the house wins with SEO is understandable given the DecorMyEyes story. I like this quote, attributed to Saint Augustine: "Never judge a philosophy by its abuse." (I'm publishing a blog post along these lines this morning, btw.) Sure, DecorMyEyes has abused the system, but that begs the question, is the system unethical, or is the abuser? Given the possibilities for disappointment and further frustration that more qualitative approaches could create, I can understand why Google has avoided sentiment analysis to this point. But perhaps that will have to change due to the overall sentiment web users have about search itself. We'll have to wait and see.

Thanks, everyone, for reading and commenting!
Mark O'Brien | January 5, 2011 1:40 PM
I'd like to comment on the point pj and Alex brought up as well. When questions like "what is the relevance of seo in the age of facebook and twitter" come up, I usually head right to Google Analytics to see what the data tells us.

Over the past 30 days Newfangled.com received 10,000 unique visitors from search engines, and well under 500 from all social media outlets combined.

We put roughly the same amount of work into optimizing a newsletter for search engines as we do distributing it through our preferred social media channels, and SEO seems to give us a 200% better return in terms of traffic.

Now, our goal conversion rate for Google is only 1.17% over this period, while Twitter's conversion rate is an impressive 4.52%. An argument about quality of traffic can be made, but the overall numbers are in still in favor of the search engines.

That being said, I think SEO and social media are both very important, and neither should be dismissed when trying to increase traffic to a marketing site through a thought leadership-based content strategy.
Chris Butler | January 5, 2011 1:46 PM
Mark's point is right on: the traffic is clearly indicative of the number of people who are searching for information using engines like Google. Incidentally, I put together a chart of this exact comparison back in my newsletter from October of 2009 to show readers how much conversions matter. I've added the chart itself below—you can see that organic search traffic was, by far, our greatest source at that point in 2009, and as Mark pointed out, that hasn't changed:

JT | January 5, 2011 7:47 PM
I may not have said it they way that @pj or @alex did, but questioning the relevance of SEO in light of social media isn't as naïve as you might think. @Chris, I think you're hinting at this when you point out the differences in perspective that an individual might have from a business. This is really where clear lines remain within the "socialtopia" right this now: individuals and corporate entities. Both have flocked to social media with very different agendas. I remember back in that social media guide you posted that there was some reader controversy over how businesses should use social media and that's clearly still a big issue. Look on any college campus and the divide is even clearer. Student get it so much that they don't think about it. Administrators don't get it so much that they're thinking about it all the time.

It seems that the natural mediation point here is that @pj and @alex are right to question the need for SEO. Perhaps it does not now or never will matter to them. But @Chris and @Mark are right to dig in with SEO from a corporate marketing perspective (esp as social media for that purpose hasn't found its legs yet). The data is nice, had the question been asked from a business perspective,but I suspect it was not.

Good conversation.
jennifer | January 6, 2011 7:41 PM
This is so helpful, thanks for the straight forward explanation of how this all works! I think I'm reading the conversation right, and I agree with JT that getting Google isn't exactly on the top of most peoples priority list, but it will jump up there right away as soon as they want to get noticed on the internet, like having an Etsy shop or something. My parents have one so this information is something that they are asking about now. Now that I've found your article, I'm definitely going to share it with them.
Chris Butler | January 26, 2011 10:47 AM
JT: Very much agreed. Our approach to SEO makes sense given a legitimate content-driven online strategy, which means not content for the sake of robot food, but content that people will actually benefit from reading/viewing/etc. The balance with SEO should always prioritize people, which doesn't mean that SEO is entirely on the robot side either--keep in mind that people are the ones using search engines and expecting them to be effective research tools.

All that said, there are plenty of companies/brands that just should not expect a content-driven online strategy to be effective. Plenty of people have chimed in on this over the years (I first saw it in blog form from Chris Brogan), but the general idea is that my complete lack of interest in reading a Pop Tarts blog, watching Pop Tarts videos, friending or liking Pop Tarts on Facebook or following Pop Tarts on Twitter says nothing about the likelihood that I will buy and enjoy a Pop Tart from time to time. Tim Malbon of Made by Many just posted on this and calls out the absurdity of most consumer brand engagement like this for what it really is. Check it out.

Jennifer: Thanks for reading! I'm glad it was helpful.
khuyen mai | August 11, 2011 5:37 AM
All that said, there are plenty of companies/brands that just should not expect a content-driven online strategy to be effective

↑ top