Skip navigation
factory /><div class=
Nolan Caudill
Lead Software Engineer

Nolan's Blog


Subscribe to this blog
Click this link to view blog as XML.

View a list of all Newfangled blogs >>
Subscribe to all Newfangled blogs >>


A List of Useful Web Development Blogs

October 22, 2009 at 11:37 am by Nolan

The Internet is a noisy place, with Twitter updates constantly streaming and blogs spewing out second-hand knowledge, making it difficult to find sources that are both credible and consistently good. Here are a few blogs that I follow that keep me informed on what's going on and what's next in Web technology.

JavaScript

Ajaxian: This is the blog on the Web for all things JavaScript and AJAX. The blog profiles new frameworks, libraries, and techniques that you can usually bring immediately into your Web development. They have a nice mix between covering the new and shiny stuff, as well as posts that instruct readers on how to become better JavaScript programmers.

ejohn.org: John Resig's blog has long been a source of JavaScript knowledge. The jQuery creator and Mozilla evangelist shares years of industry know-how and is a great programmer. He does a nice job of detailing the inner workings of JavaScript, as well detailing useful tidbits about jQuery.

PHP

Chris Shiflett: PHP security, no thanks to PHP itself, is hard to get right, as security features that should be enabled are disabled by default, and vice versa. My favorite example is the mysql_escape_string versus mysql_real_escape_string. Instead of deprecating the former, you have both and you have to know which one to use.

suspekt.org: In a similar vein as Chris Shiflett's blog, but another great resource. suspekt keeps an eye on security regressions and improvements at the PHP source level between version and monitors Suhosin patches, which harden a stock PHP install. Stefen Esser speaks often at big PHP conferences and is fantastic about putting his slides up about the state of the art in PHP security.

Security

ha.ckers.org: These guys keep up with dark side of the Web and distill that knowledge for the average web developer. My favorite part of this site is their XSS-checking tool: http://ha.ckers.org/xss.html.

Google Online Security: Google sees a lot of sites poorly designed for security and here are their suggestions to help make the Web safer for everyone.

Software Development

Uncle Bob's Software Craftsmanship Corner: Robert Martin (AKA Uncle Bob) is a great resource on how to approach software design and construction from a reasoned point of view. While it is easy to write less-than-elegant code, Uncle Bob encourages you take the high road and write your code for your future self and comrades. He literally wrote the book on clean code, titled appropriately "Clean Code," which is also highly recommended.

Google Testing: This blog is a fascinating look into the testing culture at Google. I'd imagine that the extent of their testing puts most software teams to shame, but it's nice to see how one of the biggest in Web software does it.

Michael Feathers: Similar to Uncle Bob (they actually work at Object Mentor together), Feathers' blog is another great programming resource. I see Uncle Bob as more of a nuts-and-bolts writer, while Michael Feathers takes a more wide-angle glance at software development. His book "Working Effectively with Legacy Code" is a good read if you ever find yourself working with older code.

Steve Yegge (http://steve.yegge.googlepages.com/blog-rants and continued at http://steve-yegge.blogspot.com/): While famous for his verbosity, Steve Yegge always has a interesting story to tell. Every post has its own roundabout way of making you a better programmer, or at least make you think about code you write, which is a win in my book.

HTML/CSS:

zeldman.com, meyerweb.com, snook.ca: These three, Jeffrey Zeldman, Eric Meyer, and Jonathan Snook, have long been considered the vanguards of writing semantic and standards-compliant HTML, as well as being a valuable resource of how to design for the Web using CSS.

diveintomark.org: Mark Pilgrim is an extremely funny and talented writer -- his book, Dive Into Python, has been considered the best way to learn Python for several years. He is involved in WHATWG, the group that shapes the future of Web technology and the drafters of HTML5. He maintains doctype (http://code.google.com/p/doctype/wiki/Welcome) which is a must-read for web developers.

Grab Bag

stackoverflow.com: Not a blog per se, but I couldn't leave it out. An engaging and knowledgeable community is always available to give advice on every aspect of technology, to general programming practices to very specific language/API questions.

YDN Blog: The Yahoo! Developer Network Blog covers everything from performance to accessibility to their cool APIs.

Simon Willison: Simon aggregates lots of great web development content, and seems to have a good eye for the future of Web technology.


Tags

FACEBOOK
 

The Most Important Person on the Team

July 31, 2009 at 1:54 PM by Nolan

While not being the most glamorous or high-profile position, the Quality Assurance (QA) role is the most important person on any web development team, and it doubly so here at Newfangled. Without them, our websites would be riddled with bugs, projects would never get finished on time, and everyone would generally be in a sour mood. Here, I hope to detail out what exactly these QA people do, and why I consider them the most important people in the room.

So what exactly is Quality Assurance? Wikipedia defines this as the "set of activities intended to ensure that products (goods and/or services) satisfy customer requirements in a systematic, reliable fashion." At Newfangled, the QA team ensures that the developer builds the correct features, that development progresses at a pace that hits deadlines, as well reports back to the project managers and the developers on any slips to either scope or budget. With web development, projects can quickly spiral out of control and it is QA's job to avoid this, or at least make a small impact as possible on scope or budget.

Exactly how the QAs do this involves many different skill sets and character traints, which usually are a rare combination in any one person. The first major skill is to be able to understand what you are looking at in order to see if it looks right and matches up with the client's expectations. Web technology is constantly changing, and these changes bring in new terminology, interfaces, and tools and the QA has to have the technical acumen to understand the requirements and be able to match that up with what the developer is building. This is a challenging feat but crucial in ensuring the right product is being built.

The second major part of being a good QA is a high level of attention to detail. As with any software project, the specifications of what to build is usually novella-like in its length. Absorbing these exacting requirements and making sure that the software is built exactly how the client expecting demands a very level of precision and accuracy that few people possess. This is not a problem if the requirements are straight-forward, but usually, the requirements can be ambiguous, vague, and, worst of all, contradicting. Its up to the QA to wade through these and check back with both the project managers and the developers to make sure that they are "satisfy[ing] customer requirements." Manually checking a punch-list 100 items long takes an attention to detail that, personally, I'm glad someone else has.


Photo by guitavares


The most important skill of any QA is tact. QA usually sits in the demilitarized zone, right between the client (via the project manager) and the developer. The client usually needs that feature done yesterday and the developer is fighting some obscure bug that is sapping away his time. Since the QA is essentially in the satisfaction business, they have to make sure that the client (once again, via the project manager) is getting reasonable feedback on progress as well as keeping the developers on task and on time. This requires an extreme amount of tact. For example, the requirements are usually in black-and-white and so whether a bug exists or a feature is implemented correctly is also a cut-and-dry call: it works, or it doesn't. It's difficult to have to send an email to the developer that the code he just spent the past three days writing, is not quite what the requirements spelled out (often because the requirements changed!), and then having to turn around and have to tell the project manager to tell the client that to get this software right, a day or two may need to be added to the deadline. The QA is often the bearer of bad news, and it has and go and come from two directions. Having the tact and internal fortitude to handle these two camps is something to be admired.

At the start of every project, we at Newfangled go over with the client something we call the project anatomy. This details out the steps that their project will take, from kickoff to go-live. One of the steps at the end is titled "QA." This probably should more aptly be named Quality Control, defined by Google as the "checking the software against a set of requirements and verifying that the software meets the predefined requirements", but for historical reasons, internally we call thisQA. This block of time happens at the end of the project to make sure the full product is what the client is expecting. If the QA team and the developers have done their job, this should be a breeze, with nothing more than checking off a bunch of successfully implemented requirements.

I took a little poetic license in this post in regards to the relationship between the QA and the project managers here at Newfangled. If you check out our employee page, not one person has the title of Quality Assurance Engineer. This is because our project managers and their assistants pull double duty and are our QA. With a small company, people usually wear many hats and our project managers are no exception. Our PMs can be working with the developer one minute discussing the minutia of some JavaScript widget, and then turn around and be on a training call with the client on how to use their content management system. The personality split needed for these two tasks is amazing.

Make no doubt, though, that Quality Assurance is something that happens throughout the project and not just at the end. If our QA process wasn't involved all the way through the project, the project would be late, it wouldn't be what the client asked for, and it would no doubt be over on budget.

This post was written partly to explain the QA process is and why I think it is key role of any development project, but also to highlight the fantastic work and effort that our project managers do to not only be the face of the company to the clients, but also to be the grease that keeps the Newfangled engine running smoothly.





Tagssoftware

FACEBOOK
 

New Technology, and Balancing the Fun with the Practical

May 5, 2009 at 9:08 AM by Nolan


Following technology trends on the web is a dizzying pastime. New programming languages rise to popularity seemingly every month and the buzzword-laden web frameworks make their introductions and dance on stage for half a verse before bowing out to younger, fresher technologies. When deciding which pieces of new technology to build into your web presence, there are several things that a developer should consider and some simple guidelines to pull in new, fun technology.

Obviously, the first attribute that a technology is measured by is its quality. Simply, is the code behind it good and sturdy? If you lean on it too hard, will it break? If you sense code smell, follow your gut and keep moving.

The next attribute I look for is community. You read about a new technology and it seems to be the silver bullet for your application. (We'll disregard for the sake of the example that there is no silver bullet.) Reading further you find out that the technology is driven by one developer. The online mailing list gets a question once every two days, and the IRC channel is frequented by the lead developer and a fistful of other passersby just checking things out. That is no community; that's a household.


Django Community Sprint, Photo by Nathan Borror

Now, almost every software project gets started with meager few people involved, and maybe this interesting new technology is in this stage of its life, but what if it's not? Maybe the lead developer hits the lottery and moves to the sunny French Riviera, never touching his MacBook Pro again except to order fine wines for his chateau. You now have a software project that has possibly no active developers and the project fades away to the great Internet Archive in the sky. If you had adopted this technology for a website, you are now riding with an effectively dead technology. This is not a good situation to be in when you need to hire someone that knows it to maintain your site, or to fix bugs that may pop up in the software.


The French Riviera, Photo by Serge Melki

With community, a software project has staying power. The more people involved, the higher chance it will be around longer.

Signs of good community:
1) A local user group
2) Corporate backing (not essential, but definitely nice)
3) Lots of blog posts and Twitter messages indicating that people are talking about the tech.
4) Active mailing lists, both for general help list and the developer list.
5) An IRC channel where you can pop in and find numerous people to help you with an issue.

All of these are not necessary to have a good community but the major software projects tend to have these in common.

My personal favorite trait in assessing a new hunk of technology is documentation. Well-written and kept-up documentation is something that makes most developers happy, but unfortunately writing documentation is not enjoyable as a task. If a software project has a fantastic demo, and then I click through to the documentation wiki and find more instances of "coming soon" than code samples, I usually step away.


Photo by Amit Gupta

Lack of documentation and time to implement a technology are usually inversely related. Little documentation results in lots of tinkering and cursing, fumbling in the dark and looking for that magic combination of words to get the spell working. Software is painful enough; building with it without proper guidance is torture. Some developers thrive on the poke-and-prod method of development, but when a client's money (and the developer's sanity) is on the line, a few breadcrumbs goes a long, long way.

Now that some simple guidelines on vetting technology has been laid down, how do you bring the technologies that look fun and promising, but may not be quite ripe, into the workflow? Easy. Start small and leave traces.

Starting small means don't build anything mission-critical on it. You definitely don't want to nab a big client and then decide to to build their website on a funky schema-less graph database you read about on Reddit. I can guarantee you they don't want to be your testing ground. Now, you may grab another client whose needs may line up for the new and edgy technology. Let them know what you are doing and be honest about your experience with the tech. No one enjoys any surprises in budget or functionality further along.

Leaving traces means let your co-workers know what you did. On your dev team's wiki (if you don't have one, you should), document out why you chose the technology that you did and how you used it, with lots of links and code samples. In case you hit your numbers on the weekend PowerBall and retire early, you don't want to leave your comrades high and dry poking through your code when a bug pops up.

The web is a fun place, with lots of smart people doing cool things every day. Playing with new technology is why a lot of web developers do what they do, and bringing this into enterprise doesn't have to be a pipe dream. So, feel free to experiment, but just be responsible.

*All photos licensed under the Creative Commons


Tagstechnology web-developmet

FACEBOOK
 

What is SSL?

February 13, 2009 at 11:00 am by Nolan

SSL (or secure sockets layer) is a way for a user to verify two things about a server they are trying to contact for requesting a web page. The first is that they are communicating with the exact server they are expecting to be communicating with, and secondly, that no one is listening in on this communication. SSL is essential for logins, transporting delicate information, as well guaranteeing the correct sender and receiver.

When should a website use SSL?

Basically, whenever you are transmitting something you wouldn't want a casual observer to view. If you work in an office building, a transaction may go through 2-3 hops before it leaves your building, and then it goes to out the local branch of your ISP, and further up through the various conduits (tubes, if you will) of the Internet, until your packets find the server it's aiming for. A traceroute from where I'm typing this post at work to "google.com" visits 10 different servers before getting to the Google server. If you are transmitting vital information unsecured, these are 10 different spots where someone could read your traffic. If you are using high-grade SSL, the snoop will see a stream of seemingly random bits, and even if he gets the entire conversation saved, it would take about 1000 times the age of the universe to brute-force attack the key. Obviously, there are better methods than brute forcing, but with a large enough key, you are good for several years. As of now, high-grade SSL is effectively unbreakable.

Several industries are required by law to use some sort of SSL. Bank account access must be secured, as well as most health care operations. Any transportation of credit card information must be secured. Most major webmail clients allow you to access them via HTTPS. Even Wordpress will allow you to switch on HTTPS to access its admin section. My suggestion usually is if you are transporting anything of value over the Internet, mainly username and passwords for any important business, an SSL key is recommended.

How do I get an SSL certificate?

If you are hosting a site with us, just ask us and we'll set you up. If not, Verisign and Geo Trust are two good options. Even registrars like Go Daddy allow you to buy SSL certificates from them.

With an SSL certificate, you will want to make sure it is signed by a recognized certificate authority. Since anyone can make an SSL certificate, there is no inherent trust created, since you can't tell the difference between a certificate signed by one of the trusted authorities, or some malicious entity. The trust is generated when you have one of the major authorities verify who you say you are. Browsers come pre-installed with these major authorities' root certificates so you can check the authority's signature on the SSL certificate. This is basically a "trust by association" model. You trust Company A, and Company A says Online Store B is trustworthy, therefore, you are able to trust Online Store B. A non-trusted SSL certificate will give you an error in the browser, alerting you to the possibility that the certificate has not been verified by one of the big certificate authorities.

How does SSL work?


This is a fairly technical explanation, so if you don't have any interest in the ins-and-outs of SSL, you can skip this section with the take away that SSL is an extremely secure and a neat way for browsers and servers to setup secure channels.

With that said, I'll commence with the explanation.

When a browser makes an HTTPS request, the browser first does an DNS lookup on the hostname to get the IP address. The browser makes a connection to the server at this IP address on port 443 (by default) to start setting up the secure connection. The server replies with its public SSL certificate, which the browser will then verify, checking its signature against the pre-installed certificate authority keys. This certificate authority's signature gives you faith that the entity you are talking to is whom you expect it to be. The SSL certificate from the server contains information such as what hostname the certificate was built for and how long the certificate is good for. It may also contain other information about the business, such as contact information, but this is not required. This step is where the browser will show SSL errors to the user, such as the hostname mismatch or certificate expiration. The most common error that people see is the hostname mismatch. If you typed in "https://www.example.com" and the SSL certificate was built for just "example.com", you'll see the mismatch notice.

The data is transmitted encrypted using symmetric key cryptography. Symmetric key cryptography involves the sender and receiver (in this case, the browser and the server) both having the same key which is then used to both encrypt and decrypt a message. The advantage of this type of protocol is that encryption and decryption are very fast while still being incredibly secure. The downside of this is the issue of how to get the key from the client to the browser without anyone in the middle stealing it. This is where another kind of cryptography, public key cryptography comes in.

Public-key cryptography consists of the message receiver having two keys, one being public and the other being private. To send an encrypted message, the sender uses the receiver's public key to encrypt the message. To decrypt this message, the receiver uses the private key against the message. The public key is available for anyone to use, but the private key should be never be shared with anyone, because if this key gets out, then any messages encrypted with the public key can be decrypted by anyone possessing the private key. A good analogy for this type of cryptography is to think of it like a mailbox. Anyone is allowed to drop a letter in, but only the owner of the mailbox has the key to unlock it and get the letters out.

So after the browser makes its request to begin the secure transaction, the server will reply with its SSL certificate, which contains its public key. After the browser checks out the authenticity of the certificate, it will generate a brand-new, temporary symmetric key. The browser will encrypt this new key with the SSL certificate's public key, and send it back. Since only the server is able to decrypt this message and get the symmetric key out, the symmetric key has been safely transported from the browser to the server, with no one being able to snoop on it while in transit. From then on, the browser and client will do all of its encryption and decryption with this symmetric key, due to its speed advantage mentioned earlier. Thus, a relatively fast and secure channel has been created between the server and browser.

To sum up, here's a quick rundown of the full SSL transaction:
  1. The client requests an HTTPS URL.
  2. The server replies with its public certificate.
  3. The browser will verify the authenticity of this certificate. If it's not valid in some way, the browser will show the user the problem(s) with the SSL certificate and let them decide what to do.
  4. If everything checks out from step 3, the client generates a brand-new symmetric key, encrypts it with the server's public key from the certificate, and sends it back.
  5. The server decrypts this encrypted key with its private key and sends back a message saying that the connection has been made. This message is also encrypted with the symmetric key.
  6. If the browser can decrypt this message, then a secure channel has been created and communication can continue with everything being encrypted with the new symmetric key.
  7. At the end of the session, the symmetric key is destroyed.

This protocol completes the two goals of SSL: secure communication and guaranteed authentication.

The cool thing about the SSL protocol is that the secure channel is created before anything is transferred, even the URL itself. That means if you request "https://www.example.com/some_secret_resource" the browser does a DNS lookup of "www.example.com", goes through the above steps, and after the secure channel has been created, then the full URL is sent to the server. The DNS lookup, if it doesn't come out of the client's DNS cache, will be the only way to verify what server the user is trying to contact. Someone on the outside can tell the client is trying to contact a certain IP address and see the encrypted traffic, but that's basically the sum total of an observer's view of the transaction.

--

In summary, if you don't want an outsider to see what you are transmitting between the browser and the server, an SSL certificate is a simple way to get a high-level of privacy and security.






Tags

FACEBOOK
 

Keeping Your Site Safe from the Bad Guys

January 27, 2009 at 2:05 pm by Nolan

One thing that has evolved almost as quickly as the web itself is the rise of security problems associated with it. Web developers today have to take great care in not introducing security holes into websites. These security lapses can come from something as seemingly innocuous as a simple contact form.

Security weaknesses on the web generally come in three major forms, cross-site scripting (XSS), cross-site request forgery (CSRF), and SQL injection. These acronyms may sound intimidating, but the principles behind them are fairly simple to understand.

1) Cross-site Scripting (XSS)

XSS is basically allowing someone to execute their own Javascript in the context of your site. If a malicious person manages to insert their code to run on your page and you visit that page with the malicious Javascript on it, the hacker can fetch the user's cookie, and as most sites use cookie-based authentication, this means that the villain can now appear to be you on the vulnerable site. Not good.

The hard part for the malicious user is getting his evil-doing Javascript onto the attacked website in the first place. The number one place where security breaks down is through forms on your website. Often, poorly-built web forms will display on a page whatever information the user puts into a web form. A good example of this is a search box. A common user will type in his or her search query and then on the results page it will say something like "Found 13 results for x" where x is what the user typed in. When the user plays by the rules, this is not a problem. The issue arises when a malicious user carefully crafts their search query to include HTML that gets printed out exactly the way it is put in. When this happens, the bad user can put their own Javascript on your site.

For a major type of XSS, the malicious user gets an unsuspecting victim to click a link. This link will do the insertion of the Javascript in the victim's browser, and when the Javascript is executed, the villain can do anything from having any forms on that page get submitted to him, redirect the page itself, or steal the cookie used to authenticate a user to a website.

The key to avoiding this type of attack is to never, ever, ever trust user input and always assume the worst. The biggest majority of XSS holes can be closed by simply escaping the HTML special characters (<, >, and &). If you escape the greater-than and less-than signs by converting them to their HTML entities, you close off almost every chance of the user inserting their own HTML/Javascript on to your page.   

2) CSRF

Cross-site request forgery is when a user, specifically a logged-in user, makes a request to another website unknowingly.

Let's say you have a button inside of your web application or CMS that deletes your account. Now a malicious user on another website could set up a button on his website that appears to be an inncocent button that he states takes you a game download page, but in fact is a copy of the delete account button on the original site. If you are logged into the original site and then hit the button on the malicious person's site, this delete process may execute just as if it was coming from the actual site. If the original form processing code doesn't take extra care to verify that the form submission is indeed coming from where site is expecting, bad things could happen, like your account getting deleted.

CSRF doesn't only happen through form submissions. This can also happen through normal URLs if the URLs modify the state of your data (e.g., change username, delete account, submit a link to a social media site).

One way to fix this is to have a secret key that is submitted along with the form in a hidden variable, or a secret key as a URL parameter. This should be something like an MD5 hash of the concatenation of the user's ID and a secret password known only to the internal code of the site. When the processing is done for a request, your processing code will make the sure the user is logged in, replicate the original hash, compare it to what was passed along in the form, and then, if they still match, process the request.

 

Another way to defend against this is to check the Referer header of the request. It probably best to use the hidden token method and Referer check together as there is no such thing as too much safety, and the code is not significantly more complex.

To sum up CSRF, any time a request is received that depends on the user being authorized, extra work must be done that verifies that the request is coming from your site, and not some other mailicious party.

3) SQL Injection

As soon as websites started to become driven by databases, SQL injection vulnerabilities appeared. SQL injections are when a user's data causes a database query to differ from its expected function, usually in a bad way.

An example:
$user_id comes directly from user input, probably via a form.

Your query:
DELETE from user_accounts WHERE user_id = $user_id;

Now if $user_id in fact equals the user's ID, you'd be in a good shape. This is a huge hole, though. What if the user passed in another number or, even worse, something like "1 or 1"?

The executed query would look like this:
DELETE from user_accounts WHERE user_id = 1 or 1;

This would delete every single row from your user_accounts table. Ouch. Obviously, this is a worst case example, but not by much.

This may seem obvious and you may put in some safety checks. You would probably make sure user_id is a number and have some additional verification that this number hasn't been tampered with. The issue in preparing against SQL injection is that you constantly have to keep track what data you should be getting and inserting this safely into your query. This usually means taking care of single and/or double quotes, commas, type checking (am I expecting a number or a string?), as well as SQL keywords. This is a lot to remember and it's prone to user error.

Luckily, almost every modern database has a piece of functionality called "parameterized statments" and most languages have an interface to use them. Prepared statements let you build your query, putting in placeholders where your data goes, and then the SQL engine will build out the query in its syntax tree, and then as a last step before executing, it will substitute in whatever data you passed in, without making it part of the query itself.

Basically, this means that the parameter data and the code of your SQL query are kept separate and you don't to worry about your data changing your code, or vice versa.

---

Security is one of the things that developers do that when taken care of properly, the client never notices, but if something slips up, it (or the lack of it) comes front and center. The main source of security holes on a website, with a little extra work, can be easily avoided.


Tags

FACEBOOK
 

 1 | 2   Next >>