Contingency Design - Leveraging Google when your database server takes a holiday

Published in Web Development on Friday, May 21st, 2004

For loyal followers of 37signals, the idea of contingency design (and defensive design) is nothing new. Today we ran into a little mishap and came up with an interesting solution, leveraging the power of Google to help us out with good contingency design.

Background

For those who aren't quite sure what contingency design is, the following definition is provided from a 37signals whitepaper entitled "Contingency Design: Maximizing Online Profitability By Helping People When Things Go Wrong":

Contingency design
Contingency design is design for when things go wrong. It's the error messaging, graphic design, instructive text, information architecture, backend system, and customer service that helps visitors get back on track after a problem occurs.

What went wrong

As we all know, servers sometimes have hiccups, and today this happened to our mySQL server. This blog, hand rolled as it is, currently delivers everything, even the home page, by extracting the data from a database. So when the database server goes down, so does this blog (future versions of the script will generate static pages).

Now, when we (hastily) wrote the script for this blog, in PHP, we used the following very common snippet for connecting to the database:

// Connect to the database server
$link = mysql_connect( $server, $user, $pass );
if (!$link)
die('A connection to the mySQL 
server could not be established');

Unfortunately, if the mySQL connection fails, that piece of code will leave your users stranded and you looking kind of silly. Here are some examples from Google (I like this one).

Not very nice to look at, I know. Here is what you would have seen on our site. Blech, not smooth.

How did we make it better?

1. Deal with PHP

For starters, any good mySQL tutorial will tell you to use $link = @mysql_connect( $server, $user, $pass ); to surpress the PHP errors (note the @ symbol there - we left it out because, sometimes, we like to live dangerously). Easy enough. Now what?

2. Put Google to work

This next step has to do with the die message that we use.

Rather than simply feeding the user "A connection to the mySQL server could not be established", which is useful only to the code developer, we can use Google's site cache search and the related sites search to either give the user what they were loooking for, in cached format, or at least give them suggestions of other similar sites to help them on their way.

Here is an example of our mySQL failure contingency page.

How did we do it?

The html for that page is held in a variable, call it $html. By using die($html);, our fancy mySQL error page gets served up.

It looks good on you (and your clients)

In the end, good contingency design looks good on you. While maybe you wouldn't want to send a user to a similar site if they are your competitors, this works fine for our blog, and most users will apreciate this, and they likely won't forget you because of it.

Close those doors!

There are likely many other situations where good contingency design can help close situations like that described here. Feel free to share your ideas and examples in the comments.

Comments and Feedback

How ironic that after posting this, I went to ping-o-matic to ping, and I received a "Error establishing a database connection!" error on their site.

Great contingency design. It would be great to see more sites handle these errors like described above. My only suggestion would to make alert the visitor more graphically that something is wrong. Make some the text in red or background in yellow. This would call attention to that message first than you could guide them down to the other alternative.

Neat idea, Mike.

What about adding a "Report this error" button? I think I'm not the only one that likes to know right away when a database is on the fritz. If you're using databases on client sites (and who isn't these days?), having it forward to a personal device like a cell would provide comfort for client and team alike.

Great suggestion Will.

I haven't nailed down how to send messages to a cell, however including a simple mail("[email protected]", "Db Mysite", "Short message here"); (php mail) in the backend code would automatically send me a message when this happens.

Off I go to try and figure out how to send a note to my cell...

Maybe this can be of help: Send a SMS

Not sure who your mobile carrier is, but many allow you to email messages to phones, with an email address of, say, [email protected] (the numbers being your 10-digit cell number). I used to have a simple PHP form on my site that did just this, and it worked successfully on my Cingular account. Anyway, excellent work on the contingency design... v. inspiring! (Makes me want to go back to my old hand-rolled blog, too.)

Thanks you two for the suggestions. Looking forward to digging into this...

If you're using databases on client sites (and who isn't these days?), having it forward to a personal device like a cell would provide comfort for client and team alike.

I don't think that is a good idea at all. It may seem clever on the surface, but what happens if your popular website gets hundreds of hits, and consequently hundreds of database connection errors? Would you want hundreds of text messages sent to your cellular device?

Haha! Been there, silly me; not once, but twice!

One trick is to store the last time a note was sent to you, and then don't send for a certain period of time beyond that.

Indeed, some sort of server variable, or the use of a flat file, could probably be used to manage it. Seems like a lot of effort for a blog, but it might be a good idea for a business project.

While I'm posting, can I request you add a cookie to the name, e-mail, and URL fields on the comment form?

Yeah yeah, I know ;-] Cookies. I was re-writing the commenting system but that has moved to the backburner. I'll add in a cookie first thing tomorrow morning...

Something like this (the contingency thing) is very easy to do. You just take out whatever message that you currently die ("have here") and slip in the code for a page. Since it's just a db error, all of your css and images are there to be used. Very little effort for something that looks miles better than an error message!

What an elegant solution! I hate the thought of visitors ever seeing a cryptic "programmer" error message and have tried to make all of mine "friendly" - this is even better, because it's useful to the user.

As you say, "they likely won't forget you because of it." Too right they won't.

Thanks for the tip.

A nice little cookie test.

Hey Darkblue, thanks. From embarassment rose innovation ;-]

You're welcome! :-)

Excuse the pointless comment. I am just baking my cookie for when I grace this establishment in the future :)

Just one little comment. I do not agree that good tutorials will tell you to use the @ to suppress errors. Bad tutorials will tell you this. Good tutorials will tell you to

  1. Set error reporting to E_ALL to flag ALL errors and never supress and error message.
  2. Set PHP to log errors rather than sending them to the screen.

Now, you have the full benefit of all error messages whether they occur for you or some visitor, but no error output is ever sent to the screen. Meanwhile, on your development platform, send all errors to the screen.

Cheers Mike... still working on my ski site! Sharpen your edges and get away from the computer a bit this winter, will ya?

Tom

Hmm, good call Tom, thanks. I usually try and not suppress errors, I guess I wasn't thinking too hard when I wrote that.

Good luck with the ski site. Maybe I can contribute something from the Euro ski scene?

Hello. I have been trying to get this idea working in ASP, but so far have had no luck. Does anyone have some sample code solution for asp (classic, not NET).

Home » Blog » Web Development

Check out the blog categories for older content

The latest from my personal website,
Mike Papageorge.com

SiteUptime Web Site Monitoring Service

Sitepoint's web devlopment books have helped me out on many occasions both for finding a quick solution to a problem but also to level out my knowlegde in weaker areas (JavaScript, I'm looking at you!). I am recommending the following titles from my bookshelf:

The Principles Of Successful Freelancing

I started freelancing by diving in head first and getting on with it. Many years and a lot of experience later I was still able to take away some gems from this book, and there are plenty I wish I had thought of beforehand. If you are new to freelancing and have a lot of questions (or maybe don't know what questions to ask!) do yourself a favor and at least check out the sample chapters.

The Art & Science Of JavaScript

The author line-up for this book says it all. 7 excellent developers show you how to get your JavaScript coding up to speed with 7 chapters of great theory, code and examples. Metaprogramming with JavaScript (chapter 5 from Dan Webb) really helped me iron out some things I was missing about JavaScript. That said each chapter really helped me to develop my JavaScript skills beyond simple Ajax calls and html insertion with libs like JQuery.

The PHP Anthology: 101 Essential Tips, Tricks & Hacks

Like the other books listed here, this provides a great reference for the PHP developer looking to have the right answers from the right people at their fingertips. I tend to pull this off the shelf when I need to delve into new territory and usually find a workable solution to keep development moving. This only needs to happen once and you recoup the price of the book in time saved from having to develop the solution or find the right pattern for getting the job done..