Lessons from Tuesday’s Gmail Outage
Sep 3, 2009 by Aaron Rubman
Tuesday’s Google Apps and Gmail outage from 12:30 pm to 2:10 pm marks the company’s third high profile outage this year (the others were on January 31 and May 14).
This time around, more than half of Google’s free and business subscribers received an error message informing them that their browser could not connect with their e-mail server.
While the Gmail iPhone app was originally unaffected, some of its users eventually reported that their service went down as well.
Initial reactions in the blogosphere were fairly hostile, predicting that this most recent outage would give Google a black eye in its attempts to convince corporate clients to move their communications to Google’s cloud (a large collection of interconnected computers with built in redundancy so that even if one or one hundred of them go down, the end users will still have access to their data).
However, now that service has been restored, Google’s response has, for the most part, been lauded. Engineering Director David Besbris posted a blog update in the midst of the outage suggesting alternate ways for people to access their accounts, and again just after repairs were complete to let everyone know of the resumption of service and the ongoing investigation into the cause of the outage.
Four hours later, Ben Treynor, VP of Engineering posted to the Gmail blog apologizing for the outage, summarizing it’s impact and duration, and explaining both the causes and the steps that Gmail was taking to ensure that similar outages would not occur in the future.
For more details you can read the Gmail blog.
Beyond Google
While Gmail was the most prominent e-mail service to have problems this week, the truth is that any e-mail system can suffer failures. Regardless of what service you use, this outage provides a good opportunity to re-examine your company’s contingency plans.
Identify Critical Services
Before we go any further, it makes sense to figure out what aspects of your business are essential within your regular model of operations, telephones, database access, functioning point of sale software, and e-mail are all likely candidates and whatever industry you’re in, it probably involves others.
Planning for Outages
For each of the critical services that you identified, ask yourself, can I do work without it? If so, how much?
By figuring this out in advance you can set policies in place that guarantee that you continue to be productive while awaiting the return of service. Contacting your service provider should be a part of these plans as a matter of course, but if you put everything else on hold while awaiting an response, you may be better served by shutting down.
A comparison of overhead and likely productivity after service resumes should help you figure out where this point should be.
Consider Redundancy
It still amuses me every time I see someone pull out an old fashioned credit card imprinter - but when a transistor blows out and a shopping district ends up losing its power, guess who can still take credit cards.
Local batteries and power generators can cover short power outages.
A mix of land lines, cell lines, and voice over internet services can increase your chances of having a usable telephone line in a crisis.
By having e-mail addresses with multiple hosts you will still be able to send out e-mail if your primary host goes down.
The Balancing Act
Each of these measures is a form of insurance against service outage. As with any insurance, it doesn’t come free. You as a consumer need to decide if the risk of lost business is sufficient to justify the expense of the backup solution you are considering.


As someone who uses Google as my primary email provider, this outage was a little scary. Luckily as Aaron pointed out, it was only the web-based platform that was down. That meant that since I’ve configured my Mail application, I was still able to send and receive messages outside of my browser. This would also hold true for any other email device such as a iPhone or Blackberry that accesses their services. Full instructions on configuring another client can be found here:
http://mail.google.com/support/bin/answer.py?hl=en&answer=13287
Also, I recommend installing Google Gears to work with any of their services that support it. It allows for offline access of Google Docs, Calendar, and a number of other services.
As more and more of our services move to cloud computing, Aaron is right that it’s a good plan to bring along a parachute.