Planning for Disaster

Are you prepared if your ISP goes down?

by Joel Snyder

Science-fiction writer Robert Heinlein coined the word "tanstaafl" as shorthand for a simple economic principle: "There Ain't No Such Thing As A Free Lunch." Tanstaafl is straightforward, easy to understand, and intuitive. People may not remember supply and demand curves or the first law of thermodynamics, but they can relate to the idea of "no free lunch." Internet mania, though, has turned normally rational businesses into free-market Marxists that desperately want to believe that the Internet, somehow, has become a source of free lunch.

This was the case with one of my clients, who has sales and support people all over the United States. Most of them work out of their homes. The client is overjoyed to have a presence in so many cities without having to build offices everywhere. Before last summer, the sales and support people communicated with the home office in Los Angeles using their laptop computers. They dialed an 800 number which linked them directly to modems in the home office, got onto the corporate LAN, and had access to all the network resources the company had to offer.

To keep in touch with both the home office and any customers, the sales and support people dialed in several times a day, often for several hours a day. They not only sent and read e-mail, but also did product demonstrations, searched technical support databases, and participated in online conferences. Of course, this ended up costing quite a bit of money.

Even though the company qualified for long-distance discounts, it was costing about $10 daily per person to keep them in touch; an expense of over $3,000 a month.

Jumping into The Net

Then along came the Internet and the lure of a free lunch. No longer would the company be required to maintain a pool of dial-in phone lines and modems or pay long-distance bills. Instead, the company would get a relatively inexpensive Internet connection, buy $20 per month unlimited access accounts for all its sales and support people, and cut its costs to one-sixth of what they used to be.

The free lunch part was simple: Someone else was paying for the long-distance lines. Getting from Washington, D.C., to Los Angeles was no longer the company's problem. Somehow, the Internet magically transported those bits from coast to coast, just as cheaply and quickly as it would move them across town.

Everyone loved it, especially the financial officer. It was proof that there is, in fact, such a thing as a free lunch.

That is, until the day the ISP shut down. Rats were found in the power transformers or something like that. But it was enough to take the company's ISP down, along with its connectivity to the rest of the world, for a couple of days.

The company, in its zeal to save money and capitalize on what it perceived to be a free ride, had not bothered to prepare a backup plan for its remote users to get access to the crucial data they needed when there was a problem. The company could easily have lost millions in sales just for being offline for a few days.

Best Laid Plans

Everyone agreed that a backup plan was needed. And so the company called up another ISP and installed a second leased line so that if the first ISP ever went down again, it would still be able to get out onto the Internet. This solution was very popular in the home office as everyone had been taking full advantage of the Web, listening to live audio of Australian radio stations, reading PointCast sports statistics, and so on. The original line had become quite loaded and the additional capacity was a welcome addition.

This second leased line worked fine except for one minor detail that no one had thought of: Both lines were leased from the same dial-up network provider. The company had bought all of its dial-up accounts from the same provider so it could have maximum leverage for discounts as well as the possibility of having people roam all over the country with the same account. Eventually, the inevitable happened. The dial-up provider had to shut down due to technical problems, and both the original and backup lines were rendered useless.

There was much gnashing of teeth and cursing and pulling of hair because now those formerly happy remote employees were beginning to distrust the system.

What went wrong here? Poor planning exacerbated by the lure of inexpensive and seemingly trouble-free service. What no one seems to remember is that the Internet doesn't repeal economic good sense. You still get what you pay for. For this company, the old system had reliability, security, control, and a dedicated technical staff that kept it running. But it cost a lot of money. The Internet was cheap, yet no one bothered to ask the question, "What are we giving up with this solution?"

This scenario is by no means unique. Businesses all over the world are using the Internet as a substitute for a private corporate WAN. And many are doing so without thinking about what's being traded in exchange for such great savings. When you compare the cost of dedicated lines, the Internet gets even more attractive--it's a darn cheap way to link business locations around the country and the world. But that reduction in cost comes with a corresponding reduction in reliability, responsibility, security, and control.

Damage Control

If you're going to use the Internet as a crucial part of your business infrastructure, by all means go ahead. But make sure you make Internet connections with a full understanding of the trade-offs. Here are three rules for minimizing problems.

1. Have a backup system in place. If you have employees connecting to you via dial-up service providers, make sure that you still have phone lines at headquarters which can be used in an emergency. Never mind the expense of having phone lines sitting idle, and don't be tempted to "borrow" them for a special project. If your budget just cannot stand to have resources sitting idle, then make it clear to everyone that you're settling for second-class service in order to save money. If you can't admit that then you're being negligent.

If you're using dedicated lines to tie offices to the Internet, and thus together, install backup facilities for that service and test them. ISDN is an excellent way to get lots of bandwidth for the cost of a long-distance phone call. Either make an arrangement with your ISP to connect via ISDN if your primary link goes out or, better yet, have things configured to bring up a direct ISDN line between offices, bypassing the Internet. Program your Internet router to automatically bring up the ISDN line when it detects a failure. If your Internet router doesn't support automatic dial backup (as most do), get a new router.

2. Perform failure analysis. Build a disaster-recovery plan of your own. Look at every link in the chain between your network and your remote users or offices and ask yourself two questions: "What happens if this goes down?" and "How likely is this to go down?" There are bound to be a number of key components which will take you offline if they fail. Make sure you have spare parts for any critical hardware or software elements, or find out where you can get them in a hurry. Also your local ISP may be willing to make an arrangement to lend you a spare piece of equipment in case of an emergency.

Make sure you know who to contact if something breaks down in your network. A little research ahead of time for the right phone number can save you frustration and confusion when things are falling apart around you. You should also keep a file with identifying information for everything in the network. Your phone company will be very unsympathetic if you call to report a broken dedicated line but cannot provide the circuit number.

But don't stop at your own equipment. Look beyond as far as you can. How is your ISP connected to the Internet? Does it have sufficient disaster-recovery plans and equipment available? Recently, most of MCI's Internet service on the West Coast went down for most of a day because of a massive fiber cut. Several ISPs in my city, Tucson, Ariz.,were connected to the Internet only via MCI. MCI's customers were dead in the water. If the Internet is crucial to your business, make sure that any ISPs you connect to have multiple links to multiple providers. Sure, those ISPs will probably cost more money, but if you want better service, you'll have to pay for it.

3. Don't neglect security. If your corporate LAN is connected via a dedicated line, you probably need some kind of firewall. These come in all flavors, ranging from simple configuration changes to your existing router to dedicated systems costing tens of thousands of dollars. If you don't know what's best for your network, find a consultant--your ISP can probably recommend a good one--who can explain it to you and help to build a more secure connection.

Treat everything you send over the Internet as public information, unless you go to the trouble to encrypt it. This isn't as difficult or as expensive as it sounds. Digital Equipment Corp. has a wonderful product, AltaVista Tunnel, which encrypts traffic between a PC dialed into the Internet and a central site. (For more information and a free trial version, visit Digital's site at http://www.altavista.software.digital.com.) If you are using dedicated lines, you have even more options: besides Digital, all of the big firewall and router manufacturers have encryption capabilities that can keep traffic between sites safe from prying eyes.

You don't have to go overboard on this. If you were willing to send the data openly over the Internet in the first place, you probably don't need military-quality encryption. But anything is better than nothing.

The Internet can be an invaluable part of your corporate LAN or WAN. But it is essential to approach it with reasonable expectations. No technology, no matter how wonderful, is without its flaws and occasional drawbacks. But if one takes a few simple precautions like those enumerated above and plans ahead for any troubles down the road, the long-term benefits are well worth the effort.

Joel Snyder is a senior partner at Opus One in Tucson, Ariz.
Reprinted from Internet World magazine Vol. 8 No. 6 (c) 1997 Mecklermedia Corporation. All rights reserved.


iWORLD home