Much More About Backups

All the details you need - and probably more

By Jonathan Stars

Backup Routine
(Don't let this first method bog you down. Other suggestions follow, so keep moving through the article until you find what you need.)

There are a number of backup approaches I've seen recommended over the years. The one I use seems to be a good compromise between the different methods I've seen. This process describes what happens the first 4-week cycle of the backup.

I run a backup of all my files at least once a day for two weeks, using a different removable media device each day, which might include external hard drives, Solid State Drives (SDDs including memory sticks) or rewritable or permanent CDs or DVDs. Assuming a five-day workweek, I archive the copies at the end of days five and 10. At the beginning of the third week, I reuse the media I used on the first day of Week 1. I do the same until the last day of Week 3 when I bring in a new blank media. You can't use media #5 because you're archiving it. I continue the same procedure of reusing media and archiving the last copy of each week. The following figure should help you better understand the process.

In this diagram, each number represents a separate physical media. At the beginning of Week 3, reuse media #1, which you used on day one of Week 1.

After another month goes by, you can begin reusing your archived copies, but always keep at least one archive for each month. Archive more often if you're in a period of heavy development. Put a date on it for the day it went into service. I'd start to question the reliability of media that gets to be three years old. That includes the most reliable, 5-year warranty Seagate drives. They'll replace the drive, but not your lost data - and which is more valuable to you?

Why not just overwrite your backup every day? Well, what if some of your customer records had been accidentally (or intentionally!) erased and it took you some time to discover it? You would try to retrieve the backup you made yesterday, but the missing records disappeared two weeks ago. A perfect backup of missing or corrupt data is not a backup at all. That's why you want to keep archival copies. That way you can go back as far as you need until you find a copy of the files before the data was erased or damaged.

Testing
Whatever your method, the worst thing in the world is to set up a backup schedule and not test out the backups until the day they're needed. It could be tragic to discover that you weren't backing up “properly” after a disaster hits. Every so often you should try to reconstruct the files from the backups to see if they work. Be warned: Some backup software has settings that overwrite the original file. Make sure you change that setting before you attempt the recovery test. And don't trust the software when it tells you the recovery has been completed successfully. Open the files and muck around a bit to see if they're really OK. Just don't get them mixed up with your current set of files.

Testing will also help spot other problems. I had one client who told me their backup process only took a second. That made me nervous. I stopped by their office and it turned out they were faithfully backing up the shortcut (known as an "alias" on the Mac), which was sitting on their desktop. They had many copies of the shortcut and none of the actual files that it opened. So, test those backups!

While You Work
Depending on how much data you can live without, you may want to run a backup more than once a day. I have some customers who back up their main data files every couple of hours. When I work on files, if I've put in three or four hours, I'll make a backup. I'm not anxious to do all that work over. And let me tell you, when you're developing a set of files, it's not too hard to cause a crash. Backing up only takes a few seconds. Trying to recreate your work takes much, much longer.

Using Commercial Backup Applications
There are some good products on the market for both Macintosh and Windows machines that will take care of scheduled backups for you. Retrospect for the Macintosh (although I find the interface confusing) and PC Backup for Windows come to mind. You can also program AppleScript to perform backups on the Macintosh. In Windows NT, you can use NT Scheduler. I'm currently using SuperDuper! for the Mac because I find its interface much more clear than Retrospect. They also have great customer support. But it requires you backup an entire hard drive or partition to another hard drive or partition. You can't simply target a few folders.

You can (but shouldn't) perform any of these backups while any FileMaker files are open and running. It's sort of like trying to move a container of milk full to the brim without putting the lid on. You're gonna spill something. Quitting FileMaker puts the lid on any open files and prevents you from "spilling" any of the data.

CAUTION
I don't like to see Time Machine working while I'm importing audio from tape or while I'm working in Pro Tools or iMovie. I'm suspicious that it might be competing for processor time and mess up the final product. I'd prefer to trigger the backups manually. But it's so easy to forget. I suggest setting a kitchen timer — not a timer on your computer.

I highly recommend that you back up to removable media or a portable hard drive. Don't forget to remove it from the computer and unplug it from the wall after backing up. Using another drive inside the computer is not a good choice because it's also subject to damage from power surges and lightning. And you should have multiple, portable backups anyway.

Apple's Time Machine
If you use a Mac running OS X v10.5 (Leopard) or later and you use Time Machine (TM) for backup, you may feel pretty comfortable about skipping the method I've been talking about. You would be wrong to do so.

The process of backups established over many hard earned years is described as Automatic, Redundant and Restorable or ARR. It's a slight misnomer because, although the software can provide automatic backups, there is nothing automatic about taking that hard drive with you. (That's not the case if you use a process that backs up to a remote hard drive and use TM as a secondary backup.) Anyway, Apple's TM system does not provide the Redundant part of ARR. If you have a flood, fire or theft and you lose not only your main drive but your Time Capsule backup hard drive, you're outta luck, Chuck! The software will let you target different drives on different schedules. You can probably guess that's what I would suggest.

Don't get me wrong. I use Time Machine and like it. But I use it along with my other method. The trick is that I have two hard drives that I swap out every other weekend. When I visit my mom, I drop off this week's TM drive and pick up the one I left the week before. If the Wicked Witch of the West flies away with my house, I'll have a copy of my files to get me going again.

I currently have a stack of inexpensive ($100) hard drives I swap in and out. Over the years I've wasted so much time getting everything back to normal after crashes, I've come to the conclusion that my time is worth too much to mess with slower media or compressed backups. Part of that decision has been influenced by the steady increase in hard drive space along with the steady decline of hard drive prices. In 2008 I was able to buy Seagate drives with a five-year warranty for about 16 cents per gigabyte. In 2010 it was 10 cents. This downward trend in cost per gigabyte should continue, meaning that the cost of your backup "insurance" is dropping and decreasing your excuses for not doing it.

For your daily backups, make a copy to some medium and take it somewhere off the premises. Then use a second drive to backup the next day and take it off premises and pick up the one from the previous day to bring it back. Keep rotating them and you'll at least have minimal protection.

CAUTION
Don't count on any backup media forever. You may have heard that the durable (but scratchable and meltable) CD and DVD have an expected shelf life of 25 years. It turns out that those figures only apply to factory printed media, not the kind you burn yourself, which only have a shockingly low shelf life of as little as 5 years. (That outta make you nervous! I sure surprised me.) Every other media falls somewhere short of that.

Keep moving your files to new media, roughly every 5 years. And you will probably need to import early file types into new programs to convert them. I have some old Photoshop files that even the newer versions of Photoshop won't recognize.

Roger Nicols who worked as sound engineer on tour with Steely Dan in the '90s had an experience you should be aware of. He recorded every show to hard drive. A few years later he tried to play the drives. They had "frozen" up. Apparently you need to "exercise" a hard drive every 6 months or so to keep the lubricants from drying out. It is possible to have the platters removed from the drive and put them into a new mechanism, but it must be done by a professional and it's very expensive. So along with all your other procedures I've talked about here, put an exercise schedule on your calendar. All you have to do is add and delete a little data from each drive to keep them from gumming up.

Hard Drive Problems
The driver cards in hard drive cases are notoriously unreliable. If you have a drive that goes bad, try putting it into another case. (That can be a little scary for somebody not familiar with electronics. But it can be done fairly inexpensively by a local computer shop.) If the drive is clicking loudly, it's probably not the driver card. (Loud clicking may be a little difficult to define, because there is a soft whirring, and clicking when a good drive is running. Loud clicking is definitely a different sound.)

What to do with your backups
Whatever media you choose, date each backup and move it off the premises. Get into the habit of taking the most recent backup home with you when you leave for the day. Don't do anything with it. Just take it and bring back the copy from the previous day. Time Machine or some other onsite backup can cover for you when you can't run home after yesterday's backup. That should also cover you for the days you can't make it in. If you were the only person to have a backup when it was needed, your co-workers would be stuck. So if you go on vacation, have somebody else keep track of the backups.

Another offsite backup method would be to use one of the Internet disks that are now available, such as Apple's iCloud. This method of storage provides users with a low-cost, offsite, easily accessible, backed-up hard drive. Of course, using such a system would depend on how secure you feel about the possibility of your files being intercepted while traveling across the Internet or sitting on someone else's servers. You'll also want a fast Internet connection to take advantage of this option. I found it a bit slow for my taste. But you can schedule an automatic backup in the middle of the night, so the length of time it takes won't matter to you. You'll also have to pay extra if you exceed the space limits allowed in the basic package. Another option might include DropBox.

Your System Drive
For many years I had heard that you only need to backup your data and don't need to backup your system drive or the applications. The theory was that you can always reinstall from the original system and application discs. Well those days have gone, and I'll tell you why.

We now receive system and application updates regularly via the Internet. In many cases we purchase and download applications from the internet and never own the software discs. Here's how I found out...

(If you don't what to hear my sad story, click here to skip ahead to the section titled "What to do.") In 2005 my system hard drive went bad. I put in a new hard drive and installed the system software from the last set of DVDs I had purchased. As soon as everything was running I got a notice that there were system updates ready for my computer. Of course I clicked the button that invited me to download them. After it started the process, I got a message that it couldn't install until I downloaded and installed an intermediate version of the software. I went to Apple's website, but that version was no longer available. Now I was stuck not being able to bring my computer up to date. Well, after a few hours of Google searches I was able to find someone who had posted the the intermediate version on their website. I was lucky they weren't some malicious hacker, because I didn't know them and could have easily gotten an altered version of the update that could have really messed up my computer.

So after a couple days of fooling around I finally had the system running and up to date. Now I began to install all my favorite software applications. Find the discs, install, download updates, repeat. But then I ran into a problem that some software wouldn't install from the discs because the system software was too advanced. It turns out, I had bought some of the software which was married to an earlier version of the system. When I had originally upgraded the system software before the crash, I also downloaded the updates for the application. But because the computer now had the latest system software installed, the applications were out of step and un-installable. So I started calling the software manufacturers to get the kind of personal help that requires finding the serial numbers (some of which do not match the ones on the discs because each upgrade requires a new install code), and many hours of idiotic phone trees and being kept on hold from companies who don't have toll free numbers. All this time my business is on hold - I'm not doing client work.

Then there are the software applications which never had install discs. Anyway, it took a full week to get running in basic mode and over two months of intermittent work of more than 25 hours before everything was back the way it was before the crash. Needless to say, the days of advice about only backing up your data have long since gone the way of the horse and buggy.

What to do (System backup) You need to backup your system drive. I do this about once every 2-4 weeks or whenever I add one or two applications that I don't want to lose.

Before starting this backup, if you're using a Mac, use Apple's Disc Utility to Repair Permissions.

I start up the computer from an external Firewire hard drive (you can only startup from a USB drive on some Intel based Macs) which is running the system software along with a copy of Carbon Copy Cloner or SuperDuper! (You may be able to start from a DVD with a minimal version or the system and one of these programs, or from an internal hard drive or partition set up for this purpose. By the way, these are Mac applications. I don't know what to tell you Windows folk.) Then I backup to another external drive that is the exact size and hardware style (ATA, SATA, etc.) as the internal drive running the system software. This can be in a USB enclosure (Firewire is not necessary here). Both these backup programs will make an exact, bit-for-bit copy of the internal drive. It is possible to use a bigger drive with one partition of the same size or larger than the drive or partition you're copying. Just be sure your machine will support a drive of the total capacity.

The first time you make such a backup, plan to let it run over night. Save the settings the software used to make that backup and make notes about the settings so you can call them up the next time. The next time you run the backup, it will only take the amount of time needed to update the changes since the previous backup. For me it often takes only 7-20 minutes, certainly not overnight.

The result is that if your main drive ever goes bad, you can remove it and put the backup drive right into the same drive bay and be up and running in a matter of minutes. I will also backup just before downloading any system updates, since I've often found that the new "improved" system software removes features and functions I feel I absolutely need. (For instance, one of the versions of iMovie removed a lot of features in their attempt to make it easier to use. You couldn't step back to the previous version of iMovie because it and the system were married to each other and, once again, the previous version never existed on a DVD. I used my backup drive to step back to the old software and waited for Apple to correct their mistake. This is one place where Windows users have an advantage. They have system restore.)

But you should apply some caution with this method. If it turns out your mother board is fried, and you put the replacement drive in the bay, you can fry that drive, too. Yeah, it seems there's always some alligator waiting for you in that beautiful, blue, digital lake. You should probably take your machine in and have it diagnosed before you put that drive in the bay. Bench fees are about $75. A new mother board can be between $200 and $500. And maybe your original hard drive is OK, anyway. But if you eat up both your system drives and have to get a new motherboard and spend 25 hours restoring all your software, you're going to be very unhappy. I actually have two backups of my system drive which I alternate between. One is always onsite and one is stored remotely.

The (wild and crazy) End
OK, full confession here. I have over 32 hard drives. I know that sounds crazy to most people. But here's why; I converted over 200 reel-to-reel audio tapes, 1000 cassettes and 100 videos to digital, which filled up 7 hard drives. Two backups of each of those (which I consider minimal) and you're already up to 21 drives. A couple backups of my system drive and a couple of my data drive and we're up to 27. The remaining drives are old, smaller drives I just haven't discarded yet, or for video and audio projects that get worked on then cleaned off after the project is done. So you see, it's not really compulsive drive collecting after all. It's my job. That's why I consider myself an expert when it comes to advising you about the backup process. It comes from a history of pain. And if you listened carefully, you won't have to be in that world of hurt when your hard drive goes south.

Contact me for help with this.

J **  

© 2011 Jonathan Stars