SEARCH
diglloydTools™
|
|
|
$610 SAVE $110 = 15.0% ZEISS 32mm f/1.8 Touit Lens for FUJIFILM X OUT OF STOCK in Lenses: Mirrorless
|
|
Backing up High Capacity Storage: Partitioning for Practical Backup
Related: backup, Carbon Copy Cloner, cloning, How-To, Other World Computing, OWC Thunderbay, RAID, weather events
See How to Clone a Volume for basics on cloning as well as How to Safely Transfer Data or Verify Backups.
Key terminology: Volume vs Drive.
High capacity multi-drive storage has been around for years, but with the emergence of Thunderbolt, more and more external storage options in very high capacities have emerged, such as the OWC Thunderbay 4.
With a 4-drive Thunderbolt enclosure, capacities using hard drives as of late 2014 can range up to 24TB (4 X 6GB). Popular options include 3TB or 4TB configurations, for 12TB or 16TB of capacity.
Formatting/organizing many terabytes
For this discussion, let us assume that there is 16TB of storage, e.g. four 4TB hard drives in a 4-bay OWC Thunderbay 4, e.g., this unit. There are many ways to organize this storage, but here are three approaches:
- Four independent 4TB volumes. Each drive is one volume (one partition with all the space), and each is independent of the others.
- One large RAID, for example a 16TB RAID-0 stripe, or (better), a 12TB RAID-5 which offers fault tolerance.
- Multiple RAID volumes, each using some portion of each of the 4TB drives.
The best answer for most users is to eliminate the middle option, for backup reasons. Remember that RAID is not a backup.
Backing up and chunking
A preferred way to backup is by cloning. In this case, a 4TB external drive matches the capacity of each drive in the 4-drive unit, so that it can contain clones of any volume up to 4TB capacity.
But how does one backup a 16TB or 12TB volume to a 4TB external drive?
Patient: Doctor, it hurts when I do that.
Doctor: Don’t do that!
So the answer falls out of the backup requirement: don’t do that. Meaning do not make one large RAID volume, because it cannot be cloned to a single backup drive* (too large)**. It sounds convenient, but it is not so when backup is considered.
* One can introduce complexity of partial backups to deal with the capacity mismatch but MPG recommends keeping backup processes as simple as possible.
** Of course one can purchase multiple matching 4-bay units to match capacity for cloning, but even one backup is now quite expensive, even if the capacity is only partly used. Yet MPG recommends at least two and preferably three backups.
Chunking
For matching original data to backup capacity, “chunk” the capacity into multiple volumes, each matching the size of the available backup drives/volumes. With RAID, this means limiting the size of any RAID volume to the chunk size. With single drive/single volume (no RAID at all), this happens naturally.
Taking our 4 X 4TB case, the chunk size is 4TB: master volumes (originals) are 4TB each (max, could be smaller if desired), and backup drives/volumes are 4TB each.
There are good reasons to “chunk”:
- Chunking forces some thought about how to organize one’s work (by job, timeframe, content, etc). This is huge plus over time.
- Chunking allows dedicating a fast drive to key work rather than commingling current work with 2/3/5 year old stuff (perhaps using the slowest part of the drive for current work!)
- Redundancy is king with backup. With chunking, several backups of key work (a particular volume) can be made, and these can be physically separated. If one fails, the others are still there. Moreover the backups can be staggered in time, in case of user error with the originals. Contrast that to 4 drives in a RAID used as a backup, which cannot be separate, and each such backup is expensive.
- Often there is old or archived work that has long since been backed-up; separating such older work means that backing up the newest work is fast and obvious and less likely to be put off.
Specific example
Need help deciding on the best approach? Consulting is available.
Assuming 4TB drives, 4TG chunk size.
- Main storage: OWC Thunderbay with 4TB drives (this unit), each drive formatted independently as a 4TB volume.
- Backup drives: 4TB external drives.
Configuration example
One possible configuration of the four drives inside the OWC Thunderbay:
- Volume Master for key work (one drive, one volume).
- Volume Archive for older work (one drive, one volume).
- Volume tm for a dedicated Time Machine backup (for boot drive and Master, exclude the volume Archive as it is older backed-up work and won’t fit).
- Volume MasterClone for a clone of Master. But could be appropriately partitioned into a BootClone (system drive) and MasterClone.
If using the Thunderbay RAID-5 edition, MPG advises a 2nd Thunderbay unit, because the tm and MasterClone volumes would be sharing the same drives as the originals they are backing up.
External drives:
MPG recommends no fewer than two (2) backs for each source drive.
- MasterClone1 and MasterClone2
- ArchiveClone1 and ArchiveClone2
Four external backup drives as these four volumes. Make backup clones appropriately to these drives.
