which earn me advertising fees or commissions.
As an Amazon Associate I earn from qualifying purchases.
Why You Need More Space Than You You Need
Related: hard drive, how-to, optimization, RAID, SSD, storage
Be sure to see the follow-on example page Larger Hard Drives Are Faster Than Smaller Ones for another example done in late 2010.
Hard drives are fastest on the outer area of the drive because they use a constant data density; recall the formula for the circumference of a circle:
circumference = π * diameter
The outer area of the drive is about twice the circumference of the inner area, which means that twice the data will pass under the drive’s read/write heads with each rotation of the platter.
The speed ratio is usually close to a 100/60 (outer/inner) ratio, but it varies somewhat by drive. For example, a drive that is capable of 120MB/sec on the outer tracks might yield 72MB/sec on the inner tracks.
The schematic below depicts the difference in circumference (not to scale).
This speed differential leads to a valuable performance trick: we can place our most important and frequently-accessed data on the Outer Partition, leading to higher performance:
- seek time is minimized (so long as activity is confined to that partition), because the disk heads traverse only a narrow band;
- data transfer rate is maximal because more data passes by the read heads with each revolution of the platter.
Seek time matters for some uses; this is the time it takes for the hard drive’s data heads to physically moved across the platter, from the Spindle area to the Outer tracks. Minimizing seek time can have a huge impact; most drives have seek times in the 8-12ms range, enough time to read about a megabyte of data! A small partition can drop that to 2ms or less (a 32GB partition on a 1TB drive is only 1/30 of the capacity, a very narrow band of the platter(s)).
Seek time can be made much worse if a program or programs access data on different partitions on the same drive at the same time, because the heads must skip over the entire partition. In such cases, either do not partition the drive, or use more than one hard drive to avoid “head contention”. However, even when both partitions are in use, access is often at non-conflicting intervals, and therefore the seek factor does not become an issue. For example, Photoshop might read a file from one partition, but while working on the file only the scratch partition is used. Mac OS X caching and on-board drive caches also mitigate the issue.
Outer vs inner PERMALINK
Data transfer speed testing with DiskTester shows the nearly 2X speed difference at the 0% (outer) area vs the 100% (inner) area of the drive. All four brands/models of hard drives graphed below show a steep decline in transfer speed from the outer to inner tracks, between 50 % and 60%. Remember that a minimum speed may be be critical for certain applications, such as high definition video (to avoid dropped frames).
When setting up a striped RAID with speed as the goal, you should get large enough hard disks to “waste” as much as half of the available storage so that you can maintain peak speed for your data. For example, if you have a terabyte of data, you’d want to have at least two terabytes of storage so that the fastest half of the drive is used (make a partition to ensure this). The number of drives factors into this of course. A good rule of thumb is to make the total volume size be at least 50% larger than the amount of data you expect to store (you can backup and redo it if necessary). You should also plan on keeping at least 20% free space at all times, or disk fragmentation could become an issue (related files and/or parts of files scattered around the disk).
Drive speed staircase PERMALINK
Hard drive performance across the available capacity is not a smooth curve: it is “stepped”, like a staircase that gets progressively more steep. This behavior is not true of solid state drives, like the Intel X25-M.
The staircase performance effect presumably corresponds to groups of tracks/cylinders with the same data density. Not only do the “steps” narrow in width, they also drop more in performance with each step.
This is why using the first half of the drive (or perhaps 2/3) offers much higher performance than filling a hard drive (of any capacity) to the 70%+ level. It is also why a larger capacity 5400 rpm drive can outperform a smaller capacity 7200rpm hard drive when the 7200rpm drive is more than about half full.
The graph below shows the time it takes to completely fill a Western Digital RE3 1TB hard drive with 1000 files of about ~1GB each (read and write times are identical). The test yields 1000 data points, clearly showing performance variation across the drive.
Solid state disk performance
Compare the hard drive graph above to the graph below from a solid state drive, the Intel X25-M (see review). With a well-implemented solid state drive, speed is consistent (on average) across the entire drive, except perhaps as the drive is filled to 99%+ capacity. There does appear to be more variance, but the trend is straight-line performance across the drive capacity as opposed to a staircase rolloff.
Here we see the amazing read performance of a single Intel X25-M. Write performance is also very good, but not as fast as a fast 3.5" hard drive. See review.