What a Bad Hard Drive Fails Like when Running diglloydTools 'fill-volume'
I received a 2009 Mac Pro with four internal 2TB hard drives from a friend whose computer systems I updated/replaced. That Mac Pro contained an SSD and 4 X 2TB hard drives, each with about 2.5 years of power-on time—realtively old and thus “suspicious”. He had been having strange delays and the drives might have been the culprit.
Accordingly, I decided to test all 4 of those fairly old hard drives for reliability. Generally an in-depth reliability test* is best done with the test-reliability command, but in this case I elected to do a fill-volume in order to be able to graph the results—and because fill-volume almost always kicks out a bad drive.
As shown below, -36 is an I/O error, which indicates a hardware failure of some kind.
What’s interesting about this particular test is that the failure did not occur during the write phase, but during the read phase, the implication being that a drive can be written, then later fail to be readable—a disaster for backups.
Recommendation: Users with hard drive backups might (always keep at least three) should probably make a practice of erasing a backup once a year (and one at a time), and then doing a fill-volume to see if the drive operates correctly through the entire write and read phases. If it passes, return it to backup duty.
Even brand-new hard drives can fail in this manner; most hard drives are not media-tested before being shipped. While bad blocks can be and are mapped-out, this shows up in the graph from fill-volume as erratic performance instead of a steady pattern.
* The test-reliability test is very thorough and can take several days if run fully.
Shown below, an excerpt of the test log shows that the drive failed with a -36 I/O error about halfway through the read phase.
Scanning "/Volumes/bad1/disktester-blobs"...1000 1000 files in 1 folders. ---------------------------------- Iteration 1 --------------------------------- Reading 1000 files totaling 1.81 TiB... IO method: single synchronous read buffer of size 128 MiB # Files Amt Read MiB/sec(all) MiB/sec(1) MiB/sec(10) MiB/sec(30) 0 768 MiB inf na na na 0 1.50 GiB inf na na na 1 2.23 GiB 156 130 130 130 ... 536 992.7 GiB 119 105 106 107 536 993.3 GiB 119 105 106 107 537 993.9 GiB 119 106 106 107 537 994.5 GiB 119 106 106 107 537 995.1 GiB 119 106 106 107 FileReaderTask::ReadAll: caught error: disk error (I/O error) [-36], file = "/Volumes/bad1/disktester-blobs/blob-0537.blob" CatalogInfo for "blob-0537.blob": Path: "/Volumes/bad1/disktester-blobs/blob-0537.blob" CreatorCode: BLOB FileType: BLOB DataFork: 1987051520, 1987051520, closed Locked: false TextEncoding: 0 CreationDate 2016-03-25 15:04:28 ContentModDate 2016-03-25 15:04:46 AttributeModDate 2016-03-25 15:04:46 AccessDate 2016-03-25 20:47:57 BackupDate 1903-12-31 16:00:00 Parent dir ID 113 Node ID: 652 FileReaderTask::ReadAll: exiting Task FileReaderTask exited with error: disk error (I/O error) [-36] Read 537 files totaling 995.4 GiB in 8584.08 seconds @ 119 MiB/sec
Some of the other capabilities in diglloydTools
Aside from testing hard drive or SSD or RAID performance and reliability with DiskTester, data integrity with IntegrityChecker is a must-have workflow tool for anyone with important data:
- Detecting Corruption / Validating Data Integrity Over Time and Across Drives and Backup/Restore
- What an Impaired SSD Looks Like
- How to Safely Transfer Data or Verify Backups
- Example of Verifying Data Integrity.
- Are Your Backup Drives Still Functional? Are Your Files Subtly Damaged?
- How to Automate DiskUtility Verification of Mounted Volumes.
- Selling that Computer? Wipe Out Personal Files First
- Search for diglloydTools articles.