All Posts by Date or last 15, 30, 90 or 180 days.
also by Lloyd: diglloyd.com photography and WindInMyFace.com
Thank you for purchasing through links and ads on this site.
OWC / MacSales.com...
diglloyd Deal Finder...
Buy other stuff at Amazon.com...
Capacities up to 48TB and speeds up to 1527MB/s

Thunderbolt on OS X: Spontaneous Drive Disconnect

Spontaneous Drive Disconnect
(Thunderbolt, OS X)

SEE ALSO: Thunderbolt Cabling Reliability: Check the Fit and Tension. MPG suspects that at least some disconnect issues relate to poor physical design of Apple’s Thunderbolt ports (loose/sloppy fit).

For a while now, I’ve had drives disconnecting spontaneously in my Thunderbolt enclosures. Not often, but frequently enough to be a problem, including a tendency to do so while making backups. It also seems to be provoked by having more than one OWC Thunderbay turned on at the same time. I am unsure if daisy-chaining is involved.

I’d been meaning to blog on this issue, but no one else had written with the issue, I was unsure if it was OS X or the hardware.

MPG now has reliable information confirming that the drive disconnect issue is a Thunderbolt bug in OS X — more Apple Core Rot. Does Apple have an Emoji for “my drives just went AWOL and I worry about data loss?”. It’s always a good idea to check file integrity with IntegrityChecker when drives are affected and then effected.

MPG can unequivocally state that disabling the Power Nap feature does not solve the issue* as per Eric S below. Just last night drives disconnected spontaneously while doing a clone backup, again this morning, as shown at right. Power Nap was off as it has been 'forever' and the system did not / was not sleeping but in active use (by me). It is possible that disabling Power Nap might reduce the frequency of spontaneous drive disconnects, but MPG has no sense of whether that is valid or not.

UPDATE: this morning, I am unable to complete backups because the drives are disconnecting every half hour. Now making a 3rd attempt!
UPDATE 2: even after reboot, a 3rd failure occurred (again during backups). Trying again (4th time), the backups finished. Another failure about half an hour later, for a total of five events in about 6 hours—once per hour roughly. UPDATE 3: I now have highly credible information that the issue affects all Macs with Thunderbolt ports. Update 4: with the two Thunderbay units in question on the iMac 5K and under heavy load, seeing no issues. I suspect Mac Pro hardware issue.

Update 8 April 12:40: I cannot reproduce the issue on an iMac 5K after two hours. Same two Thunderbay units, but different cable. Unplugged that cable from iMac 5K, plugged into Mac Pro, replacing prior cable. Doing so sent other units offline by jostling the other cables, a known Apple physical design flaw that I discussed back in 2014 (loose/sloppy ports on Apple Macs). Rebooted with the two Thunderbay units on the different cable, now running same procedures to see if problem recurs. After running clones all afternoon, NO MORE ISSUES. Could it have just been cabling and/or replugging Thunderbolt cables? Time will tell over some more days. But if that were the case, how does a system go bad over time? It doesn’t add up as a cabling issue.

* Disabling all features that might cause the machine to wake up still does not prevent the Mac Pro from waking up every hour, all night long, which is super annoying (spins up all Thunderbolt devices too).

Disable PowerNap in OS X Energy Saver preferences
(might or might help with Spontaneous Drive Disconnect)

My take on this so far (8 April 2016)

  • Clearly the result of OS X PCIe reconfiguration—looks strongly like an Apple bug (see system log below). But could also be a hardware bug with some particular machines.
  • Not brand specific to the storage device.
  • Can be machine specific, and can come and go. For example, de-cable and move the machine and it won’t happen any more.
  • The same Thunderbay on another machine may show no issues at all.
  • If units are daisy-chained, all units on the chain go down, but units on other busses do not.
  • Seems to be provoked by heavy I/O such as doing more than one clone at a time.
  • Usually not an issue when devices are idle. When cloning it would happen regularly; when idle it did not occur for hours at a time.

System log

What a mess. It’s clear that the system wants to reconfigure the PCI bus. My information is that OS X does this regularly. Clearly something goes awry. Here are eight drives going down in two daisy-chained Thunderbay units on a 2013 Mac Pro.

2016-04-06 14:08:07.000 kernel[0]: [ PCI configuration begin ]
2016-04-06 14:08:07.000 kernel[0]: [ PCI configuration end, bridges 48, devices 45 ]
2016-04-06 14:08:07.000 kernel[0]: disk15s2: no such device.
2016-04-06 14:08:07.000 kernel[0]: disk15s2: no such device.
2016-04-06 14:08:07.000 kernel[0]: disk17s2: no such device.
2016-04-06 14:08:07.000 kernel[0]: disk16s2: no such device.
2016-04-06 14:08:07.000 kernel[0]: disk16s2: no such device.
2016-04-06 14:08:07.000 kernel[0]: disk16s2: no such device.
2016-04-06 14:08:07.000 kernel[0]: disk16s2: no such device.
2016-04-06 14:08:07.000 kernel[0]: disk16s2: no such device.
2016-04-06 14:08:07.000 kernel[0]: disk16s2: media is not present.
2016-04-06 14:08:07.000 kernel[0]: disk16s2: media is not present.
2016-04-06 14:08:07.000 kernel[0]: disk16s2: media is not present. ...
2016-04-06 14:08:07.000 kernel[0]: disk16s2: media is not present.
2016-04-06 14:08:07.000 kernel[0]: *** kernel exceeded 500 log message per second limit - remaining messages this second discarded ***
2016-04-06 14:08:07.619 SoftRAID Driver[81]: A disk (disk12, SoftRAID ID: 068A737720301A40) for the SoftRAID volume "Backup2" (disk13) was removed or stopped responding while the volume was mounted and in use.
2016-04-06 14:08:07.620 com.apple.xpc.launchd[1]: (com.softraid.softraidd[81]) Service exited due to signal: Broken pipe: 13
2016-04-06 14:08:08.141 lsd[51805]: LaunchServices: Could not store lsd-identifiers file at /private/var/db/lsd/com.apple.lsdschemes.plist
2016-04-06 14:08:08.169 lsd[51805]: LaunchServices: Could not store lsd-identifiers file at /private/var/db/lsd/com.apple.lsdschemes.plist
2016-04-06 14:08:08.000 kernel[0]: hfs: unmount initiated on S1 on device disk16s2
2016-04-06 14:08:08.000 kernel[0]: hfs: unmount initiated on S3 on device disk14s2
2016-04-06 14:08:08.000 kernel[0]: jnl: disk14s2: close: journal is invalid. aborting outstanding transactions
2016-04-06 14:08:08.000 kernel[0]: jnl: disk16s2: close: journal is invalid. aborting outstanding transactions
2016-04-06 14:08:09.000 kernel[0]: hfs: unmount initiated on s4 on device disk15s2
2016-04-06 14:08:09.000 kernel[0]: hfs: unmount initiated on S2 on device disk17s2
2016-04-06 14:08:09.000 kernel[0]: jnl: disk15s2: close: journal is invalid. aborting outstanding transactions
2016-04-06 14:08:09.000 kernel[0]: jnl: disk17s2: close: journal is invalid. aborting outstanding transactions
2016-04-06 14:08:10.000 kernel[0]: [ PCI configuration begin ]
2016-04-06 14:08:10.000 kernel[0]: [ PCI configuration end, bridges 62, devices 45 ]
2016-04-06 14:08:11.000 kernel[0]: jnl: disk6s2: replay_journal: from: 50946048 to: 72523776 (joffset 0x7474000)
2016-04-06 14:08:11.000 kernel[0]: jnl: disk12s2: replay_journal: from: 99184640 to: 121249792 (joffset 0x7474000)
2016-04-06 14:08:11.000 kernel[0]: jnl: disk8s2: replay_journal: from: 451321856 to: 476307456 (joffset 0x7474000)
2016-04-06 14:08:11.000 kernel[0]: jnl: disk9s2: replay_journal: from: 9330688 to: 33148928 (joffset 0x7474000)
2016-04-06 14:08:11.000 kernel[0]: jnl: disk25: replay_journal: from: 445345792 to: 485974016 (joffset 0x48d0000)
2016-04-06 14:08:12.000 kernel[0]: jnl: disk24: replay_journal: from: 192376832 to: 234356736 (joffset 0x48d0000)
2016-04-06 14:08:18.324 SoftRAID Driver[61255]: A disk (disk8, SoftRAID ID: 068A73771FD3F840) for the SoftRAID volume "Backup2" (disk13) was removed or stopped responding while the volume was mounted and in use.
2016-04-06 14:08:18.330 SoftRAID Driver[61255]: A disk (disk6, SoftRAID ID: 068AB68DDD7DA4C0) for the SoftRAID volume "B1" (disk10) was removed or stopped responding while the volume was mounted and in use.
2016-04-06 14:08:18.336 SoftRAID Driver[61255]: A disk (disk9, SoftRAID ID: 068AB68DDD7DAC80) for the SoftRAID volume "B1" (disk10) was removed or stopped responding while the volume was mounted and in use.
2016-04-06 14:08:18.000 kernel[0]: jnl: disk12s2: journal replay done.
2016-04-06 14:08:18.341 SoftRAID Driver[61255]: The SoftRAID volume "B1" (disk10) encountered an error (E00002E4). A program attempted to read or write to a volume which was no longer accepting i/o requests.
2016-04-06 14:08:18.000 kernel[0]: jnl: disk6s2: journal replay done.
2016-04-06 14:08:18.347 SoftRAID Driver[61255]: The SoftRAID volume "Backup2" (disk13) encountered an error (E00002E4). A program attempted to read or write to a volume which was no longer accepting i/o requests.
2016-04-06 14:08:18.000 kernel[0]: jnl: disk25: journal replay done.
2016-04-06 14:08:18.000 kernel[0]: jnl: disk24: journal replay done.
2016-04-06 14:08:19.000 kernel[0]: jnl: disk9s2: journal replay done.
2016-04-06 14:08:19.000 kernel[0]: hfs: mounted B1 on device disk25
2016-04-06 14:08:19.869 fseventsd[49]: event logs in /Volumes/B1/.fseventsd out of sync with volume. destroying old logs. (56730 8 57767)
2016-04-06 14:08:19.889 fseventsd[49]: log dir: /Volumes/B1/.fseventsd getting new uuid: 5335FFCC-AE16-4557-B3B8-29B5CFFD9E8B
2016-04-06 14:08:19.928 fseventsd[49]: disk logger: failed to open output file /Volumes/B2/.fseventsd/fc0075e79778b356 (No such file or directory). mount point /Volumes/B2/.fseventsd
2016-04-06 14:08:19.929 fseventsd[49]: disk logger: failed to open output file /Volumes/B2/.fseventsd/fc0075e79778b356 (No such file or directory). mount point /Volumes/B2/.fseventsd
2016-04-06 14:08:19.929 fseventsd[49]: disk logger: failed to open output file /Volumes/S1/.fseventsd/fc0075e79778ada7 (No such file or directory). mount point /Volumes/S1/.fseventsd
2016-04-06 14:08:19.929 fseventsd[49]: disk logger: failed to open output file /Volumes/S1/.fseventsd/fc0075e79778ada7 (No such file or directory). mount point /Volumes/S1/.fseventsd
2016-04-06 14:08:19.929 fseventsd[49]: disk logger: failed to open output file /Volumes/S3/.fseventsd/fc0075e79778db55 (No such file or directory). mount point /Volumes/S3/.fseventsd
2016-04-06 14:08:19.930 fseventsd[49]: disk logger: failed to open output file /Volumes/S3/.fseventsd/fc0075e79778db55 (No such file or directory). mount point /Volumes/S3/.fseventsd
2016-04-06 14:08:19.930 fseventsd[49]: disk logger: failed to open output file /Volumes/S2/.fseventsd/fc0075e7977905e1 (No such file or directory). mount point /Volumes/S2/.fseventsd
2016-04-06 14:08:19.931 fseventsd[49]: disk logger: failed to open output file /Volumes/S2/.fseventsd/fc0075e7977905e1 (No such file or directory). mount point /Volumes/S2/.fseventsd
2016-04-06 14:08:19.931 fseventsd[49]: disk logger: failed to open output file /Volumes/s4/.fseventsd/fc0075e7977905de (No such file or directory). mount point /Volumes/s4/.fseventsd
2016-04-06 14:08:19.932 fseventsd[49]: disk logger: failed to open output file /Volumes/s4/.fseventsd/fc0075e7977905de (No such file or directory). mount point /Volumes/s4/.fseventsd
2016-04-06 14:08:20.000 kernel[0]: hfs: mounted s4 on device disk6s2
2016-04-06 14:08:20.000 kernel[0]: hfs: mounted S2 on device disk12s2
2016-04-06 14:08:20.386 fseventsd[49]: event logs in /Volumes/s4/.fseventsd out of sync with volume. destroying old logs. (1458 9 1479)
2016-04-06 14:08:20.000 kernel[0]: hfs: mounted B2 on device disk24
2016-04-06 14:08:20.492 fseventsd[49]: log dir: /Volumes/s4/.fseventsd getting new uuid: FB2AB8CA-1002-4179-AFA7-8A8FCFA6FF49
2016-04-06 14:08:20.665 fseventsd[49]: event logs in /Volumes/S2/.fseventsd out of sync with volume. destroying old logs. (1072 9 1093)
2016-04-06 14:08:20.711 fseventsd[49]: log dir: /Volumes/S2/.fseventsd getting new uuid: 1847267F-0227-45F1-862F-248623617B5B
2016-04-06 14:08:20.728 fseventsd[49]: event logs in /Volumes/B2/.fseventsd out of sync with volume. destroying old logs. (56984 9 57768)
2016-04-06 14:08:20.735 fseventsd[49]: log dir: /Volumes/B2/.fseventsd getting new uuid: D627C826-90DC-4277-9A5E-5F11636AE7AF
2016-04-06 14:08:21.000 kernel[0]: hfs: mounted S1 on device disk9s2
2016-04-06 14:08:21.248 fseventsd[49]: event logs in /Volumes/S1/.fseventsd out of sync with volume. destroying old logs. (1447 10 2493)
2016-04-06 14:08:21.000 kernel[0]: jnl: disk8s2: journal replay done.
2016-04-06 14:08:21.337 fseventsd[49]: log dir: /Volumes/S1/.fseventsd getting new uuid: 53AD83ED-CFDA-445A-B604-23A17BA7FF12
2016-04-06 14:08:23.000 kernel[0]: hfs: mounted S3 on device disk8s2
2016-04-06 14:08:23.322 fseventsd[49]: event logs in /Volumes/S3/.fseventsd out of sync with volume. destroying old logs. (2317 12 2493)
2016-04-06 14:08:23.389 fseventsd[49]: log dir: /Volumes/S3/.fseventsd getting new uuid: 4884B6FC-F84B-45CA-A721-2CEBF55F7B91

Reader comments

Eric S writes:

I have a tip I wanted to share with you as follows:

Quick background- I just upgraded my 2009 Mac Pro to the 2013 'Trash Can' Model. I had never dealt with Thunderbolt before, everything has been eSATA for me previously. Anyway, ran into a big problem right away while transferring data from my RAID volumes. Once mounted, they would abruptly hot disconnect, damage my data, physically damaged one of my drives, and crash the computer. Not good. These were stored previously inside the 2009 Mac Pro, connected to the internal bus and always functioned perfectly. For new Mac Pro, I had them connected externally, via thunderbolt OWC ThunderBay enclosures.

Meat of the story: so finally figured out a solution to this unnerving and debilitating problem and I would like to pass along this tip as I do not seem to find it on your site previously.

Under Apple System Preferences => Energy Saver, Apple looks like they did something rather confusing and not immediately apparent. For older Mac models, the panel had two sliders, one for Computer Sleep and one for Display Sleep. For newer models, they removed the Computer Sleep slider and added a “Power Nap” option. Apparently, Power Nap is now what controls the computer sleep and may also introduce other variations in the way internal and external hardware operates with OS X and the computer. This is not immediately apparent, at least I did not realize the connection until I stumbeld across this blog post from someone who was having the same drive hot disconnect problem: https://www.extensions.in.th/amitiae/2014/04_2014/disks_4.html

His solution was to keep the Power Nap box unchecked (it installs as checked by default and may periodically switch back on its own after further system installs) and he still never conclusively resolved whether this was the problem, but my findings are fairly confident this is the cause.

As long as external drives are connected via ThunderBolt and mounted, this Power Nap setting must always be unchecked. After any Apple system software updates, you must immediately go back and look to make sure it is still unchecked, as I have had it go back to default on it’s own a couple times.

Anyway, that is my problem and determined solution which I hope you may find useful.

If you have already come across this, I’m sorry and please disregard.

Please keep up the great work, your information is invaluable and a must have reference for so may topics not covered anywhere else.

MPG: PowerNap is definitely not involved; that’s a red herring.

Pedro F writes:

I have a Mac Pro Late 2013 and I’ve been having thunderbolt drives disconnecting occasionally, It’s been driving me nuts for months. I have been searching for a solution and was frustrated, I didn’t find any conclusive evidence it was a software problem or a drive problem and the thunderbolt cabling seemed faultless, thunderbolt cables entered normally and seemed secure in the New Mac Pro. But then while searching around the internet I read in your site that newer Macs have loose thunderbolt ports. So I put tape around the Thunderbolt cables I use and boom! Never had any single disconnect since then. The bottom line is, is it possible that all this problems you are having with your drives are not a software problem but one with the thunderbolt ports?

MPG: It’s possible, especially since I hear that replacing the Mac sometimes makes the issue disappear. I’ve replugged my cables and there is not much I can do—the fit is solid and tight on the OWC Thunderbay (excellent ports), but on the Mac Pro the six Thunderbolt ports are so darn tightly grouped that all one can do is to make sure the cables go straight in; no way to put tape around one cable with cables running into all six.

Hans M writes:

Spontaneous disconnect has occurred with me also, using an OWC Mercury Elite Pro Dual on Thunderbolt with my MacPro 2013. This drive was used as a TimeMachine backup. After a few weeks TimeMachine announced it couldn't find the disk. Checking the disks with SoftRAID indicated they were ok.

You raised the question if it could be provoked by having more than one OWC Thunderbay turned on or daisy-chained. To the contrary I can inform you that the same occurred with one of my children I loaned a drive to. The OWC Mercury Elite Pro Dual was connected by Thunderbolt to the most recent iMac as a TimeMachine backup. It also disconnected after a few weeks with TimeMachine suddenly being unable to find the drive.

MPG: My information is that it can happen on any Mac or any brand Thunderbolt device (e.g. brands like Pegasus). This report is consistent with that idea on the “any Mac” front. However, it also seems to be a good working theory that the Mac itself is the issue—a hardware problem, and that replacing the Mac may make the problem go away. Hence those with unaffected Macs scoff at those who are affected.

Eric S writes:

Ahh, mine also too 'has a mind of its own'. It does as you mention intermittently sleep and wake up on its own. I witnessed this tonight.

I thought I had fixed it by in checking Power Nap, but realize now this does not absolutely solve it. System sleeps and wake ups still are happening throughout the night with drive hot disconnect errors popping up upon return from wake and there seems nothing I can do about this.

I have gone ahead and spent the last week transferring all data to RAID 1+0 configurations via SoftRAID and had to buy additional Thunderbay enclosures and drives. Still scarred though as my Data rests in a unstable and non-consistent state as long as drives are on and computer is plugged in. Lol. What a piece if shit. Was not counting on this upon spending the thousands of dollars I just plucked down for the 'upgrade'. To be honest, surprised but also not surprised.

During the course of last several major OS X upgrades at various times, they have upon install completely destroyed all my years worth of saved Apple mail, destroyed the contents of my iTunes media library, (now way in hell I am trusting them to store all my photos in the cloud as I suspect this will also break at some point wiping away all my years worth of precious photos, something I can not conceivable let happen,) and so now looks like they are after destroying all my Data altogether. Have a hard time thinking that any of this would have been possible had Steve Jobs still been around. Just a final post script to this story. Again, thanks for posting about it. Can't imagine how anyone else is seriously using this with any degree of confidence by this point.

Oh, I just got it...It's nicknamed the Trash Can, not because of its shape, but it's because it's where all your data goes to die!! Funny /s

MPG: Apple’s credibility has never been lower on the software front, but hardware woes are nerve wracking.

Daniel K writes:

I have been struggling with this issue for 2 years. It randomly started happening with my audio interface which I use daily. It uses a firewire to thunderbolt adapter. However, after replacing the adapter and cable with a brand new one the problem persisted.

Guess what, you won't believe it, but I managed to fix it after searching everywhere online. All I did was do PRAM reset and it has been running smoothly since. Search how to PRAM reset your mac it's quite simple.

MPG: a Firewire to Thunderbolt adapter is impossible, so Daniel means Thunderbolt to Firewire.

PRAM resets do fix a problem here and there from time to time, but not in my case. PRAM reset is an oft-quoted, rarely effective but worthwhile first thing to try.

Durable and fast, up to 1800MB/s

diglloyd.com | Terms of Use | PRIVACY POLICY
Contact | About Lloyd Chambers | Consulting | Photo Tours
Mailing Lists | RSS Feeds | Twitter
Copyright © 2008-2017 diglloyd Inc, all rights reserved.
Display info: __RETINA_INFO_STATUS__