Atto R380 RAID PCI Express RAID Card
For a storage enclosure like the iStoragePro, a RAID card is needed to provide the RAID functionality, and the software to use it.
The Atto R380 RAID card is about $979. For best performance, the Atto card must be installed in an 8X slot, which means PCIe slot #2 on the 2010 Mac Pro (PCIe slots 3 and 4 are 4X slots and will impair performance slightly, as I found in my tests).
PCI Express 2.0, SATA
I would rather that the Atto R380 card be a PCI Express 2.0 card, perhaps then the speed would be sufficient in a 4X slot— I’m not sure, as I did not have its R680 sibling to test, which is PCIe 2.0 (though I hear that the R680 might have some remaining issues, so sticking to the R380 might be the smart move).
With the coming transition to SATA 6G, I’d prefer to invest in a PCI 2.0 card with 6G support, like the Atto R680, which offers both. Since the R680 costs only about 20% more than the R380, it’s the smart move for future speed (and it has double the bandwidth of the R380). Especially since new solid state drives will offer per-drive speeds approaching the 500MB/sec range in 2011.
The Atto R380 is bootable, and I tested this out several times. However, while booted off the card, I had multiple kernel panics while trying to clone from the booted volume to another. There is a special procedure for enabling a bootable partition, contact Atto for details.
Atto fan noise (updated Nov 30, 2010) Permalink
The Atto installer makes no mention of the noisy state in which you will find yourself once you install the driver, which installs the Atto fan controller daemon. You will simply find that your Mac Pro is intolerably loud.
You can listen for yourself to the difference. The first 15 seconds of the sound track are with the Atto fan controller daemon enabled. The remainder is the barely audible hum of the fans running normally, without the Atto fan controller daemon.
There is a workaround using Terminal; remove the Atto fan controller daemon:
sudo rm com.attotech.fan-controllerd.plist
Atto’s tech support response is as follows. The way I read this, one can either have an obnoxiously loud Mac Pro, or you can risk failure or malfunction of the Atto R380 RAID card (with unknown effects on your data). I will look for an alternative RAID card, but I am very unhappy about spending $1000 on a card with this catch-22.
The reason why it was not working before was because the fan controller suite in the driver package was not updated to support the 5,1 platform. After the 5,1 came out, we posted a separate installer on our website, and now that a new driver is out, the installer was included in the driver.
The PCI fan is raised to 3000 RPM with the plist in place, per the Intel spec and our testing to provide enough airflow to cool the RAID processing chip on the R380 accordingly.
The only reason we give this information out to users is so they can turn the fans down if the card is not in the machine. If the card is in the machine, I strongly recommend keeping the plist in place and using our default settings.
More, and I cannot confirm the statements about other RAID controllers as yet:
RAID chips run hot; that is just the nature of a hardware RAID controller. The RAID controller that Apple sells as an option when you're purchasing the Mac Pro will do the exact same thing -- it raises the PCI fan to 3000 RPM when the card is in the machine. Other vendors follow suit and do the same (Highpoint, LSI / MegaRAID / CalDigit).
The only vendor that I know of that doesn't do this is Areca -- and that's because they have a small fan directly screwed into the heatsink of the RAID chip on their cards. When the R380 was in the design phase, our hardware engineers considered this idea and rejected it due to research that showed even the highest quality small heatsink fan components available have extremely high failure rates, which would in turn force customers to RMA a card simply because the fan component failed.
ATTO and Intel have done extensive stress and heat testing and found that setting the Mac Pro's PCI fan to 3000 RPM provides adequate cooling and air flow to allow the RAID chip and the other components on the card to operate at the best possible performance without being damaged due to overheating.
And yes, the fans on the Mac Pro are rather loud compared to other vendors (thermaltake, antec, etc.). Do a google search for "Mac pro fan loud" -- you'll find thousands of posts on various.
I'm not sure who actually makes the fans that Apple uses. Maybe in the future they will consider switching to a vendor that uses newer technology and has lower decibel levels at higher RPMs.
I know on my workstation at home I can turn my case fans up to over 5000 RPM and I can barely detect a difference from 1000 -- but that's because they were designed to be quiet.
Again, we only recommend removing the fan controller daemon if you do not have the card in the machine.
The operating temperature and recommended CFM (air flow) specifications are published in the R380 tech sheet. Anything outside of the published specification is not considered an optimal environment.
One fact remains invariant for me— the fan noise is intolerably loud (the “default settings”). To be clear, I ran the iStoragePro with the Atto RAID card for over 24 hours continually and experienced no problems without the recommended fan daemon. But I can’t recommend daily use contrary to Atto’s and Intel’s stated requirements, and thus the Atto R380 is not a viable option for me, or anyone who is not deaf.
Initializing the RAID-5 or RAID-6 with the Atto R380
With the ATTO R380 RAID card, it’s a 20-hour process for the RAID-5 or RAID-6 to initialize itself (bad-block checking across the capacity of all drives). In the best case, I would expect this to take 9-10 hours, so it’s unclear why it should take twice that.
I did allow one complete RAID-5 initialization to occur; that’s how I know it took 20 hours! Do not skip this important step, unless it’s a testing setup.
There is one problem with the initialization step; it had completed 51% of the initialization when I rebooted, whereupon it reverted to 18% ! This was frustrating, given how long it takes to complete.
Shown below is the Atto Configuration Tool interface (Java based), with a freshly-created RAID-5 utilizing one hot spare (RAID-6 is a better choice for most users).
Various settings can be applied to a RAID. SpeedRead can be tuned for specific usage scenarios, but made no difference to my tests which were continuous I/O. Rebuild priority should be always be set to High to reduce the vulnerability to data loss from a second drive failure.
Simulating drive failure
I simulated failure by pulling one drive with tasks in progress; the unit just kept on plugging away and quickly notified me of the failure, and started rebuilding the parity information using the hot spare.
I also tested RAID-6, pulling two drives within seconds of each other, and things just kept on working.
The Atto Configuration Tool initially crashed instantly every time I launched it. Atto supplied a new version within a few days which resolved the issue, but it’s a little disturbing that two months after release of the Mac Pro that such a problem could be extant in a shipping product.
In general, Atto support answered all my questions and was helpful, though I’m not entirely happy with some of the design decisions with the software, which led to my confusion and various hassles and problems. But once sorted out, everything worked well. In particular, be sure to do the full install for proper RAID support, not just “Application only”, or your system log will fill up with nuisance messages.
Command line tricks with the Atto RAID card
If you just want to test a RAID-5 or RAID-6 , here’s how to skip the 20-hour drive check, do NOT do this for a setup onto which you will be storing data, use it while testing only:
rghaltinitialization raid-group-name set rmstate raid-group-name online
If you are forcing a failure mode by pulling a drive, the card will refuse to use that drive again ("Unavailable"). Here’s how to reset it (warning: this will NUKE your RAID if you specify the wrong drive, which I managed to do):
blockdevscan blockdevclean X
Where X is the ID number of the drive that you want to clean. You can match them up by the serial number and/or SAS address of the drive, but this isn’t obvious, so be careful. This is one really dumb thing in the ATTO Configuration Tool — you can’t just select an Unavailable drive and fix it, but must take on the risk of nuking the RAID by getting the number wrong — and don’t forget that the first drive is #0.