diglloyd Mac Performance Guide

Max Your Mac Pro at OWC

SSDStorageMemory


100% Kona, 100% Family Owned

Cell Phone Kill Switches

A “kill switch” will soon be required for all cell phones in California.

There are far more implications to a kill switch than most of the press and politicians understand. Or maybe they do understand (the scary part).

Imagine what a prize it would be for hackers to suddenly destroy 100 million phones comfortably from their couch. Or what a useful tool in time of war, or for the suppression of an entire Twitter-happy populace.

Thunderbolt Cabling Reliability: Check the Fit and Tension

Stephen B writes:

I’ve been using the ThunderBay 4 for about two months now. I have been having intermittent disconnects and I have been wondering if you’ve also experienced this as well. Typically this happens when the thunderbay is connected but not actively being used. The disconnects often cause my MBP to hang eventually and I can’t reconnect the drive without rebooting.

I haven’t been able to track down the cause. I’m running a 3-disk raid0 and 1 separate volume.

MPG: MPG had an issue one day and investigated. MPG ovviously cannot rule out other different issues with particular systems (bad drives, bad cables, bad device whatever), but what follows turned out to be the root cause for MPG.

First, the MPG experience NOT related to the Thunderbay. Tracking down the cause one day, it turned out to be loose (sloppy fit) Thunderbolt ports on the 2013 Mac Pro (and MacBook Pro) that had tension / torque on the cable.

Examining and testing both Apple and OWC brand Thunderbolt cables, MPG found that both brands fit very snugly into the OWC Thunderbay ports, but are slightly sloppy fit on the 2013 Mac Pro and MacBook Pro. So any torque / tension on the cable as it enters the Mac can cause a bad connection. Care should be taken to have all cables enter straight-on, so as to avoid any tension on the cable as it enters the port.

MPG doesn’t know if tjhe slightly loose fit on Macs is by design (e.g., a design specification) or poor quality control at Apple. But all five of the MPG OWC Thunderbay units have very snug-fitting Thunderbolt ports, and the Apple Mac Pro and Macbook Pro have slop.

Barry M writes:

Yup, I had this very problem with a Lacie 2Big drive, which kept ejecting itself. Drove me nuts, I assumed it was a faulty Lacie unit, and surely not the overpriced Apple Thunderbolt cable. I then was fastidious about making sure the cable had no tension and a straight path to the ports at either end. This did seem to help a little, but the drive would still eject without warning albeit not as frequently.

I was about to return the Lacie drive when I used a free bundled Belkin Thunderbolt cable and voila !! Everything worked perfectly !! Not one false eject since, I also used the Lacie TB cable with the same results, perfect reliable connection every time.

So check your cables, and have another brand on hand just to rule it out.

MPG: Swapping cables is often a best first step for any type of device.

Simplify Your Storage: Fast, Reliable, Expandable

OWC Thunderbay, front
OWC Thunderbay, front

For years, high performance or high capacity external storage was a painful struggle, first with Firewire 400 / 800 (yuck beyond words, fried ports, unreliable cabling), then eSATA (unreliable cables in multi-drive setups, driver kernel panics for a time), miniSAS (obnoxious PCIe card, very expensive and Mac Pro only), and now USB 3: Apple USB3 driver bugs persist to this day, it’s not a good choice for multiple external enclosure RAID though fine for single drives.

It’s simple: you want one unit that is fast, reliable expandable and quiet from a company that knows and tests for and supports Macs. The OWC Thunderbay is what I wanted all those years. Which is why I purchased five of them for all my storage needs.

MPG has observed that there is still confusion out there on the best choice for a Mac with Thunderbolt (reader feedback).

Whether you have a Mac Pro or iMac or MacBook Pro, a quality investment is the way to go: skip the multiple-USB3 hassle or dual drive stuff and just standardize on one or more Thunderbay units (this comment is not addressed at travel needs or single drive backups and so on, but main storage for serious users).

With one Thunderbay, most users can have a main Master drive (main storage, master copies), an Archive volume (overflow as Master fills up over time), a Time Machine backup volume and a spare drive/volume left over. Or RAID-5, and RAID-5 can be partitioned or not. And so on. Beautifully flexible.

For most users, the original $379 original (Thunderbolt v1) Thunderbay IV (“add your own drives” unit) is the deal—one reliable box to stuff drives into. Get the Thunderbolt 4 (Thunderbolt 2) for higher daisy-chained setups and/or get one preconfigured with or without RAID-5, and get extra Thunderbay drive sleds for swapping drives to expand the versatility even more.

OWC Labor Day Specials

Thunderbay

OWC Thunderbay, front
OWC Thunderbay, front

MPG’s favorite OWC product is the OWC Thunderbay. MPG has switched entirely to the Thunderbay for all external storage* (except for a few special-purpose high performance OWC SSDs).

The $379 original (Thunderbolt v1) Thunderbay IV (“add your own drives” unit) offers more than ample performance for hard drives and is a good deal for most users, since most do not need Thunderbolt 2 (Thunderbay 4). One reliable Thunderbolt cable to one quiet box beats the pants off USB3.

Have extra drives lying around you want to put to use? Get the Thunderbay IV and perhaps even some more Thunderbay drive sleds for swapping drives as needed (for backup, extra storage, whatever).

Want big storage? The 5TB Toshiba drives are what I use now.

* Three Thunderbay 4 units with 5TB Toshiba hard drives, and two Thunderbay IV units with various 2/3/4TB drives.

OWC deals

Free shipping within the contiguous 48 US States on most orders $49.00 & up including on all used Mac systems.

Prices listed include/reflect instant rebates which expire/end after 9/1/2014 - prices w/instant rebate valid through Labor Day Weekend.

See all Labor day specials at OWC

An additional $60 off select ThunderBay IV Thunderbolt 10Gb/s 0GB and Solutions up to 20TB. Now from $379.00

NAS-certified hard drives
3.5" 4TB HGST 7200RPM Desktop/NAS Certified HDD for $159.99 and Toshiba 5.0TB 7200RPM 3.5" HDD for $229.00

Used Apple MacBook Air
2011 i5 1.7GHz w/Thunderbolt, 4GB Memory, up to 480GB SSD
+ Includes AppleCare Warranty through December/2014!
Now from $549.00

Used Mac selection, from $139.00

Save on External Drive Enclosures from 1 Bay to 4 Bays - USB 3.0, FireWire, and Thunderbolt models. Now from $2.99!

Saving on Bus-Powered, Portable Solutions with USB 3.0, FireWire 800, and/or Thunderbolt up to 1.5TB from $69.00

2.5" Hybrid SSD/HD Notebook HDDs 750GB to 1.0TB from only $60.00

Griffin Loop Stand for X$29.99X just $7.79!

Premium, top rated, Made for i certfied NewerTech Lightning Cables 0.5M to 3.0M for $5.79 to $14.97! up to over 70% off!

diglloydTools Updated to version 2.2.7b

diglloydTools

diglloydTools has been updated to version 2.2.7b. Download page.

See the diglloydTools 2.2.7 release notes and previous 2.2.7a feature discussion.

This "b" release fixes one bug in which the duration of test-reliability was set to a default of one hour. Now unless explicitly specified, duration is the number of iterations (not time-based).

Purchase diglloydTools.

Some of the capabilities in diglloydTools

Aside from testing hard drive or SSD or RAID performance and reliability with DiskTester, data integrity with IntegrityChecker is a must-have workflow tool for anyone with important data:

diglloydTools graphing template
diglloydTools graphing template

Thunderstorms and Thunderbay

A field shooting trip (photography) kept me busy half of August, and will for a while yet, but I expect to see some very interesting products before long, both computer stuff and photographic. Wow, what nice Thunderstorms in early August in Yosemite.

When traveling and shooting, I use the OWC Mercury Envoy Pro EX for backing up my laptop with my photos, which tend to hit 100GB or more over the course of 10 days or so. A perfect little bus-powered and pocketable SSD for travel. Like the Sony RX100 III for quick handy photos.

And when home, I’m relishing the OWC Thunderbay. It’s just so straightforward and easy to add storage with Thunderbolt, and the OWC Thunderbay gives me high speed, high capacity, minimal noise, small footprint, tremendous flexibility. So many years I would have loved to have such a solution over eSATA and miniSAS and the associated hassles. All that messy business is now banished by using the Thunderbay units.

....

This year, all my fruit trees think it’s early October. Never before (in over 25 years) have I seen such a heavy and strangely early-maturing fig crop. Can’t keep up unfortunately, so much of it rots and drops.

Trout Dinner Sony RX100 Mark III
Thunderstorm over Pine Creek Drainage
Nikon D810 + Zeiss Otus 55mm f/1.4 APO-Distagon
Trout Dinner Sony RX100 Mark III
Trout Dinner
Sony RX100 Mark III

OWC Offers Thunderbay in RAID-5 Edition

See also SoftRAID 5 goes final.

All about RAID.

OWC Thunderbay 4, front
OWC Thunderbay 4, front

MPG now has converted all external storage (primaries and backups) to four Thunderbay units, purchased over the past few months.

A mix of single volumes, RAID-0 striping, RAID-1 mirrors, RAID-5 and RAID 1+0 are all used for various reasons.

See the MPG review of the OWC Thunderbay.

OWC’s ThunderBay 4 RAID 5 Edition Delivers Speeds up to 675 MB/s and is the First Ever Software RAID 5 on a Mac

Woodstock, IL – August 5, 2014 – OWC continues to innovate and build on the proven performance and flexibility of its second generation ThunderBay 4 by releasing the ThunderBay 4 RAID 5 Edition. The OWC ThunderBay 4 RAID 5 Edition gives users the freedom to use this cutting-edge technology as additional storage or backup, with the optimal balance of data-redundancy and performance via various RAID configurations.

Thinking Outside the Enclosure

Over a year ago, OWC and SoftRAID created a strong collaborative partnership to harness the full power and flexibility of ThunderBay 4 and software RAID 5 to bring enhanced performance, storage, and data-redundancy capabilities to the Mac.
“Going this route offers an extremely competitive price point with exceptional flexibility, advanced RAID / drive health monitoring features, and up to 35% higher performance compared to other RAID 5 hardware solutions that typically cost twice as much,” said Larry O’Connor, Founder and CEO, OWC.

Greater Speed + Capacity

The exceptional performance and massive capacity of ThunderBay 4, combined with a robust, proven software RAID engine, provides better performance capabilities that are up to 35% faster than many competing drives. With capacities ranging from 4TB to 20TB, and sustained data rates up to 675 MB/s for RAID 5 performance, users can optimize their workflow, and experience the first and only software RAID 5 technology developed for the Mac with the ThunderBay 4 RAID 5 Edition.
“OWC’s ThunderBay 4 brings to the market capacities that have yet to be seen in such a compact medium, as well as the performance and data security that only RAID 5 can provide!” – Les Tokar, The SSD Review

Personalized Performance + Redundancy – Bring Your Own Drives (BYOD)

Get the perfect BYOD solution with OWC’s ThunderBay 4 RAID 5 Edition Enclosure Kit to add up to four drives, SSDs, or a mixture of both hard drives and SSDs for customizable performance to meet specific storage and backup needs. In addition, users can also set up RAID 0, 1, 4, 5, or 1+0 arrays using SoftRAID that also includes drive monitoring, e-mail notification, and rapid rebuild capabilities.

Configure & Go Bigger

Use ThunderBay 4’s dual Thunderbolt 2 ports and daisy-chain up to five additional devices, plus one high-resolution display. Or take full advantage of the flexibility offered by ThunderBay 4 and SoftRAID to connect multiple ThunderBays to create even larger RAID arrays with up to 8, 12, and 16 drives for optimal performance and redundancy. Now, users can get near enterprise-level storage and performance on their desktop for a fraction of the cost.
“I remember when I spent $20,000 to buy 20GB of storage. Now, we can get far more storage, with far better performance, for about $100 per terabyte. The ThunderBay 4. I like what I see. Congratulations, OWC.” – Larry Jordan & Associates, Inc.

Key Features Include:

4 drive bays and dual Thunderbolt 2 ports
Speeds up to 675 MB/s for RAID 5 performance
Availability in large capacities up to 20TB
Flexibility to combine multiple ThunderBays into a single massive RAID array
High-Quality, Certified, Double-Shielded 1-Meter Thunderbolt cable
Utility software bundle for Mac
OWC 3-Year Limited Warranty with 48-Hour Turnaround Time

Pricing and Availability:

OWC’s ThunderBay 4 is available for immediate ordering in the following configurations:

OWC ThunderBay 4 RAID 5 Edition

Bring-Your-Own-Drives (BYOD) $649.00

4TB (1.0TB x four (4) 7200 RPM) $869.99

8TB (2.0TB x four (4) 7200 RPM) $999.97

12TB (3.0TB x four (4) 7200 RPM) $1,129.99

16TB (4.0TB x four (4) 7200 RPM) $1,499.99

20TB (5.0TB x four (4) 7200 RPM) $1,769.99

One-Stop Destination for Thunderbolt Upgrades and Thunderbolt Solutions

OWC’s Thunderbolt upgrades and Thunderbolt solutions are available at: http://eshop.macsales.com/shop/Thunderbolt/
The OWC Advantage – Over 20 Years of Storage Design and Innovation
– All solutions pre-tested to 100% capacity with a multi-hour full load burn-in (excludes BYOD)
– OWC tested for guaranteed compatibility, speed, flexibility, and reliability
– Comprehensive step-by-step installation and support videos
– 24/7 award-winning, US-based support for both ThunderBay 4 and SoftRAID is included with purchase

About OWC

OWC is the manufacturer and upgrade provider of choice for Apple enthusiasts with OWC Memory upgrades for Apple computers dating as far back as the mid 90’s. Over the last decade, OWC specifically built SSD upgrades for Apple computers winning multiple awards while also earning accolades for OWC External Drives and OWC Optical Drives. OWC also provides extensive US-based technical support for Mac users around the world. As a member of the Apple Developer Program, OWC also produces a catalog of upgrades and accessories having served the Apple community worldwide since 1988.

OWC’s Industry Awards Include:

– Inc. 5,000 “Fastest Growing Privately Owned Companies in America” list honoree 2007–2013
– Reseller Ratings Elite Member 2009-2014
– Better Business Bureau Top Rating of A+ 2010–2014
– About.com Readers’ Choice Award winner for best Mac Peripheral Reseller
– Business Intelligence Group 2013 Green Tech Company of the Year

 

OWC Thunderbay 4, front
OWC Thunderbay 4, front
5-drive RAID-5 volume in SoftRAID 5
5-drive RAID-5 volume in SoftRAID 5

SoftRAID 5 Goes Final: High Performance RAID 5, RAID 1+0, RAID-0, RAID-1

SoftRAID version 5 is now final.

SoftRAID 5
SoftRAID 5 volume icon for RAID 1+0

I’ve been working with a beta version for some months, delivering varied feedback to the SoftRAID team. My concerns were addressed, with a few user interface suggestions pending that I’m told will be improved.

RAID-5 (striping with one drive fault tolerance) and RAID 1+0 (striped pair of mirrors) are the new standout features in SoftRAID 5. Now final and with fixes for a few issues I found during the beta test cycle, I am very impressed

SoftRAID 5 delivers RAID-5 and RAID 1+0 performance in software that beats many hardware RAIDs and with far greater versatility across devices.

Of course, SoftRAID can format single volumes and do RAID-0 striping and RAID-1 mirrors also. I use RAID-5 and RAID-1+0 for some purposes, and RAID-0 and RAID-1 for others. For example, a neat trick is a RAID-1 mirror for 2 or more backup copies in one backup operation; these can be split apart (if desired) and updated later with “smart update‚—very cool feature.

Links and more information follow below these screen shots.

5-drive RAID-5 volume in SoftRAID 5
5-drive RAID-5 volume in SoftRAID 5

See also:

Performance:

4-drive RAID-1+0 setup with 4 volume in SoftRAID 5
4-drive RAID-1+0 setup with 4 volume in SoftRAID 5

FOR SALE: eSATA enclosures, Thunderbolt to eSATA

Local sale preferred. Contact.

External drive enclosures / adapters

Just Did a Big Drive Reconfig: How Did I Verify Data Integrity?

IntegrityChecker verify command
IntegrityChecker
(part of diglloydTools)

I just made a big drive swap, getting everything onto 5TB drives in four OWC Thunderbay 4 units (originals and multiple backups).

I wasn’t about to assume that everything was intact, so I used IntegrityChecker to validate a total of about 4.5TB of data. That’s a lot of data, and I like the piece of mind of knowing that not only are my originals unchanged/intact, but that every backup clone is bit-for-bit identical as well.

Here’s the tail end of the check on my Archive volume (command line is handy for invoking/stringing together as many volume verifications as desired, as shown).

ic verify Master.cloneR10; ic verify ArchiveP.cloneR10;
ic verify Archive.cloneR10 ... 100%: 132782 files @ 243.8MB/sec, processed 2.91TB Finished reading 132789 files of 132789 100%: 132789 files @ 243.8MB/sec, processed 2.91TB 100%: 132789 files @ 243.8MB/sec, processed 2.91TB Processed 2.91TB in 12529.0 seconds @ 243.8MB/sec ================================================================= ic verify Archive.cloneR10 2014-07-30 at 18:43:49 ================================================================= # Files with stored hash: 132789 # Files missing: 0 # Files hashed: 132789 # Files without hashes: 0 # Files whose size has changed: 0 # Files whose date changed: 0 # Files whose content changed (same size): 0 # Suspicious files: 0

GUI

I used the command line, but you can use the GUI wrapper if preferred:

IntegrityChecker reporting on verification results
IntegrityChecker reporting on verification results

Fast H.264 Video Compression on the MacBook Pro Beats the 2013 Mac Pro

Regarding an earlier post about astonishing MacBook Pro video encoding speed, Iain A writes:

About QuickSync, regarding H.264 compression. It’s entirely possible for a 13” MacBook Pro to outclass a Mac Pro in H.264 encoding, if done the right way.

- QuickTime Pro X will use QuickSync on supported Macs.
- FCP X will use QuickSync in “Faster Encoding” but not otherwise.
- Compressor will use QuickSync only in “Single Pass” H.264 encoding for .mov and .mp4 targets, so it’s necessary to make a preset which uses single pass and not multi-pass.

The QuickSync feature is only supported on consumer CPUs, and not on the Xeon in the Mac Pro. I performed a few tests with a colleague here, and the H.264 encodes are quite revealing:

http://www.macprovideo.com/hub/final-cut/final-cut-pro-x-performance-test

Faster doesn’t mean worse, either. While it’s possible that you could see a difference in quality with the faster encode on some sources, I simply haven’t seen it with the default (high) data rates that Apple uses, which are of course recompressed anyway by online video sharing services. QuickSync is a massive time saver, and it’s a real shame that the Mac Pro can’t use it.

MPG: Does this mean that all 2013 Mac Pros for video use should be purchased with a “MacBook Pro accelerator” add-on?

Transcoding 3840 X 2160 4K UltraHD video to 1080p using QuickTime
Transcoding 3840 X 2160 4K UltraHD video to 1080p using QuickTime

RAID 1+0 Fault Tolerance with SoftRAID 5

RAID 1+0 is an excellent fault tolerant configuration that also offers high performance. Usable capacity is 1/2, just as with a RAID-1 mirror (e.g., 4 X 4TB drives yields 8TB of usable capacity).

Anyone needing to greatly minimize the chance of downtime should be looking at RAID 1+0, since suitably configured with dual enclosures it can survive total loss of one of the enclosures.

Read about how and why I can configured four 3TB drives in a RAID 1+0 .

SoftRAID 5 has created three RAID 1+0 volumes from four 3TB hard drives
SoftRAID 5 has created three RAID 1+0 volumes from four 3TB hard drives

Four SSDs in OWC Thunderbay 4 test with RAID-0 and RAID-5

How does the OWC Thunderbay 4 (Thunderbolt 2) perform using four 240GB Mercury Extreme Pro 6G SSDs on the 2013 Mac Pro?

OWC Thunderbay 4 (Thunderbolt 2) performance with RAID-5 of four OWC SSDs
OWC Thunderbay 4 (Thunderbolt 2) performance with RAID-5 of four OWC SSDs

diglloydTools Updated to version 2.2.7a

diglloydTools

diglloydTools has been updated to version 2.2.7a. Download page.

See the diglloydTools 2.2.7 release notes. There are significant and useful changes and additions.

Note: IntegrityChecker had 2.2.7 had a bug in which it would not finish with files having resource forks. Fixed in version 2.2.7a.

Purchase diglloydTools.

Some of the capabilities in diglloydTools

Aside from testing hard drive or SSD or RAID performance and reliability with DiskTester, data integrity with IntegrityChecker is a must-have workflow tool for anyone with important data:

diglloydTools graphing template
diglloydTools graphing template

 

RAID-0 Scalability with Thunderbolt 2: Separate Bus or Daisy Chain?

Following up from the prior research using three OWC Thunderbay 4 units, I wondered what effect using three Thunderbolt 2 busses on the 2013 Mac Pro would have instead of daisy chaining.

The result show that somewhere around 6 drives, daisy chaining starts to break down, even with Thunderbolt 2 as here. Hence careful choice of which devices go on which bus is key for high performance systems (and these are hard drives, not SSDs). It might mean (for example) housing four drives in two enclosures so that each enclosure can have full bandwidth, e.g. for SSDs.

Also, in spite of its two ports, the late 2013 MacBook Pro has only one Thunderbolt bus, so it maxes out quickly. As of this writing only the 2013 Mac Pro offers more than one Thunderbolt 2 bus.

Click for larger graph.

RAID-0 striping scalability using OWC Thunderbay 4 with 5TB hard drives
RAID-0 striping scalability using OWC Thunderbay 4 with 5TB hard drives

RAID-0 Scalability: 60TB and 12 Hard Drives in Three OWC Thunderbay 4 Enclosures

Using three OWC Thunderbay 4 units, I explored the performance boundaries if I/O. Each Thunderbay unit was connected to one of the Thunderbolt 2 busses on the 2013 Mac Pro so that each would have full TB2 bandwidth.

The result show near linear scalability through 8 drives, with 9 drives and beyond dropping slightly in performance. This is a lot of bandwidth and could be useful when (for example) cloning from one RAID to another.

Note that more than 4 drives in a RAID-0 is increasingly risky because the volume is lost by failure of one drive—use RAID-5 for fault tolerance (OWC offers a Thunderbay 4 solution with RAID-5 preconfigured). But this test was about exploring the performance playground on the 2013 Mac Pro.

Click for larger graph.

RAID-0 striping scalability using OWC Thunderbay 4 with 5TB hard drives
RAID-0 striping scalability using OWC Thunderbay 4 with 5TB hard drives

60TB here for Testing (OWC Thunderbay 4)

Whoah!

Courtesy of OWC, I now have three OWC Thunderbay 4 units here (Thunderbolt 2) for whatever tests I can envision, each with four 5TB drives. That’s 3 X 4 X 5TB = 60TB. That’s more storage than you can shake a bit at. Back in my college days a 5 megabyte hard drive was a novelty (like hair on my head now).

OWC Thunderbay 4-bay enclosure  Thunderbolt 2 version
OWC Thunderbay 4
4-bay enclosure, Thunderbolt 2

July 4th deal on OWC Thunderbay IV

OWC is offering the Thunderbay IV for just $398 through July 4th. It is Thunderbolt v1, but it often doesn’t matter.

Tons of other OWC July 4th deals.

Reader Questions on SoftRAID

Fr.Damian writes:

These would apply to those who have not upgraded to the 2013 MacPro and are still using previous models.

1.) Can I configure all four of my drives as RAID5 internally with SoftRAID?

2.) Will/can I still get the same performance as you are getting with the Thunderbay & SoftRAID solution on a 2013 MacPro when using a RAID5 config inside a pre 2013 MacPro?

3.) Is it possible to connect a Thunderbay to a Pre 2013 MacPro?

4.) I am currently using v.4 30day trial and most likely will purchase it soon (of course mentioning you when purchasing).

You are testing using v.5. Why don't I see any mention of v.5 on their website?
(UPDATE: I was sent a link to v.5 from tech support to use as a beta AND
they did confirm that anyone purchasing now will get a free upgrade to v5)

I just wanted to point out how Absolutely Amazing SoftRAID's tech support has been over the past week. I just sent them a question about a small issue when running one of their tests in v4 and I can't believe how seriously they took it and to the degree they have responded. I've never seen anything like this! And I have many, many hours of tech support experience with many companies.

MPG: Taking in turn—

(1) and (2) SoftRAID works with just about anything for RAID-5, old or new. But MPG would not recommend Firewire or USB3 multi cables for RAID-5.

Internal drives in an older Mac Pro are ideal (as is Thunderbolt). Bandwidth has its limits on the older Mac Pro, to about ~650 MB/sec across all SATA busses. Plenty of speed, but less than Thunderbolt v1. But it will only matter (and only a litte) with very fast hard drives.

(3) Older Mac Pros cannot use Thunderbolt hence no Thunderbay. But the OWC Mercury Elite Pro QX2 can be used via eSATA.

(4) Questions about SoftRAID release cycle and so on need to be directed to there.

Apple to End 'Aperture' Support

Apple is discontinuing its Aperture software.

As of this writing, Apple’s Aperture page makes no mention of this key consideration. Surely prospective users deserve to be advised front and center on that page.

buy Apple Aperture

It’s a unreality idea that an iPhoto/iCloud mongrel can replace Aperture.

I’ve been wondering when this would happen, and I’ve long advised my consulting clients to hedge their bets by suitable file/folder and workflow processes. But the MacRumors piece suggests that Apple is working on a transition plan to Adobe Lightroom, with Adobe, so Apple deserves some credit if that is true.

The iCloud/iPhoto approach is a non-starter for professionals. Every time I touch iCloud I get burned; it cannot be taken seriously for anything serious. Yes it’s fine for basic stuff and has merits within circumscribed bounds—not saying otherwise. But it has many troubling issues.

Add the painfully long delay in the Mac Pro line (5 years), the Apple Core Rot and eye candy feature focus, the cancellation of hardware like XServe, and it has long been obvious that professionals have not been a target market for Apple for quite some time.

I’ve long advised my consulting clients to avoid Apple Aperture because several years ago Apple began to show a disdain for the needs of professional users: the release of Final Cut Pro X, which (incredibly) offered no compatibility with Final Cut projects (for quite some time, now it does). Professionals need to know that their investment (hardware, software, experience + workflow) will not just be discarded. Adobe to the rescue: the professional market is taken seriously.

Photographic professionals should be looking at Adobe Lightroom to replace Aperture. However, many photo workflows do not need Lightroom features; some users are better off with a simpler approach like Photoshop. It all gets down to file/folder and workflow processes.

William H writes:

Sad but inevitable. I’ve used Aperture forever but, knowing Apple, always suspected it would come to an end sometime.

I’ve tried to switch to CaptureOne and Lightroom many times. But while the alternatives certainly DO offer great output quality (probably better) their workflow is poor compared to Aperture. I average 4 hours a day with Aperture and this workflow is what matters. It is not perfect but it is good.

The lack of a major version update was always ominous, but Apple and their updates are never been something to look forward to - not their hardware nor their software. I still struggle with the ‘new’ Final Cut. If Apple HAD updated Aperture they would probably have ruined it anyway.

The question now is whether the operating system in 5 years time (or even 1 year) will still recognise Aperture. I’ve only today upgraded from 10.6.8 to this latest system because of concerns. Oh well, the price you pay for loyalty...

MPG: Apple updates to the OS give me a sinking feeling every time I hear a new one is coming, but very little of value is introduced and many new bugs and for the worse changes are inserted.

Used Macs and Mac Pros and Displays at OWC

Used Macs

Used 2009/2010/2013 Mac Pro including (as this was written) a 12-core 2013 model at $1100 off.

Cinema Displays

OWC Ships Thunderbay 4-Drive Solution with Thunderbolt 2

OWC Thunderbay 4 4-bay enclosure, Thunderbolt 2
OWC Thunderbay 4
4-bay enclosure, Thunderbolt 2

OWC is now shipping the Thunderbay 4 in a Thunderbolt 2 version. Supply is limited for the first few weeks, so hurry over and get one.

The units are also available in a RAID-5 edition. See the in-depth review of the OWC Thunderbay.

MPG will be testing the Thunderbolt 2 Thunderbay soon.

The Thunderbay IV with Thunderbolt v1 is discounted.

Do you need Thunderbolt 2 speed?

Two Mac models currently support Thunderbolt 2: the 2013 Mac Pro and the late 2013 MacBook Pro Retina.

Daisy-chaining = connecting devices in series to each other, versus direct connection for each to the host computer.

Thunderbolt 2 is twice the speed of Thunderbolt, but what are the practical implications? See 2013 Mac Pro: Thunderbolt Performance Tips as well as these summary points.

  • For hard drive solutions, Thunderbolt v1 an v2 offer no significant difference in speed, because the hard drives are not fast enough to need it. Even four hard drives capable of 200MB/sec (very fast) do not max-out Thunderbolt v1. And that's assuming all four are in use simultaneously.
  • With Macs like the 2013 Mac Pro, there are enough Thunderbolt ports that connecting a Thunderbolt v1 hard drive solution directly (without daisy-chaining) is fast enough as per previous point.
  • Daisy-chaining any two or more high peroformance solutions should use Thunderbolt 2 for best performance.
  • For SSDs, Thunderbolt v2 offers appreciably higher speed for simultaneous use of two or more SSDs, since some SSDs do have the speed to need the bandwidth.
  • Systems with only two Thunderbolt ports such as the MacBook Pro really only have one Thunderbolt port if an external display is used, hence daisy-chaining and high performance devices make Thunderbolt v2 a better choice.
  • Users of 4K displays require Thunderbolt 2 for 60 Hz operation if the display is daisy-chained to another peripheral (not connected directly to Mac).

See also:

OWC Thunderbay 4-bay enclosure  Thunderbolt 2 version
OWC Thunderbay 4
4-bay enclosure, Thunderbolt 2
OWC Thunderbay 4 as seem in system info
OWC Thunderbay 4 as seem in system info

Excellent USB3 Expansion: TRIPP LITE 7+1 Port USB3 Expansion Hub

For several months now, two units of the TRIPP Lite 7+1 Port USB3 hub have been in use on the MPG 2013 Mac Pro and MPG MacBook Pro.

Trouble-free operation with full USB3 speed on all 7 data ports (8th port is a extra high power charging-only port). Highly recommended.

Review: Expanding USB3 Ports (TRIPP LITE 7+1 USB3)

OWC has the TRIPP LITE USB3 hub discounted and in stock.

See also fast USB card readers.

TRIPP LITE 7-Port x USB 3.0, 1-Port x USB 3.0 Charging Hub for iPad 2
TRIPP LITE 7-Port x USB 3.0,
1-Port x USB 3.0 Charging Hub for iPad 2

Through June 30: Take 20% off Consulting with purchase of diglloydTools

DiskTester
DiskTester

Want to build out a Mac Pro system, or similar system? Or transition from an iMac?

Get the best performance, reliability, backup, fault tolerance for your needs without spending for things not needed (or that drop performance instead of increasing it). I usually save my clients money, often more than my fee.

Whether it’s Photoshop, Lightroom, CaptureONE Pro or similar, workflow and organizational structure, backup and fault tolerance, getting the right gear and procedures into place can yield years of efficient usage.

For the remainder of May, take 20% off my usual consulting rates with a new purchase of diglloydTools. (Must be scheduled in advance, hours flexible, international clients often use Skype or FaceTime to my phone for example).

Reader Question: “A way to make DiskTester iterate test for 50 times or more?”

DiskTester
DiskTester

Leslie C writes:

I was wondering if there is a way to make DiskTester iterate a test for 50 times or more? (Currently it can do 5 times)

For example, in sequential suite, each transfer size is 5 times, but I want it iterate 50 times on each size to have larger samplings .... Possible?

MPG: The GUI interfaces in diglloydTools are basic, for mainstream use, the default being 5 iterations for run-sequential-suite.

The command line is versatile and powerful in its capabilities. In Terminal, here is how to iterate 50 times on volume Scratch:

disktester run-sequential-suite --iterations 50 Scratch

It’s also possible to control the starting and ending transfer sizes for run-sequential-suite using additional parameters, as well as the total test size. For example, here the transfers are from 128K to 128MB over a 2GB test file size:

disktester run-sequential-suite --starting-xfer 128K --ending-xfer 128M --test-size 2G Scratch

The command line (Terminal) is admittedly daunting for most users, but extremely powerful over the range of commands, including scriptable usage. Yet one can approach it in steps, gaining confidence gradually. For example, this command invokes DiskTester with the command run-sequential-suite on volume Scratch, using default parameters:

disktester run-sequential-suite Scratch

Reader Feedback: “WORTH it. My drive (SSD) is back and so is my productivity”

DiskTester
DiskTester

Andres R writes:

I just wanted to let you know that I resisted paying $40 for a few days, but I knew my external SSD’s performance had gone down the drain.

Sadly, other than on your site, there was little about this anywhere. Most people seems to just be happy with their drives. I’m a photographer and use the drive for intermediary photoshop work. In any case, one hour after paying my money, it was WORTH it. My drive is back and so is my productivity.

Details: LaCie Rugged USB 3.0 Thunderbolt Series 120GB Solid State Drive used as an external drive via Thunderbolt. I was down to 19 MB/sec a second for writes and it is now back up to the 300-350+ range.

MPG: the DiskTester recondition command is designed expressly for SSDs. Not all SSDs benefit, but many types do.

At the least it clears out data to zeroes, perhaps easing the load on the SSD controller as it shuffles data around internally. The dgl wipeFree command is also effective for similar reasons.

DiskTester
DiskTester recondition

While the Apple SSDs in the MacBook Pro are generally quite good, a single recondition pass brought the minimum write speed up substantially (the first pass showed a big drop in performance near the end of the free space). This test on a late 2013 MacBook Pro Retina with a 512GB SSD about half full.

DiskTester recondition of late 2013 MacBook Pro Retina 512GB SSD
DiskTester recondition of late 2013 MacBook Pro Retina 512GB SSD

OWC Thunderbay Special

OWC Thunderbay 4-drive Thunderbolt enclosure
OWC Thunderbay 4-drive Thunderbolt enclosure

OWC is having a special on the Thunderbay IV with additional savings when drives are purchased:

We are now offering savings of $20 off two drives or another $40 off with 4 drives ($60 total) purchased with an OWC Thunderbay IV 0GB 4-Bay Thunderbolt Enclosure Kit which offers up to 895MB/s sustained data rates with 4 x 7200RPM HDDs (now $479).

Qualifying drives include select Toshiba, HGST, and OWC SSDs. Free delivery within the 48 contiguous US States.

See the MPG review of the Thunderbay IV.

Detecting Corruption / Validating Data Integrity Over Time and Across Drives and Backup/Restore

IntegrityChecker verify command
IntegrityChecker

An overlooked aspect of data management is data integrity: are the files intact tomorrow, a year from now, on the original drive and backup drive(s), or perhaps even on a DVD or BluRay. Or after having been transferred across a network.

Knowing that files/data is intact with no damage is a key part of any system restoration/update/backups/archiving. In some situations it could be mandatory (record keeping). The more valuable the data, the more important it is to consider the risks to loss, which include loss by file corruption as well file deletion (not to mention viruses and software bugs and user errors).

“Data” can mean image files (JPEG, TIF, PSD/PSB, etc) or video clips or projects, Lightroom catalogs, etc. Or it could mean spreadsheets, word processing files, accounting data, and so on. Knowing that these files are 100% intact leads to a comfort level in making system changes in storage approaches.

How can data be damaged? Disk errors, software bugs in applications or drivers or the system itself can happen. Moreover, the “damage” could be user-induced: saving over or replacing/deleting a file inadvertently. Simply having a “warning flag” could be useful in noting that “no expected changes” is violated.

For example, suppose that a new computer system is acquired and various drives need to be transferred over. Or that you have upgraded to a newer and larger hard drive. Or swapped SSDs. Or there is a need to restore from a backup. Or that you burned files to a DVD or BluRay—are they intact with no changes? Even RAID-5 with its parity data does not validate files when reading them, and a validate pass is over the entire volume with no selectivity for the desired file(s).

Enter IntegrityChecker, part of diglloydTools: at any time, files of any and all types can be checked against a previously computed “hash”, a cryptographic number unique to the file. If there is a mismatch, the file has been altered, somehow. This check can be made at any time: on the original, or on a 1000th-generation copy of that file. The only requirement is that the hash be computed once and remain in the same folder as the file for later reference.

How it works with IntegrityChecker.

IntegrityChecker computes a SHA1 cryptographic hash for each file in a folder, storing those hash numbers in a hidden “.ic” file within that folder. Thus, all files in the folder have a “hash value” against which its current state can be checked.

The process can be run on folder(s), or an entire volume.

  1. Run Update on the original files (computes and writes the hash values for every file in each folder into a hidden “.ic” file in that folder).
  2. Make the copy or backup or burn the DVD/BluRay or whatever (this naturally carries along the hidden “.ic” file in each folder).
  3. At any later time (tomorrow or a year later), run Verify on any backup or copy (this recomputes the hashes and compares to the values in the “.ic” file).

For example, some pro photographers burn DVD or BluRay discs containing folders on which IntegrityChecker has been run; these discs carry along the “.ic” file in each folder, and thus can be verified at any time. There are numerous such uses.

Usage

Both command line (Terminal) and GUI versions are provided. The GUI is basic, but the internals are what counts: one of the most efficient multi-threaded programs of any kind you’ll ever find. IntegrityChecker runs as fast as the drive and CPUs can go. Available commands include 'status', 'update', 'verify', 'update-all' and 'clean'.

See How to Safely Transfer Data or Verify Backups and also Example of Verifying Data Integrity.

Continues below.

IntegrityChecker reporting on verification results
IntegrityChecker reporting on verification results

Worth doing or happy go lucky?

For many computer users, the consequences are of little importance if a few things go bad: a song, a picture, a particular document; no big deal. But even such users would be upset losing years of photos—bugs in software (gray swan?) can have widespread impact; data integrity checking is a sanity check on assumptions.

But in a financial and obligatory professional duty sense, professionals need to consider the end-to-end processes they use. When data is one’s livelihood, attention to data integrity takes on new importance.

The greater the value of the data and the greater the time span over which the data has value, the more important it is to implement processes that minimize the chances of loss, because over years the storage format is likely to change with transitions and copying, etc. Also, knowing that a backup restored from a crash is valid takes some of the sting out of a crash.

Big Fault Tolerant Storage: OWC Now Offers a Supported and Preconfigured RAID-5 Solution via SoftRAID + Thunderbay

RAID-5 is going mainstream.

RAID-5 is a fault-tolerant form of RAID that is all but essential for anyone storing large data sets, or for whom downtime is unacceptable. Videographers, photographers, etc. Note however that RAID is not a backup.

OWC now offers a high performance preconfigured and fully supported plug-and-play RAID-5 solution utilizing SoftRAID 5.

View OWC Thunderbay RAID-5 solution and/or pre-tested cold spares

The solution consists of the OWC Thunderbay with four drives configured as a RAID-5. Optional and highly recommended extras are the pre-tested and pre-configured cold spare. Should a drive fail, OWC supports the solution via its usual technical support and 48 hour turnaround. The cold spare means that in the event of drive failure, the unit can get back to rebuilding its fault tolerance immediately (no ordering or shipping delays).

Usable capacity with Thunderbay configured as RAID-5 solution:

20TB ===> 15TB
16TB ===> 12TB
12TB ===> 9TB

See also:

Three 5TB RAID-5 volumes using four 5TB drives
Three 5TB RAID-5 volumes using four 5TB drives
OWC Thunderbay 4-drive Thunderbolt enclosure
OWC Thunderbay 4-drive Thunderbolt enclosure

SoftRAID 5: RAID-5 Can Be One Huge Volume, or Partitioned to Suit

SoftRAID 5 makes it easy to partition a fault tolerant setup using RAID-5 into one huge volume or several smaller ones.

RAID-5 Can be Partitioned to Suit

Three 5TB RAID-5 volumes using four 5TB drives
Three 5TB RAID-5 volumes using four 5TB drives

Includes performance implications.

Three 5TB RAID-5 volumes using four 5TB drives
Three 5TB RAID-5 volumes using four 5TB drives

SoftRAID 5: How it Works (Screen Shots)

Some new pages are published showing SoftRAID 5 in operation.

RAID 5 in degraded state
RAID 5 in degraded state

SoftRAID 5: Performance with RAID-0, RAID-5, RAID-1+0

Just published are multiple pages showing the performance of SoftRAID 5 with RAID-0, RAID-5, RAID-1+0 and single drives.

All tests with DiskTester.

Spare hard drive mounted in drive sled for use as cold spare
RAID 5 performance with 4 and 5 hard drives, SoftRAID 5 beta

Fault Tolerant RAID: Cold Spare Advised

When using fault tolerant RAID, the smart move is to have a cold spare on hand.

For example, a RAID-5 reverts to RAID-0 when a drive fails. You want that puppy rebuilt ASAP, to restore fault tolerance (failure of a 2nd drive would mean loss of the RAID).

In plain language, this means that when purchasing a unit like the 4-drive OWC Thunderbay to be used as a RAID-5 or RAID-1+0, buy a 5th spare drive and pre-test and pre-initialize it.

Spare hard drive mounted in drive sled for use as cold spare
Spare hard drive mounted in drive sled for use as cold spare

Testing Multiple Drives for Consistency for a RAID or Similar

diglloydTools

Among its many features, diglloydTools DiskTester offers a run-area-test command that can be used for relatively fast characterization of drive performance across its capacity. The results can be graphed to get a good idea of the drive performance.

For example, suppose a high performance RAID is envisioned: one laggard can cut the performance, since the slowest drive determines the peak speed.

As shown below, 8 samples were tested, and one of those eight is a significant laggard; this slower drive is best set aside as a cold spare or backup drive.

While DiskTester can test any number of drives simultaneously (via command line), testing should take care not to saturate the bandwidth; for example four fast hard drives can demand almost all of the Thunderbolt v1 bandwidth, and other devices on the bus could suck up bandwidth, disturbing the results.

Purchase diglloydTools

Click for larger graph.

diglloydTools DiskTester: graphing performance across volume capacity for 8-drive volume (RAID 1+0)  disktester run-area-test --iterations 5 --test-size 4G --delta-percent 5 r1+0
diglloydTools DiskTester: performance across volume capacity for eight drives
disktester run-area-test --iterations 5 --test-size 4G --delta-percent 5

diglloydTools Updated to version 2.2.7b1

diglloydTools

diglloydTools DiskTester has been updated with some new and modified functionality; see the release notes. It can be downloaded as a beta version (more changes/additions are planned).

Purchase diglloydTools

This beta release focuses on DiskTester, the most powerful OS X tool available for understanding drive performance (hard drives and SSDs, RAID, etc).

See the recent discussion of disktester graphing.

diglloydTools DiskTester: graphing performance across volume capacity for 8-drive volume (RAID 1+0)  disktester run-area-test --iterations 5 --test-size 4G --delta-percent 5 r1+0
diglloydTools DiskTester: graphing performance across volume capacity for 8-drive volume (RAID 1+0)
disktester run-area-test --iterations 5 --test-size 4G --delta-percent 5 r1+0

diglloydTools and data integrity verification

Aside from testing hard drive or SSD or RAID performance and reliability with DiskTester, data integrity with IntegrityChecker is a must-have workflow tool for anyone with important data:

Thunderbolt: Does the Cable Length Matter for Performance?

Fascinating results for both hard drives and SSDs.

OWC 30 Meter Optical Thunderbolt Cable.

UPDATE: MPG goofed on the SSD graph. The graph is now corrected. With SSDs, it shows only a minor difference in speed, diminishing with increased transfer sizes to insignificant levels.

Testing using DiskTester, with the revealing run-sequential suite command.

Transfer speed over range of I/O sizes, 1 meter vs 30 meter cable
Transfer speed over range of I/O sizes, 1 meter vs 30 meter cable
Transfer speed over range of I/O sizes, 1 meter vs 30 meter cable
Transfer speed over range of I/O sizes, 1 meter vs 30 meter cable

Max Your Mac Pro at OWC

diglloyd.com | Terms of Use | PRIVACY POLICY
Contact | About Lloyd Chambers | Consulting | Photo Tours
Mailing Lists | RSS Feeds | Twitter
Copyright © 2008-2014 diglloyd Inc, all rights reserved.