diglloyd Mac Performance Guide

Max Your Mac Pro at OWC

SSDStorageMemory


100% Kona, 100% Family Owned

FOR SALE: eSATA enclosures, Thunderbolt to eSATA

Local sale preferred. Contact.

eSATA gear with Thunderbolt adapter

Moving all to Thunderbolt, so I am selling my eSATA gear. High quality stuff:

At new prices, that's about $2200. Asking $500 or best offer.

External drive enclosures / adapters

Just Did a Big Drive Reconfig: How Did I Verify Data Integrity?

IntegrityChecker verify command
IntegrityChecker
(part of diglloydTools)

I just made a big drive swap, getting everything onto 5TB drives in four OWC Thunderbay 4 units (originals and multiple backups).

I wasn’t about to assume that everything was intact, so I used IntegrityChecker to validate a total of about 4.5TB of data. That’s a lot of data, and I like the piece of mind of knowing that not only are my originals unchanged/intact, but that every backup clone is bit-for-bit identical as well.

Here’s the tail end of the check on my Archive volume (command line is handy for invoking/stringing together as many volume verifications as desired, as shown).

ic verify Master.cloneR10; ic verify ArchiveP.cloneR10;
ic verify Archive.cloneR10 ... 100%: 132782 files @ 243.8MB/sec, processed 2.91TB Finished reading 132789 files of 132789 100%: 132789 files @ 243.8MB/sec, processed 2.91TB 100%: 132789 files @ 243.8MB/sec, processed 2.91TB Processed 2.91TB in 12529.0 seconds @ 243.8MB/sec ================================================================= ic verify Archive.cloneR10 2014-07-30 at 18:43:49 ================================================================= # Files with stored hash: 132789 # Files missing: 0 # Files hashed: 132789 # Files without hashes: 0 # Files whose size has changed: 0 # Files whose date changed: 0 # Files whose content changed (same size): 0 # Suspicious files: 0

GUI

I used the command line, but you can use the GUI wrapper if preferred:

IntegrityChecker reporting on verification results
IntegrityChecker reporting on verification results

Fast H.264 Video Compression on the MacBook Pro Beats the 2013 Mac Pro

Regarding an earlier post about astonishing MacBook Pro video encoding speed, Iain A writes:

About QuickSync, regarding H.264 compression. It’s entirely possible for a 13” MacBook Pro to outclass a Mac Pro in H.264 encoding, if done the right way.

- QuickTime Pro X will use QuickSync on supported Macs.
- FCP X will use QuickSync in “Faster Encoding” but not otherwise.
- Compressor will use QuickSync only in “Single Pass” H.264 encoding for .mov and .mp4 targets, so it’s necessary to make a preset which uses single pass and not multi-pass.

The QuickSync feature is only supported on consumer CPUs, and not on the Xeon in the Mac Pro. I performed a few tests with a colleague here, and the H.264 encodes are quite revealing:

http://www.macprovideo.com/hub/final-cut/final-cut-pro-x-performance-test

Faster doesn’t mean worse, either. While it’s possible that you could see a difference in quality with the faster encode on some sources, I simply haven’t seen it with the default (high) data rates that Apple uses, which are of course recompressed anyway by online video sharing services. QuickSync is a massive time saver, and it’s a real shame that the Mac Pro can’t use it.

MPG: Does this mean that all 2013 Mac Pros for video use should be purchased with a “MacBook Pro accelerator” add-on?

Transcoding 3840 X 2160 4K UltraHD video to 1080p using QuickTime
Transcoding 3840 X 2160 4K UltraHD video to 1080p using QuickTime

RAID 1+0 Fault Tolerance with SoftRAID 5

RAID 1+0 is an excellent fault tolerant configuration that also offers high performance. Usable capacity is 1/2, just as with a RAID-1 mirror (e.g., 4 X 4TB drives yields 8TB of usable capacity).

Anyone needing to greatly minimize the chance of downtime should be looking at RAID 1+0, since suitably configured with dual enclosures it can survive total loss of one of the enclosures.

Read about how and why I can configured four 3TB drives in a RAID 1+0 .

SoftRAID 5 has created three RAID 1+0 volumes from four 3TB hard drives
SoftRAID 5 has created three RAID 1+0 volumes from four 3TB hard drives

Four SSDs in OWC Thunderbay 4 test with RAID-0 and RAID-5

How does the OWC Thunderbay 4 (Thunderbolt 2) perform using four 240GB Mercury Extreme Pro 6G SSDs on the 2013 Mac Pro?

OWC Thunderbay 4 (Thunderbolt 2) performance with RAID-5 of four OWC SSDs
OWC Thunderbay 4 (Thunderbolt 2) performance with RAID-5 of four OWC SSDs

diglloydTools Updated to version 2.2.7a

diglloydTools

diglloydTools has been updated to version 2.2.7a. Download page.

See the diglloydTools 2.2.7 release notes. There are significant and useful changes and additions.

Note: IntegrityChecker had 2.2.7 had a bug in which it would not finish with files having resource forks. Fixed in version 2.2.7a.

Purchase diglloydTools.

Some of the capabilities in diglloydTools

Aside from testing hard drive or SSD or RAID performance and reliability with DiskTester, data integrity with IntegrityChecker is a must-have workflow tool for anyone with important data:

diglloydTools graphing template
diglloydTools graphing template

 

RAID-0 Scalability with Thunderbolt 2: Separate Bus or Daisy Chain?

Following up from the prior research using three OWC Thunderbay 4 units, I wondered what effect using three Thunderbolt 2 busses on the 2013 Mac Pro would have instead of daisy chaining.

The result show that somewhere around 6 drives, daisy chaining starts to break down, even with Thunderbolt 2 as here. Hence careful choice of which devices go on which bus is key for high performance systems (and these are hard drives, not SSDs). It might mean (for example) housing four drives in two enclosures so that each enclosure can have full bandwidth, e.g. for SSDs.

Also, in spite of its two ports, the late 2013 MacBook Pro has only one Thunderbolt bus, so it maxes out quickly. As of this writing only the 2013 Mac Pro offers more than one Thunderbolt 2 bus.

Click for larger graph.

RAID-0 striping scalability using OWC Thunderbay 4 with 5TB hard drives
RAID-0 striping scalability using OWC Thunderbay 4 with 5TB hard drives

RAID-0 Scalability: 60TB and 12 Hard Drives in Three OWC Thunderbay 4 Enclosures

Using three OWC Thunderbay 4 units, I explored the performance boundaries if I/O. Each Thunderbay unit was connected to one of the Thunderbolt 2 busses on the 2013 Mac Pro so that each would have full TB2 bandwidth.

The result show near linear scalability through 8 drives, with 9 drives and beyond dropping slightly in performance. This is a lot of bandwidth and could be useful when (for example) cloning from one RAID to another.

Note that more than 4 drives in a RAID-0 is increasingly risky because the volume is lost by failure of one drive—use RAID-5 for fault tolerance (OWC offers a Thunderbay 4 solution with RAID-5 preconfigured). But this test was about exploring the performance playground on the 2013 Mac Pro.

Click for larger graph.

RAID-0 striping scalability using OWC Thunderbay 4 with 5TB hard drives
RAID-0 striping scalability using OWC Thunderbay 4 with 5TB hard drives

60TB here for Testing (OWC Thunderbay 4)

Whoah!

Courtesy of OWC, I now have three OWC Thunderbay 4 units here (Thunderbolt 2) for whatever tests I can envision, each with four 5TB drives. That’s 3 X 4 X 5TB = 60TB. That’s more storage than you can shake a bit at. Back in my college days a 5 megabyte hard drive was a novelty (like hair on my head now).

OWC Thunderbay 4-bay enclosure  Thunderbolt 2 version
OWC Thunderbay 4
4-bay enclosure, Thunderbolt 2

July 4th deal on OWC Thunderbay IV

OWC is offering the Thunderbay IV for just $398 through July 4th. It is Thunderbolt v1, but it often doesn’t matter.

Tons of other OWC July 4th deals.

Reader Questions on SoftRAID

Fr.Damian writes:

These would apply to those who have not upgraded to the 2013 MacPro and are still using previous models.

1.) Can I configure all four of my drives as RAID5 internally with SoftRAID?

2.) Will/can I still get the same performance as you are getting with the Thunderbay & SoftRAID solution on a 2013 MacPro when using a RAID5 config inside a pre 2013 MacPro?

3.) Is it possible to connect a Thunderbay to a Pre 2013 MacPro?

4.) I am currently using v.4 30day trial and most likely will purchase it soon (of course mentioning you when purchasing).

You are testing using v.5. Why don't I see any mention of v.5 on their website?
(UPDATE: I was sent a link to v.5 from tech support to use as a beta AND
they did confirm that anyone purchasing now will get a free upgrade to v5)

I just wanted to point out how Absolutely Amazing SoftRAID's tech support has been over the past week. I just sent them a question about a small issue when running one of their tests in v4 and I can't believe how seriously they took it and to the degree they have responded. I've never seen anything like this! And I have many, many hours of tech support experience with many companies.

MPG: Taking in turn—

(1) and (2) SoftRAID works with just about anything for RAID-5, old or new. But MPG would not recommend Firewire or USB3 multi cables for RAID-5.

Internal drives in an older Mac Pro are ideal (as is Thunderbolt). Bandwidth has its limits on the older Mac Pro, to about ~650 MB/sec across all SATA busses. Plenty of speed, but less than Thunderbolt v1. But it will only matter (and only a litte) with very fast hard drives.

(3) Older Mac Pros cannot use Thunderbolt hence no Thunderbay. But the OWC Mercury Elite Pro QX2 can be used via eSATA.

(4) Questions about SoftRAID release cycle and so on need to be directed to there.

Apple to End 'Aperture' Support

Apple is discontinuing its Aperture software.

As of this writing, Apple’s Aperture page makes no mention of this key consideration. Surely prospective users deserve to be advised front and center on that page.

buy Apple Aperture

It’s a unreality idea that an iPhoto/iCloud mongrel can replace Aperture.

I’ve been wondering when this would happen, and I’ve long advised my consulting clients to hedge their bets by suitable file/folder and workflow processes. But the MacRumors piece suggests that Apple is working on a transition plan to Adobe Lightroom, with Adobe, so Apple deserves some credit if that is true.

The iCloud/iPhoto approach is a non-starter for professionals. Every time I touch iCloud I get burned; it cannot be taken seriously for anything serious. Yes it’s fine for basic stuff and has merits within circumscribed bounds—not saying otherwise. But it has many troubling issues.

Add the painfully long delay in the Mac Pro line (5 years), the Apple Core Rot and eye candy feature focus, the cancellation of hardware like XServe, and it has long been obvious that professionals have not been a target market for Apple for quite some time.

I’ve long advised my consulting clients to avoid Apple Aperture because several years ago Apple began to show a disdain for the needs of professional users: the release of Final Cut Pro X, which (incredibly) offered no compatibility with Final Cut projects (for quite some time, now it does). Professionals need to know that their investment (hardware, software, experience + workflow) will not just be discarded. Adobe to the rescue: the professional market is taken seriously.

Photographic professionals should be looking at Adobe Lightroom to replace Aperture. However, many photo workflows do not need Lightroom features; some users are better off with a simpler approach like Photoshop. It all gets down to file/folder and workflow processes.

William H writes:

Sad but inevitable. I’ve used Aperture forever but, knowing Apple, always suspected it would come to an end sometime.

I’ve tried to switch to CaptureOne and Lightroom many times. But while the alternatives certainly DO offer great output quality (probably better) their workflow is poor compared to Aperture. I average 4 hours a day with Aperture and this workflow is what matters. It is not perfect but it is good.

The lack of a major version update was always ominous, but Apple and their updates are never been something to look forward to - not their hardware nor their software. I still struggle with the ‘new’ Final Cut. If Apple HAD updated Aperture they would probably have ruined it anyway.

The question now is whether the operating system in 5 years time (or even 1 year) will still recognise Aperture. I’ve only today upgraded from 10.6.8 to this latest system because of concerns. Oh well, the price you pay for loyalty...

MPG: Apple updates to the OS give me a sinking feeling every time I hear a new one is coming, but very little of value is introduced and many new bugs and for the worse changes are inserted.

Used Macs and Mac Pros and Displays at OWC

Used Macs

Used 2009/2010/2013 Mac Pro including (as this was written) a 12-core 2013 model at $1100 off.

Cinema Displays

OWC Ships Thunderbay 4-Drive Solution with Thunderbolt 2

OWC Thunderbay 4 4-bay enclosure, Thunderbolt 2
OWC Thunderbay 4
4-bay enclosure, Thunderbolt 2

OWC is now shipping the Thunderbay 4 in a Thunderbolt 2 version. Supply is limited for the first few weeks, so hurry over and get one.

The units are also available in a RAID-5 edition. See the in-depth review of the OWC Thunderbay.

MPG will be testing the Thunderbolt 2 Thunderbay soon.

The Thunderbay IV with Thunderbolt v1 is discounted.

Do you need Thunderbolt 2 speed?

Two Mac models currently support Thunderbolt 2: the 2013 Mac Pro and the late 2013 MacBook Pro Retina.

Daisy-chaining = connecting devices in series to each other, versus direct connection for each to the host computer.

Thunderbolt 2 is twice the speed of Thunderbolt, but what are the practical implications? See 2013 Mac Pro: Thunderbolt Performance Tips as well as these summary points.

  • For hard drive solutions, Thunderbolt v1 an v2 offer no significant difference in speed, because the hard drives are not fast enough to need it. Even four hard drives capable of 200MB/sec (very fast) do not max-out Thunderbolt v1. And that's assuming all four are in use simultaneously.
  • With Macs like the 2013 Mac Pro, there are enough Thunderbolt ports that connecting a Thunderbolt v1 hard drive solution directly (without daisy-chaining) is fast enough as per previous point.
  • Daisy-chaining any two or more high peroformance solutions should use Thunderbolt 2 for best performance.
  • For SSDs, Thunderbolt v2 offers appreciably higher speed for simultaneous use of two or more SSDs, since some SSDs do have the speed to need the bandwidth.
  • Systems with only two Thunderbolt ports such as the MacBook Pro really only have one Thunderbolt port if an external display is used, hence daisy-chaining and high performance devices make Thunderbolt v2 a better choice.
  • Users of 4K displays require Thunderbolt 2 for 60 Hz operation if the display is daisy-chained to another peripheral (not connected directly to Mac).

See also:

OWC Thunderbay 4-bay enclosure  Thunderbolt 2 version
OWC Thunderbay 4
4-bay enclosure, Thunderbolt 2
OWC Thunderbay 4 as seem in system info
OWC Thunderbay 4 as seem in system info

Excellent USB3 Expansion: TRIPP LITE 7+1 Port USB3 Expansion Hub

For several months now, two units of the TRIPP Lite 7+1 Port USB3 hub have been in use on the MPG 2013 Mac Pro and MPG MacBook Pro.

Trouble-free operation with full USB3 speed on all 7 data ports (8th port is a extra high power charging-only port). Highly recommended.

Review: Expanding USB3 Ports (TRIPP LITE 7+1 USB3)

OWC has the TRIPP LITE USB3 hub discounted and in stock.

See also fast USB card readers.

TRIPP LITE 7-Port x USB 3.0, 1-Port x USB 3.0 Charging Hub for iPad 2
TRIPP LITE 7-Port x USB 3.0,
1-Port x USB 3.0 Charging Hub for iPad 2

Through June 30: Take 20% off Consulting with purchase of diglloydTools

DiskTester
DiskTester

Want to build out a Mac Pro system, or similar system? Or transition from an iMac?

Get the best performance, reliability, backup, fault tolerance for your needs without spending for things not needed (or that drop performance instead of increasing it). I usually save my clients money, often more than my fee.

Whether it’s Photoshop, Lightroom, CaptureONE Pro or similar, workflow and organizational structure, backup and fault tolerance, getting the right gear and procedures into place can yield years of efficient usage.

For the remainder of May, take 20% off my usual consulting rates with a new purchase of diglloydTools. (Must be scheduled in advance, hours flexible, international clients often use Skype or FaceTime to my phone for example).

Reader Question: “A way to make DiskTester iterate test for 50 times or more?”

DiskTester
DiskTester

Leslie C writes:

I was wondering if there is a way to make DiskTester iterate a test for 50 times or more? (Currently it can do 5 times)

For example, in sequential suite, each transfer size is 5 times, but I want it iterate 50 times on each size to have larger samplings .... Possible?

MPG: The GUI interfaces in diglloydTools are basic, for mainstream use, the default being 5 iterations for run-sequential-suite.

The command line is versatile and powerful in its capabilities. In Terminal, here is how to iterate 50 times on volume Scratch:

disktester run-sequential-suite --iterations 50 Scratch

It’s also possible to control the starting and ending transfer sizes for run-sequential-suite using additional parameters, as well as the total test size. For example, here the transfers are from 128K to 128MB over a 2GB test file size:

disktester run-sequential-suite --starting-xfer 128K --ending-xfer 128M --test-size 2G Scratch

The command line (Terminal) is admittedly daunting for most users, but extremely powerful over the range of commands, including scriptable usage. Yet one can approach it in steps, gaining confidence gradually. For example, this command invokes DiskTester with the command run-sequential-suite on volume Scratch, using default parameters:

disktester run-sequential-suite Scratch

Reader Feedback: “WORTH it. My drive (SSD) is back and so is my productivity”

DiskTester
DiskTester

Andres R writes:

I just wanted to let you know that I resisted paying $40 for a few days, but I knew my external SSD’s performance had gone down the drain.

Sadly, other than on your site, there was little about this anywhere. Most people seems to just be happy with their drives. I’m a photographer and use the drive for intermediary photoshop work. In any case, one hour after paying my money, it was WORTH it. My drive is back and so is my productivity.

Details: LaCie Rugged USB 3.0 Thunderbolt Series 120GB Solid State Drive used as an external drive via Thunderbolt. I was down to 19 MB/sec a second for writes and it is now back up to the 300-350+ range.

MPG: the DiskTester recondition command is designed expressly for SSDs. Not all SSDs benefit, but many types do.

At the least it clears out data to zeroes, perhaps easing the load on the SSD controller as it shuffles data around internally. The dgl wipeFree command is also effective for similar reasons.

DiskTester
DiskTester recondition

While the Apple SSDs in the MacBook Pro are generally quite good, a single recondition pass brought the minimum write speed up substantially (the first pass showed a big drop in performance near the end of the free space). This test on a late 2013 MacBook Pro Retina with a 512GB SSD about half full.

DiskTester recondition of late 2013 MacBook Pro Retina 512GB SSD
DiskTester recondition of late 2013 MacBook Pro Retina 512GB SSD

OWC Thunderbay Special

OWC Thunderbay 4-drive Thunderbolt enclosure
OWC Thunderbay 4-drive Thunderbolt enclosure

OWC is having a special on the Thunderbay IV with additional savings when drives are purchased:

We are now offering savings of $20 off two drives or another $40 off with 4 drives ($60 total) purchased with an OWC Thunderbay IV 0GB 4-Bay Thunderbolt Enclosure Kit which offers up to 895MB/s sustained data rates with 4 x 7200RPM HDDs (now $479).

Qualifying drives include select Toshiba, HGST, and OWC SSDs. Free delivery within the 48 contiguous US States.

See the MPG review of the Thunderbay IV.

Detecting Corruption / Validating Data Integrity Over Time and Across Drives and Backup/Restore

IntegrityChecker verify command
IntegrityChecker

An overlooked aspect of data management is data integrity: are the files intact tomorrow, a year from now, on the original drive and backup drive(s), or perhaps even on a DVD or BluRay. Or after having been transferred across a network.

Knowing that files/data is intact with no damage is a key part of any system restoration/update/backups/archiving. In some situations it could be mandatory (record keeping). The more valuable the data, the more important it is to consider the risks to loss, which include loss by file corruption as well file deletion (not to mention viruses and software bugs and user errors).

“Data” can mean image files (JPEG, TIF, PSD/PSB, etc) or video clips or projects, Lightroom catalogs, etc. Or it could mean spreadsheets, word processing files, accounting data, and so on. Knowing that these files are 100% intact leads to a comfort level in making system changes in storage approaches.

How can data be damaged? Disk errors, software bugs in applications or drivers or the system itself can happen. Moreover, the “damage” could be user-induced: saving over or replacing/deleting a file inadvertently. Simply having a “warning flag” could be useful in noting that “no expected changes” is violated.

For example, suppose that a new computer system is acquired and various drives need to be transferred over. Or that you have upgraded to a newer and larger hard drive. Or swapped SSDs. Or there is a need to restore from a backup. Or that you burned files to a DVD or BluRay—are they intact with no changes? Even RAID-5 with its parity data does not validate files when reading them, and a validate pass is over the entire volume with no selectivity for the desired file(s).

Enter IntegrityChecker, part of diglloydTools: at any time, files of any and all types can be checked against a previously computed “hash”, a cryptographic number unique to the file. If there is a mismatch, the file has been altered, somehow. This check can be made at any time: on the original, or on a 1000th-generation copy of that file. The only requirement is that the hash be computed once and remain in the same folder as the file for later reference.

How it works with IntegrityChecker.

IntegrityChecker computes a SHA1 cryptographic hash for each file in a folder, storing those hash numbers in a hidden “.ic” file within that folder. Thus, all files in the folder have a “hash value” against which its current state can be checked.

The process can be run on folder(s), or an entire volume.

  1. Run Update on the original files (computes and writes the hash values for every file in each folder into a hidden “.ic” file in that folder).
  2. Make the copy or backup or burn the DVD/BluRay or whatever (this naturally carries along the hidden “.ic” file in each folder).
  3. At any later time (tomorrow or a year later), run Verify on any backup or copy (this recomputes the hashes and compares to the values in the “.ic” file).

For example, some pro photographers burn DVD or BluRay discs containing folders on which IntegrityChecker has been run; these discs carry along the “.ic” file in each folder, and thus can be verified at any time. There are numerous such uses.

Usage

Both command line (Terminal) and GUI versions are provided. The GUI is basic, but the internals are what counts: one of the most efficient multi-threaded programs of any kind you’ll ever find. IntegrityChecker runs as fast as the drive and CPUs can go. Available commands include 'status', 'update', 'verify', 'update-all' and 'clean'.

See How to Safely Transfer Data or Verify Backups and also Example of Verifying Data Integrity.

Continues below.

IntegrityChecker reporting on verification results
IntegrityChecker reporting on verification results

Worth doing or happy go lucky?

For many computer users, the consequences are of little importance if a few things go bad: a song, a picture, a particular document; no big deal. But even such users would be upset losing years of photos—bugs in software (gray swan?) can have widespread impact; data integrity checking is a sanity check on assumptions.

But in a financial and obligatory professional duty sense, professionals need to consider the end-to-end processes they use. When data is one’s livelihood, attention to data integrity takes on new importance.

The greater the value of the data and the greater the time span over which the data has value, the more important it is to implement processes that minimize the chances of loss, because over years the storage format is likely to change with transitions and copying, etc. Also, knowing that a backup restored from a crash is valid takes some of the sting out of a crash.

Big Fault Tolerant Storage: OWC Now Offers a Supported and Preconfigured RAID-5 Solution via SoftRAID + Thunderbay

RAID-5 is going mainstream.

RAID-5 is a fault-tolerant form of RAID that is all but essential for anyone storing large data sets, or for whom downtime is unacceptable. Videographers, photographers, etc. Note however that RAID is not a backup.

OWC now offers a high performance preconfigured and fully supported plug-and-play RAID-5 solution utilizing SoftRAID 5.

View OWC Thunderbay RAID-5 solution and/or pre-tested cold spares

The solution consists of the OWC Thunderbay with four drives configured as a RAID-5. Optional and highly recommended extras are the pre-tested and pre-configured cold spare. Should a drive fail, OWC supports the solution via its usual technical support and 48 hour turnaround. The cold spare means that in the event of drive failure, the unit can get back to rebuilding its fault tolerance immediately (no ordering or shipping delays).

Usable capacity with Thunderbay configured as RAID-5 solution:

20TB ===> 15TB
16TB ===> 12TB
12TB ===> 9TB

See also:

Three 5TB RAID-5 volumes using four 5TB drives
Three 5TB RAID-5 volumes using four 5TB drives
OWC Thunderbay 4-drive Thunderbolt enclosure
OWC Thunderbay 4-drive Thunderbolt enclosure

SoftRAID 5: RAID-5 Can Be One Huge Volume, or Partitioned to Suit

SoftRAID 5 makes it easy to partition a fault tolerant setup using RAID-5 into one huge volume or several smaller ones.

RAID-5 Can be Partitioned to Suit

Three 5TB RAID-5 volumes using four 5TB drives
Three 5TB RAID-5 volumes using four 5TB drives

Includes performance implications.

Three 5TB RAID-5 volumes using four 5TB drives
Three 5TB RAID-5 volumes using four 5TB drives

SoftRAID 5: How it Works (Screen Shots)

Some new pages are published showing SoftRAID 5 in operation.

RAID 5 in degraded state
RAID 5 in degraded state

SoftRAID 5: Performance with RAID-0, RAID-5, RAID-1+0

Just published are multiple pages showing the performance of SoftRAID 5 with RAID-0, RAID-5, RAID-1+0 and single drives.

All tests with DiskTester.

Spare hard drive mounted in drive sled for use as cold spare
RAID 5 performance with 4 and 5 hard drives, SoftRAID 5 beta

Fault Tolerant RAID: Cold Spare Advised

When using fault tolerant RAID, the smart move is to have a cold spare on hand.

For example, a RAID-5 reverts to RAID-0 when a drive fails. You want that puppy rebuilt ASAP, to restore fault tolerance (failure of a 2nd drive would mean loss of the RAID).

In plain language, this means that when purchasing a unit like the 4-drive OWC Thunderbay to be used as a RAID-5 or RAID-1+0, buy a 5th spare drive and pre-test and pre-initialize it.

Spare hard drive mounted in drive sled for use as cold spare
Spare hard drive mounted in drive sled for use as cold spare

Testing Multiple Drives for Consistency for a RAID or Similar

diglloydTools

Among its many features, diglloydTools DiskTester offers a run-area-test command that can be used for relatively fast characterization of drive performance across its capacity. The results can be graphed to get a good idea of the drive performance.

For example, suppose a high performance RAID is envisioned: one laggard can cut the performance, since the slowest drive determines the peak speed.

As shown below, 8 samples were tested, and one of those eight is a significant laggard; this slower drive is best set aside as a cold spare or backup drive.

While DiskTester can test any number of drives simultaneously (via command line), testing should take care not to saturate the bandwidth; for example four fast hard drives can demand almost all of the Thunderbolt v1 bandwidth, and other devices on the bus could suck up bandwidth, disturbing the results.

Purchase diglloydTools

Click for larger graph.

diglloydTools DiskTester: graphing performance across volume capacity for 8-drive volume (RAID 1+0)  disktester run-area-test --iterations 5 --test-size 4G --delta-percent 5 r1+0
diglloydTools DiskTester: performance across volume capacity for eight drives
disktester run-area-test --iterations 5 --test-size 4G --delta-percent 5

diglloydTools Updated to version 2.2.7b1

diglloydTools

diglloydTools DiskTester has been updated with some new and modified functionality; see the release notes. It can be downloaded as a beta version (more changes/additions are planned).

Purchase diglloydTools

This beta release focuses on DiskTester, the most powerful OS X tool available for understanding drive performance (hard drives and SSDs, RAID, etc).

See the recent discussion of disktester graphing.

diglloydTools DiskTester: graphing performance across volume capacity for 8-drive volume (RAID 1+0)  disktester run-area-test --iterations 5 --test-size 4G --delta-percent 5 r1+0
diglloydTools DiskTester: graphing performance across volume capacity for 8-drive volume (RAID 1+0)
disktester run-area-test --iterations 5 --test-size 4G --delta-percent 5 r1+0

diglloydTools and data integrity verification

Aside from testing hard drive or SSD or RAID performance and reliability with DiskTester, data integrity with IntegrityChecker is a must-have workflow tool for anyone with important data:

Thunderbolt: Does the Cable Length Matter for Performance?

Fascinating results for both hard drives and SSDs.

OWC 30 Meter Optical Thunderbolt Cable.

UPDATE: MPG goofed on the SSD graph. The graph is now corrected. With SSDs, it shows only a minor difference in speed, diminishing with increased transfer sizes to insignificant levels.

Testing using DiskTester, with the revealing run-sequential suite command.

Transfer speed over range of I/O sizes, 1 meter vs 30 meter cable
Transfer speed over range of I/O sizes, 1 meter vs 30 meter cable
Transfer speed over range of I/O sizes, 1 meter vs 30 meter cable
Transfer speed over range of I/O sizes, 1 meter vs 30 meter cable

Awesome: SoftRAID with RAID-5 rebuild

I’ll be reporting on SoftRAID version 5 in some detail. With the combination of RAID-5 and the OWC Thunderbay, software RAID has finally come of age—as in a 'killer' solution with many advantages over hardware RAID (ditto for RAID 1+0 and RAID-1).

Fault tolerance in the context of storage means that when a drive fails, you keep working and keep your data. You replace the failed drive, the solution rebuilds things onto the replacement and nothing has been lost. You can even keep working the entire time (at least with SoftRAID). Fault tolerance is critical to workflows where going down even for a few hours or a day is a big problem, but be clear that fault tolerance is not a backup.

SoftRAID 5 fault tolerance via RAID-5

Really cool demo of fault tolerance—

  1. I took the front cover off the OWC Thunderbay, a 20TB unit (four 5TB drives), which I had set up as a RAID-5 (fault tolerant to one drive failure).
  2. I yanked out one drive (hot unplug). SR5 promptly advised me that a drive had gone missing.
  3. I then reinserted the drive and in under a minute the RAID-5 had been rebuilt, good as before.

Had the drive failed entirely, the rebuild would of course taken far longer (the entire volume size would have to be updated on a replacement drive). But this level of fail/recover is incredibly cool and not something hardware RAID delivers.

Cold spares

I also tested the fail/replace scenario (using a small volume to speed up the test cycle; a volume can be any size desired). I tested by hot-unplugging a drive and then hot-plugging an uninitialized cold spare. I did this twice. Each time the RAID-5 rebuilt successfully. I had also filled the volume with data and ran IntegrityChecker before, during and after the rebuild—perfect data integrity. SoftRAID also has a Validate command and I ran that also. All this after simulating two failures with two cold spare replacements.

There was one glitch the 2nd time which involved the SoftRAID app not starting the rebuild, claiming the drive was not responding; a reboot cured that and allowed the rebuild to proceed. I am told that this is on the “hot list” of bugs to be fixed before SR5 goes final.

The case of rebuild under intensive write activity had an issue, which developer Tim Standing is keen to fix before final release (most solutions suggest or even preclude using a RAID-5 while a rebuild is in progress, so context must be kept in mind here and please note that this is a beta version, NOT a final release).

Bottom line: this is a very exciting solution. I’ve spent a lot of time testing it and working with the developer, because I think it’s a breakthrough approach with the new world of multi-drive products like the OWC Thunderbay.

SoftRAID 5 rebuilding after hot unplug and replug of one of four drives in a RAID-5
SoftRAID 5 rebuilding after hot unplug and replug of one of four drives in a RAID-5
SoftRAID 5 rebuild success in under a minute after hot-yanking a drive in a RAID-5
SoftRAID 5 rebuild success in under a minute after hot-yanking a drive in a RAID-5

Partitioning too

This particular volume was 15TB, but SoftRAID allows any volume size. You could even go crazy using these four drives: a 4TB RAID-5 plus a 5TB RAID-0 stripe plus a RAID-5 or RAID 1+0 of the remainder. That’s not a recommendation (and the sizes are just stated as examples), but it shows just how powerful SoftRAID is.

Two recommended scenario for a 20TB OWC Thunderbay:

  • A single 15TB RAID-5 volume, say for storing video. Backup strategies vary here, since 15TB won’t fit onto any single drive solution.
  • Three 5TB RAID-5 volumes (say, Master, Archive1, Archive2), for storing photography work in discrete chunks (e.g. 1998 through 2008 on Archive1, 2008 through 2012 on Archive2, current work on Master). To be backed up to 5TB single external drives, thus size-matching the volume to the backup drive (5TB volumes could be clone to 4TB or even 3TB backup drives, so long as the actual data fits).

FIXED in 10.9.3: OS X File System Hang Bug

Hooray!

As far as I can determine, the OS X File System Hang Bug has been fixed in OS X 10.9.3. While my bug report has not been updated to reflect the fix, I have yet to see the issue resurface in 10.9.3.

However, Apple developer support tells me that the bug is NOT fixed in 10.9.x (but to be fixed in 10.10 Yosemite), so the workaround of disabling file system journaling is still a wise precaution when testing drives.

Through June 9: Take 20% off Consulting

2013 Apple Mac Pro, innards
2013 Apple Mac Pro, innards

Want to build out a Mac Pro system, or similar system? Or transition from an iMac?

Get the best performance, reliability, backup, fault tolerance for your needs without spending for things not needed (or that drop performance instead of increasing it). I usually save my clients money, often more than my fee.

Whether it’s Photoshop, Lightroom, CaptureONE Pro or similar, workflow and organizational structure, backup and fault tolerance, getting the right gear and procedures into place can yield years of efficient usage.

For the remainder of May, take 20% off my usual consulting rates.

  • Must be scheduled in advance (hours flexible, international clients often use Skype or FaceTime to my phone for example.
  • Single client only (group calls excluded).

OWC Thunderbay as RAID-5 or RAID 1+0 Video Storage Solution

Shooting video (particularly 4K video) means big storage capacity (including the working versions that Final Cut Pro and similar programs like to generate).

But having it all “go away” due to a drive failure is not something pleasant to contemplate. Backups are a given of course, but restoring even 4TB is an all-day job. Restoring 8/12/16TB or more could take your work down for several days.

Enter the idea of fault tolerance, storage that keeps working when a drive fails (replace the drive ASAP to restore fault tolerance).

A video storage solution should include fault tolerance, which means using a RAID-1 mirror, RAID-5 or RAID 1+0. Of these, RAID-5 is most attractive, offering an ideal blend of high performance and the ability to tolerate a drive failure. A RAID 1+0 (stripe of mirrors) reduces capacity, but adds somewhat more fault tolerance. RAID-1 mirrors of 2/3/4 drives are also possible (2 or more identical copies).

Whatever the RAID choice, keep an extra matched drive handy (“cold spare”) so that when a drive fails it can be removed and replaced so that fault tolerance can be restored. And remember that 2/3/4 years down the line, it might be hard to find a matching drive, so it is wise to get that spare from the get-go.

Achieving fault tolerance

SoftRAID 5 is in late beta and should arrive on the scene sometime soon. MPG has been working with SR5, and the driver is reliable and fast.

Enter the 16TB or 20TB OWC Thunderbay units: using SoftRAID 5, any of the above RAID configurations are possible using the 4 drives in the Thunderbay solution. Together with a single Thunderbolt cable from 0.5 to 30 meters long, the unit can be placed next to the computer or well away in a closet or similar.

As can be seen, fault tolerance does not have to come with a performance penalty; these are excellent speeds!

Get DiskTester.

Sustained I/O transfer speed varies with the size of the chunk read or written
Sustained I/O transfer speed varies with the size of the chunk read or written

Making a RAID-5

One might hope for a plug-and-play solution to appear once SoftRAID 5 goes final. It seems logical at least, so let’s hope that works out in the market. But SR5 is very easy to use: select the drives, enter a volume name, pick the type of RAID, click Create. You’re done.

The MPG recommendation for most situations is to use RAID-5 within a single 4-drive Thunderbay unit. That’s a huge amount of storage with a 16TB or 20TB Thunderbay (12TB or 15TB usable capacity in RAID-5).

But it is also possible to make a RAID-5 out of two units (8 drives). So just to make the point, shown below is SoftRAID 5 creating a 35TB capacity volume from eight 5TB drives in two Thunderbay units. That configuration is not a recommendation (more robust to make two 4-drive volumes, each a RAID-5), but it makes the point.

MPG created and tested this very configuration on a MacBook Pro, connecting two OWC Thunderbay 20TB units using two cables, one to each port on the computer. Just for fun, a 4K display was daisy-chained off one Thunderbay unit with no performance consequences (though the refresh rate is limited to 30 Hz on TB1).

SoftRAID 5 making a 35TB RAID-5 using dual OWC Thunderbay units (8 drives)
SoftRAID 5 making a 35TB RAID-5 using dual OWC Thunderbay units (8 drives)

Recommended units are shown below. The 16TB Thunderbay solution yields 12TB usable space in RAID-5, while the 20TB unit yields 15TB usable capacity.

How to Configure 2013 Mac Pro + many Mac Pro Configs in stock at B&H Photo

Search for in-stock models using the handy search buttons on the diglloyd gear page. Don’t forget the savings on AppleCare either. Thank you for using those links.

My number one recommended configuration of the 2013 Mac Pro is the 6-core with 1TB SSD (flash).

Note that MPG strongly recommends getting the 1TB flash option because there is no upgrade path at this time. 512GB might be OK for some users, but avoid the 256GB option (too small).

At this time MPG considers the D300 GPUs a fine choice for the vast majority of users (video use excepted), so don’t sweat the GPUs, meaning the “faster” GPUs remain unproven as to any merit over the D300 GPUs.

Not sure which system and config is best for you? Take 20% OFF on MPG consulting, through the end of May. MPG can help you choose the most appropriate machine, the best way to configure storage for your own workflow, a backup strategy, etc.

Stuff you’ll want

Memory

Head over to OWC and grab 64GB of memory. Don’t forget the OWC memory rebate for the memory being replaced, which defrays the cost even further vs buying 64GB Apple memory.

16GB memory module at 1866 MHz for 2013 Mac Pro
16GB memory modules at 1866 MHz for 2013 Mac Pro

Storage

Add the 16TB or 20TB OWC Thunderbay and you’re “good” (though you might want external backup storage too).

A single Thunderbolt cable up to 30 meters long (!) connects the Thunderbay, whose four drives can be used singly or in RAID-1, RAID-0 or RAID-5 or RAID 1+0.

 

Dell UP2414Q 4K UltraHD Display Review Updated

My review of the Dell UP2414Q 4K UltraHD display over at diglloyd.com has been updated with a discussion of the display aspect ratio vs common image formats as well as a conclusions page.

Dell UP2414Q displaying an UltraHD diglloyd aperture series
Dell UP2414Q displaying an UltraHD diglloyd aperture series

CPU Cores: Specifics Matter

The question of how many CPU cores are useful is always relevant to selecting the ideal system for your own particular workflow needs. For MPG that meant the 8-core 3.3 GHz system (special upgrade) because that particular CPU offers the fastest clock speed coupled with 8 cores, useful for intensive photography work.

The question is never cut and dried unless specific workflow is considered, because more CPU cores are generally a tradeoff with reduced (slower) CPU clock speed, so only a few slower CPU cores are used, then the workflow is slower on a machine with more cores!

There are other considerations too, such as background jobs while continuing to work interactively without disruption. Still other programs chew up CPU cores (“busy waiting”), but don’t actually get work done any faster ( software that does not scale).

Bottom line: on machines with 6/8/10/12 cores, can the software being used “feed the beast” efficiently enough for those cores to matter? This actually applies even more to high performance GPUs, which is why “faster” GPU choices are often not any faster: empty work queues.

See also: MPG Experience: a Month Using the 8-core 3.3 GHz 2013 Mac Pro

CPU Core usage on 8-core Mac Pro   CPU Core usage on 8-core Mac Pro
CPU Core usage on 8-core Mac Pro
(800% is full CPU usage)
This CPU core is underutilized: less than full height bars (black space)  Actual usage looks to be around 30% on average  
This CPU core is underutilized: less than full height bars (black space)
Actual usage looks to be around 30% on average

4K UltraHD Display: Dell UP2414Q

See the review of the Dell UP2414Q 4K UltraHD display over at diglloyd.com (as well as the wide-gamut professional NEC displays).

A 4K UltraHD display is a radical viewing experience with the right source material, see for example the diglloyd publications which utilize 4K UltraHD images.

Dell UP2414Q connected to late 2013 MacBook Pro Retina
Dell UP2414Q connected to late 2013 MacBook Pro Retina

OS X 10.9.3 Fixes 90° Screen Rotation Bugs

The screen rotation issues are fixed in OS X 10.9.3 , as are various other graphics driver bugs affecting the 2013 Mac Pro with its AMD GPUs.

90° screen rotation broken on 2013 Mac Pro (should offer 1600 X 2560)
90° screen rotation broken on 2013 Mac Pro
(should offer 1600 X 2560)

Remainder of May: Take 20% off Consulting

Want to build out a Mac Pro system, or similar system? One that gives you the best performance, reliability, backup, fault tolerance for your needs without spending for things not needed (or that drop performance instead of increasing it). I usually save my clients money, often far more than my fee.

Whether it’s Photoshop, Lightroom, CaptureONE Pro or similar, workflow and organizational structure, backup and fault tolerance, getting the right gear and procedures into place can yield years of efficient usage.

For the remainder of May, take 20% off my usual consulting rates.

  • Must be scheduled in advance (hours flexible, international clients often use Skype for example, or Apple FaceTime sound messaging).
  • Must be completed in May.
  • Single client only (group calls excluded).
2013 Apple Mac Pro, innards
2013 Apple Mac Pro, innards
© Copyright 2014 DIGLLOYD INC. All Rights Reserved

United Airlines WiFi: Google Chrome and Firefox Work, Safari DOA

On a United Airlines flight today, I decided to try the in-flight Wi-Fi just to see if it was a viable service.

In Google Chrome or Safari, attempting to use the in-flight WiFi shows the payment page to initiate access. In OS X Safari (OS X Mavericks 10.9.3), Safari just hangs with a blank page. Retrying and reboots to no avail.

mpg image
UnitedWiFi.com in Google Chrome (or Firefox)
mpg image
UnitedWiFi.com in Safari

So I tried Chrome. After figuring out that the sign-in payment form had truncated my confirmation code because I had pasted in a space (rather confusing error), I ran into another roadblock. Presumably it all works, usually, but I checked with my seat neighbor and it had gone dead for quite a long time.

UnitedWiFi.com varies in availability
UnitedWiFi.com varies in availability

Special: 4TB for $165, also, 1TB Laptop Drives

MPG has about 20 of these 4TB Hitachi drives, now 4TB Hitachi on sale for $165 at OWC.

Update: the 4TB NAS version for $169 looks even more attractive—a few bucks more with a 3 year warranty (“NAS” rated for continous duty usage).

Other deals:

$75.00 for 1.0TB 2.5" HGST Travelstar 7K1000 7200RPM SATA 6Gb/s 9.5mm Notebook Drive 32MB Cache

$79.99 for 1.0TB Toshiba HD/SSD Hybrid High-Performance 2.5"

While supplies last or through 25 May.

Crypto Won’t Save You Either

Well worth reading; skip the technical as that is not the point.

Crypto Won’t Save You Either

Pages 10 - 31 give an gobsmacking summary of how cryptography is not defeated, but simply bypassed.

Safely Recording and Storing Video Footage: a Few Pointers

Video files can be large, particularly if usage extends into the 4K UltraHD realm with formats like 10-bit 4:2:2 (very high quality) or HD video at 60 fps or similar.

A few quick tips on storage particularly applicable for video shooters:

  • Fault tolerant RAID is well advised for professional use, particularly on a deadline or working with a crew where the footage has to be accessible continually: a RAID-1 mirror or a RAID-5 or RAID-6 or RAID-1+0) lets you keep working even if a drive fails (keep a cold spare on hand in case of drive failure). SSDs for capture are all well and good but even SSDs can fail, and some capture devices support
  • RAID fault tolerance is not a backup. RAID is not a backup. RAID is not a backup. RAID is not a backup!!!
  • Don’t assume that SSDs are fail proof. Don’t assume that SSDs are free of performance drop-outs: choose gear carefully and pre-flight all, including pre-testing to full capacity and considering a data integrity validation process.

Compressor: Needs some Bug Fixes

I was going to run some Compressor tests (part of Final Cut Pro, Compressor 4.1.1), but it has several bugs including the one shown below.

It runs reliably as long as I don’t try to enable multiple instances , otherwise it likes to fail as shown, or stall/wait. Also, quitting it leaves orphaned daemons around still computing heavily. Multiple reboots show that it sometimes works and sometimes does not (several successful runs with multiple instances enabled). This is on the 2013 Mac Pro, a fresh install. But between the flaky behavior and mdworker kicking in while testing, I’ll likely defer any efforts around Compressor.

Transcoding 3840 X 2160 4K UltraHD video to 1080p using QuickTime
One of several bugs using multiple instances in Compressor
Transcoding 3840 X 2160 4K UltraHD video to 1080p using QuickTime
Multiple instances in Compressor —flaky behavior

As an FYI/context, shown below is a 4K video clip set up for batch processing a 4K video clip to 1080P.

Transcoding 3840 X 2160 4K UltraHD video to 1080p using QuickTime

Deducing GPU Usage for Final Cut Pro and Compressor by Watts

I wanted to find some supportive evidence that the GPUs were being used for video transcoding, given that Final Cut Pro and Compressor both use nearly all the CPU power. There are not tools I know of to monitor GPU usage (I tried Instruments with no luck).

So I applied a simple logic: use of GPUs should increase the power draw.

Using a Watt’s Up AC power meter, I measured the power draw in watts of the 2013 Mac Pro (8-core 3.3 GHz, D700 GPUs).

MemoryTester 'stress'*:              ~ 240 watts (800% CPU usage)

Compressor transcoding to mp4:       ~ 195 watts (670% CPU usage)

Final Cut Pro transcoding to mp4:    ~ 290 watts (800% CPU usage)

FCP + Unigine Valley render:         ~ 380 watts (800% CPU usage)

* Using the MemoryTester 'stress' command of diglloydTools

Transcoding 3840 X 2160 4K UltraHD video to 1080p using QuickTime

Notes:

  • The power draw while using Compressor suggests no GPU usage, since it uses most of the CPUs yet draws notably less power than MemoryTester (which uses no GPU). This makes no sense for Compressor given its nature, but that’s the implication.
  • The Final Cut Pro power draw is significantly higher than with just CPU usage alone, suggesting that GPU usage is employed by FCP.
  • Transcoding in FCP while simultaneously running Unigine Valley render (video fly-thru) becomes slightly jerky, but pushes the power draw up substantially, suggesting that GPU capacity is underutilized by FCP, or simply that some capabilities are inapplicable to transcoding.

There is a lot of hot air coming out the top of the 2013 Mac Pro in the simultaneous FCP + Unigine Valley render!

Fabio I writes:

Why don't simply use iStat Menus to watch GPU current drawing (in Amps) and cores temperature? You could set these values (among others, the same about CPUs for example) on the top screen bar.

MPG: total power draw is one easy number. But this is a very good idea to look at to see if one or two GPUs are being used.

Transcoding Panasonic GH4 UltraHD 4K Video to 1080p: Surprising Speed on 2013 Mac Pro with Final Cut Pro and QuickTime

Here’s a surprise I found today: transcoding Panasonic GH4 3840 X 2160 4K video to 1080p using QuickTime Player, the MacBook Pro Retina (late 2013) blew the doors off an 8-core 3.3 GHz 2013 Mac Pro with D700 GPUs. Both tested on fast internal SSDs to rule out I/O as a factor.

4K video chews up space, store it on a high capacity Thunderbolt unit.

When I investigated, I found a simple explanation: the Mac Pro uses all 8 of its 3.3 GHz CPU cores, whereas the MacBook Pro uses very little CPU time. This suggests that GPU support in QuickTime Player is not used on the 2013 Mac Pro in that context (see comment from reader Michael C at end). But it does show that the GPUs can rock, if used!

Further use with Final Cut Pro and Compressor suggest that this speed differential vanishes, so perhaps there is a quality or other aspect that goes missing when using a simple QuickTime Player transcode.

Note that older Mac Pros can benefit for video processing from one or more fast video cards—still photography to a lesser extent, much more so for video.

Sidebar issue: attempting to use DaVinci Lite on the 2013 Mac Pro to process video results in an AWOL GPU subsystem that requires reboot—but it won’t reboot due to a hang, hence forced power off is the only way to restore proper system function. I thought that DaVince Lite might make use of the GPUs for transcoding, but I have not been able to make it function.

Transcoding 3840 X 2160 4K UltraHD video to 1080p using QuickTime
97 seconds to convert 24 sec 4K clip

Final Cut Pro requires 97 seconds to transcode the 4K video clip to 1080p mp4 file. The MacBook Pro Retina takes about 30% longer. Similar results with Compressor. So something very different occurs with a simple transcode in QuickTime Player vs Final Cut Pro/Compressor (similar file sizes for QTP, but smaller so maybe there is a lower quality hence faster, though if so, it’s not obvious in the output while viewing it).

Apple’s web site states “And Final Cut Pro uses both GPUs to accelerate background rendering and speed...”. Having no frame of prior reference, maybe I’m missing something as to what does and does not take time, quality metrics, etc. It’s unfortunate that there are no monitoring tools for GPU usage.

The MPG Mac Pro has the D700 GPUs and 3.3 GHz 8-cores, so one does have to ask with some workflow concern: is 4X to 10X real time (4 hours to transcode one hour of video) as fast as Compressor will go on the 2013 Mac Pro? At the most basic level, the question is whether those dual GPUs really being used?

So I investigated whether the GPUs are used, by the indirect means of power draw.

MORE info and context: Compressor (part of Final Cut) is far slower on the MacBook Pro than a simple Quicktime Player 1080p conversion on the same MBP, and ~30% slower than the Mac Pro. Moreoever, both use the CPUs fully in FCP/Compressor.

As a novice video user, my expectations are ingenuous; starting to dip me feet into the water, I just tried features such as “Export” or “Share” and I ask what happens. Best advice: test your own workflow with your own video and see what happens. It’s clear that I need to learn a lot more about video transcoding and the like to make sense of it all.

Reader comments below.

Transcoding 3840 X 2160 4K UltraHD video to 1080p using QuickTime
Transcoding 3840 X 2160 4K UltraHD video to 1080p using QuickTime
Transcoding 3840 X 2160 4K UltraHD video to 1080p using QuickTime
Transcoding 3840 X 2160 4K UltraHD video to 1080p using QuickTime

Michael C writes:

The MacBook Pro has Intel Iris graphics, which accelerates video encoding. A link to Intel's white paper about it:

http://newsroom.intel.com/servlet/JiveServlet/previewBody/3880-102-1-6896/WP-HD-Graphics-4200.pdf

The graphics cards in the Mac Pro from AMD, might be more capable at many other tasks, but AMD does not implement video encoding acceleration.

That probably explains the difference in encoding speed you are seeing between the MacBook and Mac.

MPG: I’ll ask some questions (pleading ignorance); I don’t know the answer here. It does seem very hard to believe that the dual GPUs in the 2013 Mac Pro would not be well suited to video work, including transcoding: what else is the point of having them?

Amir N writes:

Your updates on the post re: FCP vs. Quicktime seem to support that quicksync in Quicktime X is the factor. Quicksync transcodes will be slightly lower quality (and smaller in size) than CPU-only transcodes, which is exactly what you're seeing. Moreover, GPU based transcode should put almost no load on the CPU, and you're seeing all CPU cores light up on the Mac Pro during a Quicktime transcode while the MBP shows almost none, meaning that the Mac Pro is doing a pure CPU/software transcode, while the MBP is doing a quicksync transcode.

Then the fact that the transcode times and CPU usage times look more reasonable in the FCP transcoding confirms it to me. I'd bet dollars to donuts that quicksync is enabled by default in Quicktime X, and that neither FCP or Quicktime X are currently able to take advantage of the Mac Pro's discrete GPUs.

MPG: makes sense, but power usage suggests that the GPUs are in fact being used.

OWC Thunderbay Now up to 20TB as 4 X 5TB Drives

OWC Thunderbay 4-bay enclosure
OWC Thunderbay 4-bay enclosure

The OWC Thunderbay 4-bay Thunderbolt enclosure is my go-to solution for storage on my 2013 Mac Pro (three of them now attached).

Higher-capacity drives are starting to appear on the market (5TB and 6TB models), and OWC has now qualified 5TB drives for the Thunderbay and is offering a 20TB Thunderbay solution.

The specs indicate that the Thunderbay 20TB uses the 5.0TB Toshiba MD04ACA500 (7200 rpm, 128MB cache). Note that the single-drive OWC Mercury Elite Pro is also available in a 5TB capacity. Ditto for the Elite Pro Dual 10TB.

Separately, another high capacity drive now on the market is the the 6.0TB HGST Ultrastar He6. MPG has not yet tested either drive.

How to use all that space? Simply hooking up any high-capacity unit is only a first step. Questions of RAID (or non-RAID, and which type), organizing data, which data should go onto fast SSDs vs external HDD storage, backup strategy, etc all are important issues to consider. MPG advises photographers and videographers on the best ways to address all these issues via consulting.

See also yesterday’s quiet fan comments as well as the fan upgrade for models released earlier this year.

OWC Thunderbay Fan Noise = Nil

I’ve settled on the OWC Thunderbay 4-bay Thunderbolt enclosure as the go-to solution for storage on my 2013 Mac Pro (three of them now attached). It’s such a clean solution with 4 bays via Thunderbolt—what a relief from the days of eSATA and MiniSAS.

When the 3rd unit arrived, I inserted two SSDs into it, turned it on, and was disappointed to find it wasn’t working. Well, what I mean is that the fan was so quiet I thought it wasn’t working! Putting my ear close to the unit, I could hear it running and indeed the drives mounted and all was well. With hard drives installed, “spinner” noise will dominate any fan noise.

If you have one of the original builds (much louder fan), get a replacement quiet fan for the OWC Thunderbay (for units produced early in 2014—builds as of March 10 had a quieter fan and April builds might have been still quieter).

OWC Thunderbay 4-bay enclosure
OWC Thunderbay 4-bay enclosure

diglloydTools Updated to version 2.2.5

diglloydTools

diglloydTools has been updated to version 2.2.5. Download page.

Purchase diglloydTools.

Changes/additions:

  • DiskTester: commands which have graphing templates now output the graph data into a text file inside the diglloydTools directory under the Documents directory of the user’s home folder.
  • MemoryTester: the compute command now outputs the graph data into a text file inside the diglloydTools directory under the Documents directory of the user’s home folder.

The graphing templates are improved in formatting and the separation of graph data into output files makes it easier to simply copy the data and paste it into the template.

diglloydTools graphing template
diglloydTools graphing template
diglloydTools graphing template
diglloydTools graphing template
diglloydTools graphing template
diglloydTools graphing template
diglloydTools graphing template
diglloydTools graphing template
diglloydTools graphing template
diglloydTools graphing template

Other capabilities in diglloydTools

Aside from testing hard drive or SSD or RAID performance and reliability with DiskTester, data integrity with IntegrityChecker is a must-have workflow tool for anyone with important data:

diglloydTools Updated to version 2.2.4

diglloydTools

diglloydTools has been updated to version 2.2.4. Download page.

Changes/additions:

  • DiskTester: The 'recondition' command for SSDs can now work with as little as 128MB of contiguous space (was 1GB). This helps under conditions where the volume is highly fragmented.
  • dgl: new command line tool (command line only at this juncture) with wipe and wipeFree commands. These commands are beta versions at present.

The wipe command supports an easy to use convenience feature: appending the suffix -wipeMe to any file or folder tags it for wiping (“dgl wipe”). All local volumes are scanned for such files in highly efficient fashion, making it convenient to mark many items for wiping, then do it in one invocation. Double confirmation is needed before any files are actually wiped.

The wipeFree command is handy for wiping free space in a variety of circumstances (including interesting cases like free space on disk images, for much improved compression of a dmg file, or helping out SSDs by clearing blocks).

Other capabilities in diglloydTools

Aside from testing hard drive or SSD or RAID performance and reliability with DiskTester, data integrity with IntegrityChecker is a must-have workflow tool for anyone with important data:


Max Your Mac Pro at OWC

diglloyd.com | Terms of Use | PRIVACY POLICY
Contact | About Lloyd Chambers | Consulting | Photo Tours
Mailing Lists | RSS Feeds | Twitter
Copyright © 2008-2014 diglloyd Inc, all rights reserved.