Mac Pro Westmere Digital Camera RAW File Processing Speed (Bibble Pro)
BibblePro eclipses other RAW converters in absolute speed, and I deem it exceptionally well written for a RAW converter in terms of efficiency. According to a BibblePro engineer, there is some room for improvement, and so further gains might be forthcoming this year (or so I hope).
How much is gained for the 12-core model depends on I/O speed, as clearly shown by using a RAM disk, with a triple-SSD RAID-0 also doing well. It’s not worth the time or risk of setting up RAM disk to get these speeds, but it does show the potential of a system with very fast disk I/O.
But see notes below for why this is so, as well as the JPEG results further below. Note that with a RAM disk, it is faster to process to 16-bit uncompressed TIF than to 100% quality JPEG, presumably because the JPEG step requires compression.
A BibblePro engineer contacted me to state the following. Unfortunately, the assumptions are incorrect—the BibblePro team needs to order themselves an MPG Pro Workstation with a dual or triple SSD RAID-0 stripe!
I look forward to seeing a version of BibblePro that removes these limitations (I intend to work with the BibblePro team to improve its performance).
The reason why the conversion to 16-bit TIFF doesn't scale as well as conversion to JPEG is because the TIFF encoding introduces a disk bottleneck and thread serialization. To try and optimize disk throughput we only allow one TIFF to be encoded at a time because the TIFF encoding is primarily disk-bound. This results in the blocking of other threads that are also disk bound. For example, another thread may be trying to read the next RAW file off the disk.
With TIF, I observed disk I/O demands of no more than 160MB/sec when converting to TIF—far short of the ~500MB/sec which my dual-SSD RAID-0 stripe is capable of. Even single hard drives today can deliver 148MB/sec, such as the Western Digital RE4. If two TIF files were processed together, I/O demands would double, yet still be well below what my system delivers. Its not a seek-time issues; batches of related files are generally located on the same or adjacent tracks, and with solid state drives as I have, there is effectively zero seek time or latency. Old assumptions must be reworked for current realities.
Scalability is not perfect with JPEG files, but there is a 50% gain with the 12-core.
Because the I/O demands with JPEG are far lower, there must be something else at play; the 12-core time ought to be closer to 56 seconds.
Bibble deserves tremendous credit for nearly pegging out a 12-core system. If you want a fast workflow, then BibblePro is your RAW converter.
Note that on the 6-core system, hyperthreading(virtual cores) is of considerable value, but that on the 12-core system, turning it off is slightly faster. The difference might be due to internal inefficiencies in BibblePro that arise at 12 cores (in part), as well as memory bandwidth and scheduling inefficiencies in Mac OS X.
CPU core usage
See Monitoring system performance with Activity Monitor. Black areas are idle, red and green are usage.
The best performer is BibblePro 5.1G. It also runs about 3-4X faster than Lightroom 3! Its preview function could be improved however, because it uses only half the CPUs on a 12-core system.