diglloyd Mac Performance Guide

Up to 8TB of Thunderbolt Storage!

100% Kona, 100% Family Owned

Understanding Compressible vs Incompressible Data

Last updated May 06, 2012 - Send Feedback

Compressible data — data that shrinks when lossless compression is applied.

Incompressible data — data that does not shrink when lossless compression is applied, or even grows in size.

Many types and brands of solid state drives (SSDs) utilize the Sandforce processor chip to implement the SSD, such as the OWC Mercury Accelsior and the the OWC Mercury Extreme Pro SSDs.

The Sandforce processor uses lossless compression to minimize the write cycles to the flash memory and to speed up write performance (as of May 2012 this was true for all SSDs based on Sandforce).

Not all SSDs compress data, which is not to say this makes other types inherently faster. This is a two-edged sword: the write performance is consistent for all types of data, but the lifetime of the flash memory might be lessened due to increased write activity.

OWC says:

Sandforce processor will do wear leveling and garbage cleanup as an ongoing process in between active periods and during activity in real time if resources allow. It’s not ongoing and completes quickly - but will occur outside data activity depending on load. Also, it adjusts algorithmic response as drive use occurs and usage is profiled.

Compression is something of huge consideration; the Sandforce processor today 2xxx series is worst case 3x write amplification with incompressible data and best case less than 0.3 with compressible. Non SF processors are typically 2 to 20x write amp today. The higher the write amp, the shorter the flash life. This will be a lot more important as NAND geometries shift to lower and lower Write PE as circuitry shrinks for higher density.

A drive under heavy worst case loads with Sandforce at 3x WA has NAND lifespan of up to 6x+ that of non SF based that don't manage in this way. That's not even with wear leveling considerations

This becomes really significant with 19nm NAND coming that will move PE to 1k or 2k write cycles. With 3k makes a difference for heavy user with time.

What happens with incompressible data

When the Sandforce processor encounters incompressible data, there is a performance reduction, because it takes more time to process the data, figure out that it is incompressible, than take Plan B for dealing with it.

The incompressible-write speed hit is for writes. In general, reading files with incompressible data is affected minimally or not at all. Since most files are written only once, the write speed is thus irrelevant to normal usage.

In the real world

In the real world, most I/O is read access, most data is at least somewhat compressible, and except for specialized circumstances, one should not be concerned with the incompressible data write speed.

Disk intensive activity using Photoshop tests (for example) show that the Sandforce-based OWC Accelsior PCIe SSD shows outstanding performance.

Type of incompressible data

Incompressible data includes certain types of TIF and PSD*, JPEG, video and sound files, ZIP files, etc.

* When possible, disable compression for TIF and PSD; this can be done in Photoshop (for example) by electing to not compress files in CS5, certain preferences in CS6, etc. This can be a major performance boost, whether or not an SSD is used.


The Speed You Need

diglloyd.com | Terms of Use | PRIVACY POLICY
Contact | About Lloyd Chambers | Consulting | Photo Tours
Mailing Lists | RSS Feeds | Twitter
Copyright © 2008-2014 diglloyd Inc, all rights reserved.