diglloyd Mac Performance Guide
Great Deals Every Day...

The Best Way to use Apple’s Photos App

As a professional photographer, and noting the demise of Apple’s Aperture, it appears that the following approach is the best way to use Apple’s new Photos.app .

Optional (in Terminal): sudo rm -rf /Applications/Photos.app

IntegrityChecker reporting on verification results
Best way for a professional to use Apple Photos app

Speaking to professionals, MPG strongly recommends eliminating as a workflow dependency any Apple software product except OS X. Apple has thumbed its nose at both video and photography professionals now, both with hardware and software. Migrate work to Adobe Photoshop CC or Adobe Photoshop Lightroom.

Professional users are now a tiny minority of Apple’s customer base, and thus are at extreme risk of being shrugged off and ignored (well, that’s long since happened). MPG has written on this starting three years ago, and subsequently many times in various ways.

Arne K writes

More serious anger from pro users:

http://preta3d.com/os-x-users-unite/ “OpenCL GPU drivers for OS X are broken and are unable to render but the simplest of scenes”

http://lists.blender.org/pipermail/bf-committers/2015-May/045367.html “going to abandon OS X as a serious 3D/graphics development platform”

MPG: tip of the iceberg. Professional users that relied on Apple for years feel betrayed. The above are just two specific examples, a tiny sliver of the issues.

Max Your Mac Pro at OWC...

Isolating/Blocking Hard Drive Vibration, Especially on a Hardwood Desk or Floor

MPG gets emails on fan noise periodically, so here’s summary of issues and mitigations.

Fan noise

Fan noise is often only a portion of the noise that one hears as “fan noise” in an external drive enclosure (unless the unit is using only SSDs, which need cooling, hence the fan). Full size hard drives make their own noise, particularly if they are 7200 rpm drives and there are four of them in an enclosure; the noise can “overlap” in harmonics in some cases, which can be annoying.

In general, decibal ratings for noise are fairly useless because the pitch as well as any oscillations in intensity or unpleasant harmonics can be far more annoying than a steady whir.

Fan noise can be completely avoided with single drive enclosures, such as the OWC Mercury Elite Pro (no fan).

For noise, get a 3 meter or longer Thunderbolt cable, and place the unit under the desk or in a closest or some distance away from right next to your work area. Put some sound-absorbing material around the unit too.

Vibration

Placed on surfaces like concrete flooring, hard drive vibration is of little concern. But placed on a hardwood desktop or floor, those vibrations can resonate into the material in a very disturbing way (often a low humming vibration). Or not, depending on the material. While the enclosure itself can eliminate some frequencies, certain frequencies are not attenuated and make it through to the surface on which the enclosure rests.

Even the best drives have some vibration, and if with 4 or 8 drives in a RAID, it gets worse. If there is a vibration problem, try 1.25 sorbothane bumpers. They work well to detach any vibrating device from a hardwood floor or wooden desk or similar.

What also works very well are Auralex MoPAD Monitor Isolation Pads. MPG uses these pads under the OWC Thunderbay on hardwood floors, because the flooring picks up vibrations very efficiently.

As reader Don H points out (and MPG has used the same approach at times), the closed-cell foam in which OWC drives are shipped works great as a base also.

Auralex MoPAD Monitor Isolation Pads
(leveling inserts not shown; can sit level too)

Detecting Corruption / Validating Data Integrity Over Time and Across Drives and Backup/Restore

IntegrityChecker verify command
IntegrityChecker

An overlooked aspect of data management is data integrity: are the files intact tomorrow, a year from now, on the original drive and backup drive(s), or perhaps even on a DVD or BluRay. Or after having been transferred across a network.

Knowing that files/data is intact with no damage is a key part of any system restoration/update/backups/archiving. In some situations it could be mandatory (record keeping). The more valuable the data, the more important it is to consider the risks to loss, which include loss by file corruption as well file deletion (not to mention viruses and software bugs and user errors).

“Data” can mean image files (JPEG, TIF, PSD/PSB, etc) or video clips or projects, Lightroom catalogs, etc. Or it could mean spreadsheets, word processing files, accounting data, and so on. Knowing that these files are 100% intact leads to a comfort level in making system changes in storage approaches.

How can data be damaged? Disk errors, software bugs in applications or drivers or the system itself can happen. Moreover, the “damage” could be user-induced: saving over or replacing/deleting a file inadvertently. Simply having a “warning flag” could be useful in noting that “no expected changes” is violated.

For example, suppose that a new computer system is acquired and various drives need to be transferred over. Or that you have upgraded to a newer and larger hard drive. Or swapped SSDs. Or there is a need to restore from a backup. Or that you burned files to a DVD or BluRay—are they intact with no changes? Even RAID-5 with its parity data does not validate files when reading them, and a validate pass is over the entire volume with no selectivity for the desired file(s).

Enter IntegrityChecker, part of diglloydTools: at any time, files of any and all types can be checked against a previously computed “hash”, a cryptographic number unique to the file. If there is a mismatch, the file has been altered, somehow. This check can be made at any time: on the original, or on a 1000th-generation copy of that file. The only requirement is that the hash be computed once and remain in the same folder as the file for later reference.

How it works with IntegrityChecker.

IntegrityChecker computes a SHA1 cryptographic hash for each file in a folder, storing those hash numbers in a hidden “.ic” file within that folder. Thus, all files in the folder have a “hash value” against which its current state can be checked.

The process can be run on folder(s), or an entire volume.

  1. Run Update on the original files (computes and writes the hash values for every file in each folder into a hidden “.ic” file in that folder).
  2. Make the copy or backup or burn the DVD/BluRay or whatever (this naturally carries along the hidden “.ic” file in each folder).
  3. At any later time (tomorrow or a year later), run Verify on any backup or copy (this recomputes the hashes and compares to the values in the “.ic” file).

For example, some pro photographers burn DVD or BluRay discs containing folders on which IntegrityChecker has been run; these discs carry along the “.ic” file in each folder, and thus can be verified at any time. There are numerous such uses.

Usage

Both command line (Terminal) and GUI versions are provided. The GUI is basic, but the internals are what counts: one of the most efficient multi-threaded programs of any kind you’ll ever find. IntegrityChecker runs as fast as the drive and CPUs can go. Available commands include 'status', 'update', 'verify', 'update-all' and 'clean'.

See How to Safely Transfer Data or Verify Backups and also Example of Verifying Data Integrity.

Continues below.

IntegrityChecker reporting on verification results
IntegrityChecker reporting on verification results

Worth doing or happy go lucky?

For many computer users, the consequences are of little importance if a few things go bad: a song, a picture, a particular document; no big deal. But even such users would be upset losing years of photos—bugs in software (gray swan?) can have widespread impact; data integrity checking is a sanity check on assumptions.

But in a financial and obligatory professional duty sense, professionals need to consider the end-to-end processes they use. When data is one’s livelihood, attention to data integrity takes on new importance.

The greater the value of the data and the greater the time span over which the data has value, the more important it is to implement processes that minimize the chances of loss, because over years the storage format is likely to change with transitions and copying, etc. Also, knowing that a backup restored from a crash is valid takes some of the sting out of a crash.

Great Deals Every Day...

2013 Mac Pro 6-core for $3175

If you need a larger SSD, OWC has up to 2TB internal SSD upgrade for the 2013 Mac Pro. The D300 GPUs are just fine for most everything, but MPG recommends the 32GB or 64GB memory kit (up to 128GB possible). Add a Thunderbay 4 for main storage, two backup drives for clone backups, and you’re “good”.

2013 6-core Mac Pro for $3175

OWC Announces Video Commercial Contest Starting May 1, Grand Prize Package Worth Over $12,000

OWC announces a big incentive to win this one. Details.

OWC Announces Video Commercial Contest Starting May 1, Grand Prize Package Worth Over $12,000

Recognized technology company encourages fans to tell the OWC story in their own way

WOODSTOCK, IL - May 1, 2015 – Other World Computing (OWC), a leading zero emissions Mac and PC technology company, announced today that OWC is hosting its first ever Video Commercial Contest. Contest entrants are challenged to bring their vision to life, in 30 to 60 seconds, showing why everything is possible with OWC, and how OWC upgrades and solutions can transform a creative professional’s approach to their craft. The contest will begin accepting submissions on May 1st and will continue until June 30th.

One Grand Prize Package, worth over $12,000 will be awarded, in addition to a Runner Up Prize Package worth over $1,400. Winners will be chosen through public voting on a contest portal beginning on July 7 and continuing through July 31, 2015, with winners announced on August 10, 2015, on the Other World Computing website.

Grand Prize Package

The lucky winner of the Grand Prize Package will be able to further develop their video creation skills thanks to the high-end computer hardware and software components and camera that make up the Grand Prize:

  • An Apple Mac Pro computer with six cores of processing power operating at 3.5GHz, the professional computer for the future.
  • A 4K monitor, providing pixel-perfect clarity for precision editing.
  • One year subscription to Adobe Creative Cloud service, featuring some of the world’s best creative software applications.
  • Avid Artist Transport console, a compact media controller that allows creative professionals to navigate and edit projects with just one hand.
  • Avid Artist Color control surface, a control surface that provides deep, hands-on control of all color correction and grading tasks.
  • NewerTech Wireless Keypad, a wireless numeric keypad that complements Apple’s aluminum keyboards.
  • OWC Thunderbolt 2 Dock, our high performance dock providing 12 ports of connectivity at 20 Gbps speed.
  • OWC Aura Pro 2TB SSD, OWC’s largest internal SSD for the Mac Pro, and an
  • OWC Envoy Enclosure for Mac Pro, which allows the lucky winner to repurpose the factory-installed Mac Pro SSD as an elegant external drive.
  • OWC ThunderBay 4 RAID 16TB RAID Edition, the most advanced software RAID 5 technology developed for the Mac, with massive storage capacity.
  • 64GB OWC Memory for Mac Pro, to ensure high performance of the system.
  • GoPro Hero4 Black, GoPro’s newest and most advanced video camera.

Runner Up Prize Package

The second highest vote gatherer will receive:

  • An OWC ThunderBay 4 RAID 16TB RAID Edition External Storage Solution
  • OWC Thunderbolt 2 Dock.

Contest Guidelines:

Contestants are asked to create an original video commercial between 30 and 60 seconds long. Be creative! Show the world why OWC solutions are the key to unlocking true creative potential using one of the following three themes:
Everything is Possible.

Go Further. The Speed to Create. The Capacity to Dream.

Some general rules to be aware of: all submissions should be English-language content only - this is to ensure the largest possible number of voters can understand the video. All voting results will be final, and the winners will be notified by email. For contest details, visit http://eshop.macsales.com/shop/video-contest/entries.

Entering the contest is easy, just submit your video beginning May 1 at http://eshop.macsales.com/shop/video-contest/entries. The key dates to be aware of are:

The opening date for submissions is May 1, 2015. Entrants will need to have a Facebook page in order to participate.

Submissions will be accepted until 11:59 PM central time, June 30, 2015.
Videos that follow the guidelines in the terms and conditions will be posted online starting on July 7, 2015 and the public voting period will run through July 31, 2015.

The winners will be announced August 10, 2015 on Other World Computing’s website.

“Over the years we have seen that OWC fans include some of the most creative and talented people out there, and this contest gives them a chance to show the world how far they can go with OWC,” said Larry O'Connor, Founder and CEO, Other World Computing. “You don’t need your own studio to enter -- the next viral OWC video commercial can be anything from simple footage shot on an iPhone to a grand production with scripting, music, animation, special effects -- the sky’s the limit, and we urge entrants to just go for it!”

The OWC Trust Advantage®

  • Over 25 years of product design and innovation
  • OWC tested for compatibility, speed, and reliability
  • Comprehensive step-by-step installation and support videos
  • 24/7 award-winning, U.S.-based technical support

About OWC

OWC is the manufacturer and upgrade provider of choice for Apple and PC enthusiasts and has been providing OWC memory upgrades for Apple computers since 1989. For the last decade, OWC has been building SSD upgrades specifically for Apple computers, winning multiple awards while also earning accolades for OWC External Drives and OWC Optical Drives. OWC memory, SSDs and a growing number of our products are designed, engineered and manufactured in Austin, TX. Widely recognized for its exemplary customer service, OWC provides extensive US-based technical support for Mac and PC users around the world. As a member of the Apple Developer Program, OWC also produces a catalog of upgrades and accessories, having served the Apple community worldwide since 1988.

OWC's industry awards include:

  • Videomaker Best of CES 2015 Spotlight Award
  • 2015 CES Innovations Honoree
  • Storage Visions, Visionary Product Award 2015
  • Inc. 5,000 "Fastest Growing Privately Owned Companies in America" list honoree 2007–2013
  • Reseller Ratings Elite Member 2009-2014
  • About.com Readers' Choice Award winner for best Mac Peripheral Reseller
  • Business Intelligence Group 2013 Green Tech Company of the Year
  • American Business Awards "Stevie" Gold (2012) and Silver (2014)
  • Better Business Bureau Top Rating of A+ 2010–2014
  • No purchase necessary to enter. Open to legal US residents 18 and over. Limit one entry per person/team. Winners must comply with all terms and conditions of the official rules, and winning is contingent upon fulfilling all requirements. OWC reserves the right to disqualify any submission that does not follow the guidelines and content restrictions listed in the terms and conditions. All submissions become property of OWC. OWC reserves the right to utilize every entry for promotional purposes. Winners will be notified via email. All prizes are nontransferable and no substitution can be made. Entrants agree to receive OWC special offers via email. Void where prohibited. For full contest rules please visit http://eshop.macsales.com/shop/video-contest.
OWC Thunderbolt 2 Dock...

Apple Pro Video Updates: Endless Repeated Updates

Mike B writes:

Just another example of Apple Core Rot. This is the 5th time the system is asking to update the Pro Video formats

MPG: the 4th repetition for me. But I was gone for two days, or I’d be up to five or six I suppose.

UPDATE: see fix below.

UPDATE 2: the fix does not work.

Apple software update repeatedly wants to update the same thing (Pro Video Formats)

Fix (does not work)

It’s an absurd situation of course. Why Apple does not monitor updates and immediately take corrective action is unclear.

An Apple discussion goes into a workaround. Here is what worked for me, using Terminal. The '#' line is a comment.

cd /Library
sudo mv QuickTime/ QuickTime.save
# install and/or do the update, no reboot needed
sudo cp -r QuickTime/*.component QuickTime.save/
sudo rm -rf QuickTime/*.component
sudo mv QuickTime.save/*.component QuickTime
sudo rm -rf Quicktime.save   

How To Partition a Drive to Free Space Using Disk Utility

Here’s how to wipe out all volumes on a drive (SSD/flash drive or hard drive). This does not write over the data, but it destroys all file systems and partitions on a drive.

Essentially the drive becomes “unformatted” and like it was out of the box, though it generally does not wipe out set-aside special areas on the drive, e.g. writeable areas or writeable non-volatile flash used by firmware.

There might or might not be salutary effects with Apple-supported flash/SSD drives (eg TRIM total reset).

  1. Select the drive on the left (not the volume(s) listed under the drive.
  2. In the Partition tab, choose 1 Partition with Format: Free Space.
  3. Click Apply.

This results in 100% data loss on this drive, so don’t choose the wrong drive.

Partition a disk / drive to free space
Great Deals Every Day...

OWC Jupiter Ideal for Video Storage; NAS and/or SAN at 10 GBps, up to 512TB

I stopped by at the OWC booth at NAB to see the OWC Jupiter system.

With up to 512TB of capacity and functioning as a NAS or SAN and with 10 gigabit connectivity (additional 10 Gbps cards can be installed for more bandwidth), the unit is a heavy duty rack-mount solution for those looking for massive storage for a workgroup. Redundant power supplies of course.

LAS VEGAS, NV - April 15, 2015 – Other World Computing (OWC), a leading zero emissions Mac and PC technology company, announced today that it will feature at its NAB 2015 Booth #SL13905 in Las Vegas, April 13-16, a range of outstanding and flexible solutions aimed at revolutionizing creative professional workflows.

The innovative solutions showcased at OWC’s NAB booth include:

Jupiter Callisto and Kore Unified Storage

OWC Jupiter Callisto and Korerack-mounted storage solutions offer the most flexible, compatible, and scalable high-performance storage systems for creative professionals working with HD, 2K, and 4K workflows. Connect via 10Gb/E and mini-SAS interfaces to unleash speeds up to 10Gb/s via Callisto and up to 48Gb/s with Kore.

Jupiter Callisto, available now and starting at $4,988 for a 16TB configuration, is a unified storage appliance that functions as both a NAS and SAN at the same time. Mac and Windows compatible, it offers storage management with IP-based remote management software. Whether building from the ground up or integrating into an existing system, Jupiter’s 8-bay 2U racks and 16-bay 3U racks provide installation flexibility with scalable ZFS architecture for the creation of an ideal professional workstation environment.

The power of the Jupiter Kore, available now and starting at $3,288 for a 16TB configuration, lies in its mini-SAS interface architecture, with the ability to run dual 24Gb/s direct mini-SAS cables to achieve speeds up to 48Gb/s. Users can get up to 64TB of storage in an 8-bay rack and up to 128TB of direct-attached or expansion storage capacity in a 16-bay rack. Callisto users can add up to three Jupiter Kore expander rack solutions for a total of 512TB of centralized storage. Need even more power? No problem. Simply add an external SAS switch and watch limitations on speed and power vanish.

The OWC Trust Advantage®
Over 25 years of product design and innovation
OWC tested for compatibility, speed, and reliability
Comprehensive step-by-step installation and support videos
24/7 award-winning, U.S.-based technical support

The Speed You Need...

Apple Watch is Out and In Crazy Demand, What’s it For?

In With the Coming of the Apple Watch, Digital Monitoring is the Next Big Thing, I suggested that Apple Watch would be a Really Big Deal.

Then I looked at what it meant to me as a serious cyclist. I’ve looked some more and I cannot find a single feature that does anything for my cycling needs. But it might incent “gotta do it” exercisers and that might end up being a public health benefit in a statistical way. More power to 'em and to Apple for that—applauded.

Then I looked some more (those hyper polished Apple videos), and while it’s all done with the usual Apple style and elegance, I was left wondering how I would possibly use one. Huh. Maybe a really good voice recorder with voice to text might itself be a worthwhile app for me at times, but to wear the watch for that seems a bit silly.

Now the Apple Watch is out—and sold out. Which is to be expected with a new and gorgeous product from the world’s most successful company of all time. I’ve enjoyed not wearing a watch for years now (though I have a quality self-winding classic), so I await something with 4X the battery life, self-charging by solar and motion, no dependency on an iPhone, a built-in camera and phone. Or a really compelling feature. I might have to wait a long time for that.

My teenagers aren’t interested in iWatch, and they tell me there isn’t much interest in iWatch among their high school friends, but I remain skeptical that it won’t be a hit with teenagers at some point—maybe some iWatch killer app that goes viral will ignite it. Perhaps the main issue is that it requires an iPhone to work well, and my kids attend a government school with a wide range of student backgrounds, and most kids there can’t afford an iPhone. But at least in Palo Alto, where even 8 year olds have iPhones, it ought to sell well to kids.

It will be very interesting to see how the iWatch market works out. New technology can disrupt unexpected areas of life, and I have a sense that the iWatch might well do that—health care perhaps, but maybe somewhere else too.

Great Deals Every Day...

OWC Thunderbolt 2 Dock: All Sorts of Ports for Every Mac with Thunderbolt

MPG reported on the about $249 Thunderbolt 2 Dock back late last year, followed up by a review of the OWC Thunderbolt Dock. Supply has caught up with demand and the dock is now in stock, so get it on the loading dock for you. :;

In use since then on the MPG MacBook Pro system, the OWC Thunderbolt 2 Dock is the best docking option out there, particularly for port-challenged systems, like laptops, iMacs and MacMinis. It’s Thunderbolt 2, but will work with Thunderbolt v1 also.

  • Five USB3 ports, two of which are high power ports suitable for charging iPhone / iPad. Can be used *in addition* to the Mac’s built-in ports.
  • Daisy-chain to another Thunderbolt device.
  • Gigabit ethernet (no need to waste a Thunderbolt port for a Thunderbolt to Ethernet adapter).
  • Pass-thru to 4K display (for Macs that support it).
OWC Thunderbolt 2 Dock , rear ports
OWC Thunderbolt 2 Dock , rear ports
OWC Thunderbolt 2 Dock, connectivity options

On Macs with only two Thunderbolt ports, (MacBook Pro, iMac, MacMini), plugging in the dock uses one Thunderbolt port, but that leaves the other port on the Mac free as well as one port on the dock. So you still get two Thunderbolt ports plus the five USB3 ports and the others.

Ports on a Laptop
  Laptop alone With Thunderbolt 2 Dock
USB 3 ports 2 7
USB 3 high power ports 2 of 2 4 of 7
Thunderbolt 2 ports 2 2
Firewire with Thunderbolt to Firewire adapter
(uses Thunderbolt port)
1
Gigabit ethernet with Thunderbolt to Ethernet adapter
(uses Thunderbolt port)
1
HDMI possible with adapter
(uses Thunderbolt port)
1
Sound varies additional sound in and out
Digital camera card reader SDXC reader built-in on some models Attach a fast USB3 card reader

High power ports are especially good for charging devices like the iPad, or powering certain USB3 devices (e.g. external SSDs) that pull more than the standard 0.9 amps on a standard USB3 port.

OWC Thunderbolt 2 Dock , high power ports

MacBook Pro: What Could be Coming Next?

The Apple MacBook Pro hasn’t changed in any material way since late 2013, and that’s why MPG is still using that model—there is no meaningful difference vs the current model (hint: a used late 2013 MBP with AppleCare is still a terrific laptop).

MPG’s current recommendation for power users for a new laptop is the 15-inch 2.8 GHz model with 1TB SSD. Along with (for home/office use), the OWC Thunderbolt Dock plus keyboard and mouse (both wired) and separate 4K display.

Current trends in the Macbook / MacBook Pro line are iPad-wannabe laptops (a very rough analogy, but the new MacBook is of this genre), but not to more powerful high end units. All well and good, but it would be nice to see product line expansion in both slimness direction and power user direction. And possibly a hybrid model that is an iPadBook, so to speak—convergence.

Being optimistic that Apple might not abandon power users in all areas as seems to be the trend, what might arrive in the high-end revised model later this year?

  • Faster internal flash drive (SSD) offering up to 1600MB/sec or so. Sounds great and yes it is, but except for extreme computing needs, this won’t make any difference whatsoever for 99% of users 99% of the time.
  • Higher capacity internal flash drive (SSD), perhaps up to 2TB. Now that offers real value for some, instead of lugging along extra storage.
  • More memory? 16GB has been a hard ceiling forever in laptop land. MPG does not expect this to change, which is a pity, since it’s the main thing holding back the MacBook Pro from being a full desktop replacement for power users.
  • Faster CPU? Nothing happening here that has meaningful gains. Better battery life and marginal speed gains are the trend.
  • Fewer ports with a move to USB-C that (ugh) require adapters or different cables for all your proven stuff. No fun at all if you have perfectly good existing peripherals (SSDs, card readers, etc). No pain no gain. Except the gain won’t be there right away (product selection and pricing and compatibility), at least not until the new and cool laptop is well worn.
  • Possible removal of Thunderbolt or dumbing-down to just one port, just to make it thinner (unlikely but Apple likes thin and sleek).
  • Larger screen? That’s a moon shot; the Apple trend is to thinner, smaller, lighter, not larger. But we could see a 4K display in the same size, which would be slick.
  • Force-touch trackpad? This might excite some, and that’s fine for laptops used on the road and so on. But on the desktop, the ergonomics are awful, creating serious risks of bodily injury (overuse stress injuries—hey laugh it up until you actually have a neck or wrist injury)—MPG thus uses a mouse and keyboard and display when not on the road, pretending the MBP is a desktop, so a trackpad becomes irrelevant.

Apple’s strategy seems to be to move down into the thin/light/portable areas. So it will be a nice surprise if some attention is given to the power user who want a laptop over a desktop.

Max Your Mac Pro at OWC...

OS X 10.10.3 Security Updates

Apple OS X 10.10.3 incorporates a variety of security fixes among its other changes. Installing it is a smart move for that reason.

Apple does seem to be regularly fixing the most egregious security bugs. The troublesome part is just how many bugs there are with which an attacker could have gained complete control over the system. But that’s the “game”: for every bug fixed, there are probably a dozen others to be found of similar risk. This isn’t like to change any time soon.

Of particular note are numerous bug fixes to PHP, a programming language used for web sites. Also, fixes to OpenSSL, code signing and WebKit.

Peter G writes:

Unless you use Capture One for processing your RAW files you may update, but the update shows some strange behavior that probably Phase One has to fix.
http://www.phaseone.com/Search/Article.aspx?articleid=1263&languageid=1

MPG: always risks to any update for some software.

Saving Money on Internet Speed: When Less is No Different—Comcast Business Class 100 megabit vs 75 megabit

For business reasons, MPG had been using Comcast 100 X 20 business class internet (100 megabit down, 20 megabit up). Burst speeds have been observed as high as ~120 megabits, or even ~180 megabits when Comcast was testing their 150 megabit service.

But it turns out that with the vast majority of sites, real speeds and latencies run a lot closer to a few megabits a second (e.g., hit a bunch of tabs, and sites take 5-10 seconds to load, even though the total bandwidth needed is trivial, since they are mostly text-based). A whole cornucopia of ways to go slower is involved, along with the hop-to-hop issues across the internet.

Testing speed to bandwidth testing sites is irrelevant; what matters is whether the sites and services one actually uses load appreciably faster with a faster internet service.

Downgrading internet speed might mean the SAME speed

For at least a year, MPG noticed that real-world just-about-everything at 100 megabit was nowhere close to that speed. Plenty of latency on most sites. A site visit by a Comcast technician found all in order, and bandwidth testing to test sites all proved out to rated speed.

So the service was downgraded to 75 X 15 megabit (still business class), the expectation being no perceptible change in performance. Which has proven itself to be true: no detectable difference between 100 megabit and 75 megabit when using the internet over a wide variety of web sites.

Big downloads

But what about those big downloads? Shouldn’t 75 megabit be noticeably slower than 100 megabit?

Today’s OS X 10.10.3 update was a ~2.02 gigabyte download, or about 16.16 gigabits of data, providing a perfect opportunity for the best possible case in favor of a faster internet connection.

Continues below...

Real-world download speeds even from big sites rarely matches nominal service speeds

Observed speed — twice

As the download began, burst speed exceeded 75 megabits for a short time, then settled down, bouncing around between 16 and 60 megabits. Average speed as seen in the graph (blue) shows that real download speed averaged somewhere around 21 megabits—a far cry from 75 and even farther from 100 megabits.

Timing the download portion of the 2.02 GB update on another system (Mac Pro) showed that it took 759 seconds for the download (16.16 gigabits of data), which equates to 21.3 megabits per second (on a 75 megabit business class connection). E.g., the same fast-then-slow behavior.

That combination of seeing initial burst speed exceed the rated speed, and then grossly under-perform seems suspicious at best (e.g., throttling by Comcast). To see it on just one system is one thing; to see the same consistent speed loss on two systems upgraded an hour apart is fishy. Comcast claims no throttling for business class, but these results seem to call that claim into doubt.

Repeating a similar test the next day with a big XCode download, speeds of around 24 Mbps were observed—1/3 of the rated link speed which tests out in excess of its claimed 75 X 15 as shown below. Which is the point: speed test sites are irrelevant to real world performance on most all sites most all of the time. The fact that sometimes full speed happens is all well and good, but it’s not a common occurence in my observations.

Comcast almost certainly gives high priority routing to sites like SpeedTest.net—it would be crazy not to make their service look its best. Sort of like camera lens manufacturers gaming the system with lens designs that test well for typical focusing distances.

Testing 75 X 15 megabit Comcast business class internet at SpeedTest.net — exceeds rated speed for the test duration

Conclusion

Apple download servers are among the fastest and most robust available today.

Explanations for the underperformance might be found, but the point of this post is that it doesn’t matter what the explanation is: what could possibly be the point of paying for 100 megabit if 75 delivers less then 1/3 its claimed speed? This is why MPG downgraded to 75 megabit and seeing no difference, now has zero reservations about doing so.

Remember, this is a 75 megabit business-class link using the best Comcast Docsis router. According to MPG inquiries with Comcast technicians over the past few years (site visits), business class has traffic priority over consumer internet (true or not, MPG is uncertain). A regular consumer (non business) internet link would surely not perform better.

Great Deals Every Day...

Comcast Customer Portal Login

Isn’t it nice when behemoth companies with millions of diverse customers do not bother to test their customer portals?

As shown below, it was not possible to login (multiple attempts on multiple days), even with cookies set to “Always Allow”.

This from the same company that silently truncates passwords that are too long: that one took me quite a while to figure out why every new password I chose would fail when I tried to login again.

Comcast.com customer portal cookie bug

SoftRAID 5.0.7 Delivers Performance Enhancements

SoftRAID 5.0.7 has been released. Details further below.

SoftRAID requires Mac OS X 10.6.8 or later. Support for TRIM commands on SSDs requires 10.7.5 or later. SoftRAID requires a Mac which contains a Core 2 Duo or later processor. It will not work on early Intel Macs with Core Duo processors.

See also:

Working through backup and fault tolerance issues for a professional setup? MPG offers consulting on that and much more.

SoftRAID is included with the OWC Thunderbay 4 RAID-5 Edition.

MPG testing previously showed that RAID-4 can be faster than RAID-5, but MPG has not re-evaluted this finding with SoftRAID version 5.0.7. RAID-5 and RAID-4 offer the same fault tolerance (RAID-4 uses a dedicated parity drive and RAID-5 uses distributed parity).

Three 5TB RAID-5 volumes using four 5TB drives
Three 5TB RAID-5 volumes using four 5TB drives

Fom the SoftRAID 5.0.7 elease notes—

New Features in SoftRAID 5.0.7

  • Support for RAID 4, 5 and 1+0 volumes. Macs can use any SoftRAID volume as a startup volume.
  • Complete support for high resolution displays.
  • Improved volume validation which will optionally fix parity or mirror data during the validation operation.
  • Improved error reporting.
  • Improved the predicted disk failure feature to provide more detailed information about how a disk is failing.
  • Up to 30% improvement in read/write performance for mirror volumes.
  • Numerous bug fixes in the SoftRAID application.
  • A SoftRAID status indicator which is always displayed in the menu bar.
  • Full support for the new kernel security features introduced in Mavericks (Mac OS X 10.9).

Known Bugs in SoftRAID 5.0.7

  • Your Mac may hang at shutdown if your remove a SoftRAID disk while your Mac is running. This bug only occurs if the disk is connected via SATA (or SATA over Thunderbolt) and you are running Mac OS X 10.9 or 10.10. This bug is fixed in Mac OS X 10.10.2.
  • There is a bug in Parallels software which can cause data corruption in your Windows virtual machines. We do not recommend using Parallels at this time.
  • The function in the SoftRAID application which allows converting volumes from one format to another is not yet implemented.

New features in version 5.0.7

  • The driver now automatically uses all disks when reading from from mirror (RAID 1) and RAID 1+0 volumes. The “Use all disks for mirror reads” preference has therefore been removed. This improves the read speed for mirror (RAID 1) volumes by up to 56% on HDDs and up to 86% on SSDs. This also improves the read speed for RAID 1+0 volumes by up to 56% on HDDs and up to 67% on SSDs.
  • The write speed of RAID 1+0 volumes which contain SSDs has been improved by up to 11%.
  • The write speed of stripe (RAID 0) volumes has been improved by up to 13%.
  • Volume validate and rebuild operations now proceed uninterrupted for the first 15 seconds to give feedback to the user that they the operations have started. After 15 seconds, they revert to being regulated by the volume optimization setting.
  • The cache flush code in the SoftRAID driver is changed so that writes to disks are always made in sequential order. Previous builds of the driver would issue out of order writes to disks in response to cache flush operations. This significantly speeds up cache flushes to HDDs (disks with rotating media).
  • SoftRAID can now detect all SSDs even if they are not on SATA buses.
  • The SoftRAID application now displays any SMART errors, predicted failures or i/o errors in small disk tiles. These tiles are used when a user is creating a volume or adding / removing a disk from a volume.
  • SoftRAID now writes the disk label and disk SoftRAID ID or disk serial number into all entries in the SoftRAID.log file.
  • The SoftRAID application now writes entries to the SoftRAID.log file to indicate which disk is added to or removed from a volume.

Bugs fixed in version 5.0.7

  • Fixed a bug in the SoftRAID application which could cause the position of disk tiles to change every second. This would happen when using a hardware RAID box which created multiple LUNs in the same device and reported all devices as “LUN 0.”
  • Fixed a bug in the SoftRAID application which caused volume tiles to not be in the correct order after creating a new volume.
  • Fixed a bug in the SoftRAID driver which caused writes to a volume to fail when using the repair functions in DiskWarrior and DriveGenius.
  • Fixed a bug in the SoftRAID application which prevented volumes from being mounted or unmounted if they were rebuilding or validating.
  • Fixed a bug in the SoftRAID driver which prevented volumes from remounting after they were converted to read-only volumes.
  • Fixed a bug in the SoftRAID application which prevented some SoftRAID volumes from being resized.
  • Fixed a bug in the SoftRAID application which caused mirror volumes to display “missing secondary disks” after a disk was removed from them.
  • Fixed a bug in the SoftRAID application which caused the incorrect disk in a RAID 4 volume to be labelled the parity disk.
  • Fixed a bug in the SoftRAID driver which could cause data corruption when small blocks were written to a degraded RAID 4 volume.
  • Fixed a bug in the SoftRAID application which prevented the sizes displayed in volume tiles from updating correctly if the application preference which determines how many bytes are in 1 KB was changed (1,024 bytes vs 1,000 bytes).
  • Fixed a bug in the SoftRAID driver which affected volume permissions.
  • Fixed a bug in the SoftRAID application which caused it to not allow a user to resume certifying or verifying a disk if a Mac was restarted or shutdown while the operations were in progress.
  • Fixed a bug which caused the incorrect dialog box to appear when deleting a volume which contained open files.
  • Fixed a bug which caused the application to hang if a disk was removed while the SoftRAID application was blinking the disk light.
  • Fixed a bug in the SoftRAID driver which caused a kernel panic if there was a stripe volume with one or more disks missing and the user selected “Remove Missing Disks” from the “Volume” menu in the SoftRAID application.
The Speed You Need...

Apple Core Rot: Runaway Spotlight (mds_stores Process)

If the Mac is running sluggishly with odd delays and pauses, think “low memory” and/or “runaway process” and particularly Apple Core Rot.

  1. In the Utilities folder (cmd-shift-U in Finder), open Activity Monitor.
  2. Choose All Processes and take a look at what’s going on—% CPU, Real Mem, memory pressure, Disk and Network activity. The columns can be sorted by clicking on the column title.

Here, the MPG Mac Pro has 64GB memory. Apple’s Spotlight (mds_stores process) is consuming 54GB of that memory and causing the virtual memory system to resort to compressed memory. The mds_stores process process is also using an entire CPU core (100% CPU usage of one core).

Everything was feeling sluggish, Apple Mail was not working properly, multi-second delays were seen doing normal things, etc: the system was essentially out of memory and this was causing severe performance issues.

The only fix was to reboot; this persisted for more than two hours before I diagnosed it (being somewhat distracted by phone calls and misc).

Exclude backup volumes from Time Machine; don’t backup a backup!
mds_stores (Spotlight) consuming all available memory and 100% CPU

William T writes:

Found your post - I’m having the same problem, and your’s is the only mention of this specific issue that I’ve seen. But reboot doesn’t help.

In my case, the run-away mds_stores only seems to occur if Mail is opened.

MPG: the issue has some set of conditions necessary for it to occur on my system. It has not recurred as yet.

Time Machine Running Slow and Low on Space? Exclude Older Items.

Exclude items from Time Machine backup
Exclude items from Time Machine backup

Apple’s Time Machine* is best used for relatively recent work, due to its hourly/daily/weekly snapshots—it’s not bootable and very slow to restore. MPG utilizes and likes Time Machine (recommended!), but recommends it in addition to full clone backups stored safely elsewhere (not with the machine being backed up).

Excluding Items from Time Machine Backup

Used with full backups, Time Machine doesn’t need a ton of space; here’s one trick to use it without having to have a Big Honkin Drive for All Your Stuff:

Time Machine keeps versions of files and deleted files, so sometimes the best thing to do with Time Machine is to make it forget everything it knows by erasing the Time Machine backup volume (via Disk Utility), so it can start over with a new baseline backup. It’s baffling why this simple feature is not just built into Time Machine (thus baffling its users!), because inevitably TM will run out of space.

As far as the Time Machine volume, a USB3 external drive will do, but the beauty of a 4-bay OWC Thunderbay 4 is that one of the drives can be dedicated for Time Machine use, leaving three other drives for additional purposes.

* Time Machine is the software; Time Capsule is an optional backup hardware device that MPG is lukewarm on.

Exclude backup volumes from Time Machine; don’t backup a backup!
Exclude backup volumes from Time Machine; don’t backup a backup!
Great Deals Every Day...

2013 Mac Pro Flash Drive (SSD) Can Fail

(this is not an “April Fool’s” joke, especially the MacMini part!)

2013 Apple Mac Pro, innards

Chris R had the internal SSD (“flash drive”) fail in his 2013 Mac Pro:

Just a quick update on my 2013 MacPro, I have just collected it from Apple repair and they said the SSD had definitely failed, the rest of the system
was intact and fault free
.

Whilst it was away being repaired I bought a basic 2014 MacMini 2.6ghz to get me by (cheapest modern option to accommodate my OWC Thunderbay) as my old 2 GHz iMac was grinding to a halt with modern raw files.

I was surprised that the MacMini was not as slow as I was expecting—100 Raw Canon 5D Mark III files can be batched in around 9 min in CaptureOne Pro, however, it’s no match for my 6 core 2013 Mac which does the same job in 1min 15 sec!

I also, with your links and recommendation, bought a 240GB 6G OWC SSD along with a Mercury Elite Pro and I’m using that as a back up boot drive now for both the MacPro and MacMini rather than just fit the SSD into the MacMini. It now serves two machines with one drive and with it read/writing at a around 430mb/sec it has certainly made the MacMini twice as snappy! happy days!

All in all, it’s a big thank you to you and MPG for the superb no nonsense articles which have kept me going last week without any loss to any data along with your excellent consultation many years ago and following your MPG back up strategy blog.

MPG: always have a bootable clone backup.

MPG suggests the Mercury Elite Pro Mini USB3, preferably the SSD version, but HDD (hard drive) is fine too.

Note that OWC offers a replacement SSD for the 2013 Mac Pro with up to 2TB capacity (Apple offers at most 1TB).

Hard Drives: Odds of Failure with Time, When to Upgrade/Replace

Have an older Mac, say 3 years or older with hard drives? If those drives have been spinning much of the day for those years, their odds of failure are rising fast.

Drive hours of use (SoftRAID)

Particularly for older Mac Pro systems, having four “spinners” that have been in there for 3+ years means that the likelihood of a failure is an increasing concern (the same applies for external enclosures with drives). Drives used sporadically are probably at much lower risk, provided that the drives have not been spun up/down through many cycles (spin up is a higher stress demand on the drive).

If the drives are in a non fault-tolerant striped RAID-0, failure of just one drive means loss of the volume(s) for that RAID. And even if the RAID is fault-tolerant, e.g., a RAID-1 mirror or RAID-5, getting a matching replacement drive are slim, with new models now having quite different performance characteristics (generally much faster).

It’s not just failure; capacity and performance come to bear; see:

When to upgrade

Several strategies come to mind:

  • Wait for failure: move to a new set of drives when a failure occurs; once one of a batch fails, it’s a “heads up” on the others. Use the older drives as backup drives (if their capacity makes that not too much of a hassle). This approach has the stress of fixing a problem which could happen at a Very Bad Time, e.g., when backups are stale for a few weeks, or a major deadline approaches.
  • Schedule a change: once the drives pass the 3+ year mark, both capacity and performance are now far advanced over the older drives. Move to higher capacity (and possibly fewer drives) with higher performance, evaluating new solutions, such as a Thunderbolt enclosure. This approach is problematic in some ways, in that a new investment into old hardware (e.g. SAS or eSATA) might be the wrong move when a new machine may also be in the cards (with Thunderbolt).
  • Switch when the machine changes: the transition might be from, say, an older Mac Pro to a new Mac Pro. Leave the old stuff alone, treating it as a backup moving to a Thunderbolt enclosure, evaluating new capacity and performance options, interaction with SSD solutions, as well as backup design and discipline.

MPG offers consulting for anyone approaching these or other issues, including partitioning, RAID, backup strategy, etc.

Which drives and enclosure?

For modern Macs, MPG strongly recommends at least 4TB drives, even if that much capacity is deemed overkill; see the articles noted above on speed vs capacity.

As of mid 2015, the the drive of choice at MPG is the Hitachi HGST Deskstar 6TB NAS Hard Drive. The 5TB and 4TB Hitachi and Toshiba drives are also good choices. See hard drives at OWC (the OWC 90-day DOA guarantee is a big plus BTW).

Enclosure choice is a no-brainer: MPG has five OWC Thunderbay units for reasons discussed extensively in previous blog posts.

Where to buy?

A new drive fails within 90 days? No fun at all. OWC will replace it with a new one.

Compare that to most vendor policies: send in the drive to the manufacturer, wait 4-6 weeks, get a refurbished (used) replacement.

OWC 90 day new drive replacement
Up to 960GB of Storage!...

Making an Old Dog of a Laptop Run Like a New One: Wow!

As relevant if not more so here in 2015, anyone using any kind of Mac without an SSD should look at upgrade options that really boost performance. From 2013:

Making an Old Dog of a Laptop Run Like a New One: Wow!

Here in 2015, SSD prices have declined very nicely vs 2013. The OWC Electra 6G series is wallet friendly at just $132 for 240GB, or $235 for a 480GB SSD. The OWC Extreme Pro series offers a bit more performance, but the night and day difference comes from upgrading the hard drive to an SSD.

Note that old-style laptops and MacMini models can accept dual internal drives (SSD or hard drive) via the OWC Data Doubler. If the goal is storage capacity, current model 2.5-inch hard drives offer up to 2TB capacity.

TRIM

With Apple removing TRIM support in the latest OS X versions, the drives recommended above do not need TRIM and never have, which in part is why MPG has been recommending them for years.

MPG likes “set and forget” products which just keep working reliably—some SSDs will slow down as they fill up and become internally fragmented, and this may be not only a speed issue, but a NAND (flash memory) wear issue (longevity).

Hard Drives: Staying the Course with Higher Performance and up to 8TB Capacity

While I’d of course love to have a couple of 8TB Viper SSDs (or even one), the cost is commensurate with the outrageous performance, or so I’m expecting as soon as I can get one to test.

Meanwhile, hard drives (HDD) keep increasing their merit by delivering speeds over 200MB/sec, and capacities of 5TB, 6TB and now a whopping 8TB. The 8TB Ultrastar He8 hard drive offers 1/3 more capacity than 6TB but at about double the price. Still, 8TB drives should drop substantially in price fairly quickly as consumer (non enterprise) models appear.

MPG’s diglloyd swapped to 5TB drives for main storage in multiple OWC Thunderbay 4 units less than a year ago, but done today, it would now be the HGST 6.0 TB Deskstar NAS drive*. See the performance tests; it’s a 'killer' fast drive.

* For the 2009/2010 Mac Pro models, I’ve had two reports that at least some production batches might have warm restart issues (cold bootup is fine), e.g. do a restart and the drives won’t appear, but shut down and start up and they come up fine. I cannot confirm this as a general issue, but it is something to be aware of in those Macs when used *internally* (no reports of issues externally). Possibly it involves a certain production batch, etc—no way to be sure.

Storage design

How to configure storage for backup, reliability and fault tolerance (everyone’s needs can vary) is a subject I often cover during consulting.

The best setup for any system is to:

(1) Utilize an SSD (“flash drive”) for the boot/system/apps volume and for any files that benefit from high performance, such as Adobe Lightroom catalogs, scratch space, video transcoding temp files, databases, etc.

(2) Utilize hard drives for main storage of large files: image files, video footage, music, etc. (But ).

While USB3 drives make good choices for single drives (e.g. for backup), a tangled pile of separate drives can lead lead to frustration and reliability issues of various kinds.

Instead, MPG strongly recommends investing in a high quality 4-bay unit like the OWC Thunderbay 4. Its 4 bays are eminently flexible, allowing drives to be used singly and/or in various RAID flavors, including RAID-5. Nearly all current Macs have Thunderbolt, a wonderful solution compared to past years.

For those desiring a smaller form factor and/or easy SSD support, there is the 4-by Thunderbay 4 Mini, which can be configured to use SSDs and/or HDDs. (2.5-inch drives).

See also Backing up High Capacity Storage: Partitioning for Practical Backup.

* HDD = Hard Disk Drive Spinning magnetic media as compared with an SSD which uses memory chips, no moving parts.

Hard drive performance vs capacity utilization over 4TB partition (4/5/6TB hard drives)
Hard drive performance vs capacity utilization over 4TB partition (4/5/6TB hard drives)
Great Deals Every Day...

iMac: How to Upgrade the Internal Drive

Many users have chosen the iMac, but there have been obstacles to upgrading the internal drive, the main one being the need for custom thermal regulation, a problem which OWC has solved with their upgrade kit.

OWC has do-it-yourself videos and toolkits, or the iMac can be upgraded by a professional installer.

Unlike Prior Generations, Apple iMacs from late 2009 to current 2014 & later utilize digital reporting and even custom firmware for SMC drive temperature reporting.

OWC's Exclusive HDD Kits include our custom digital monitor that 'talks Apple SMC' and maintains proper temperature reporting and Apple Diagnostic compatibility allowing you to upgrade an existing 3.5" HDD with any 3.5" SATA HDD of your choice up to 6.0TB.

MPG suggests the HGST 6.0 TB Deskstar NAS drive for such an upgrade, but there are many other HDD choices.

SSD addition

With certain models, an SSD can be added to the hard drive for dual internal drive operation (hard drive + SSD), which enables Apple’s 'Fusion' technology.

Add an OWC Solid State Drive (sold separately) to your hard drive only equipped iMac to experience faster booting, near instant application launches and data transfer speeds up to 559MB/s!

Which Mac to Get? How to Backup? Planning for Growth.

By engaging me in consulting, I can help in the following ways—

  • Evaluating the backup protocol and the risks to user data.
  • Choosing just the right machine for your needs: CPU speed and cores.
  • Determining the right amount of memory.
  • Assessing whether more advanced solutions (such as multiple solid state drives) can actually help with a performance issue.
  • Deciding on the best solution for performance, reliability and fault-tolerance.
  • Planning for needs 6 months / 1 year / 2 years ahead, not just right now.
  • Evaluating whether a “faster” video card is faster, or just a waste of $200 or more.
  • Workflow, display, peripheral issues.
  • Tips on saving money when buying a machine and its add-ons.

Hours are flexible, and I work with clients all over the world.

Visit Thunderbolt Central...

Apple: A Trillion Dollar Company Dominating Personal Electronics, Cars and (incidentally) Computers

Apple is the most wildly successful company in history. Competition from Apple is a scary thing for any company in its chosen field of play, for good reason—Apple goes “all in” or doesn’t enter the market. In phones, no company makes money any more except Apple. Now the Apple iWatch is imminent. The Apple Car may be next.

Here’s a short but intriguing video (thanks to reader Martin D):

The Four Horsemen: Amazon/Apple/Facebook & Google--Who Wins/Loses (Scott Galloway)

Apple computers sell for premium prices at high profit margins in a cutthroat industry. But OS X and computers as we know them are now being marginalized as Apple products; they just don’t have much impact to Apple’s bottom line. This simple fact explains the slowdown in both computer and OS X development in terms of breadth and depth (and quality of OS X).

Great Deals Every Day...

OS X Security: Various Developments

Various recent security developments.

What to do in general

Apple security fixes

Apple has issued several security updates recently. Keep your Mac up to date with security updates.

Crapware for Macs via Java

For some of my programming needs (server side code and other stuff), I use the Java programming language*.

So it’s disturbing that Oracle now has started bundling crapware with the java SDK. MPG considers the way Oracle is doing this to be unethical. Apparently this was done for years with PCs, but now is doing it with Macs. Beware!!!

Oracle adds crapware to Java for Macs
A close look at how Oracle installs deceptive software with Java updates

As noted previously, do not install any software or add-ons or “free” stuff unless you absolutely positively require it and know exactly what it’s good for.

* Don’t be confused by bad name choice: the Java programming language is NOT related to javascript (used by vast numbers of web sites including this one on web pages). Java in the web browser = BAD (disable it). Javascript = necessary.

Safari browsing history

Don’t assume claimed privacy features actually work.

Years-old Safari private browsing bug saves URL of every page visited, remains unfixed

The Speed You Need...

diglloyd.com | Terms of Use | PRIVACY POLICY
Contact | About Lloyd Chambers | Consulting | Photo Tours
Mailing Lists | RSS Feeds | Twitter | Retina Image Control
Copyright © 2008-2015 diglloyd Inc, all rights reserved.