NVMe over Thunderbolt: Killer external storage with caveats 5

NVMe over Thunderbolt: Killer external storage with caveats

Posted Friday, Aug 10, 2018 by Jeff Safire

NVMe in an external Thunderbolt enclosure will give an older Mac a nice performance boost, and a newer one a spectacular one. But compatibility issues exist.

By Jon L. Jacobi

NVMe over Thunderbolt: Killer external storage with caveats 1

It’s all Apple’s fault. I was perfectly happy with my 2011 27-inch Core i7 iMac and its SATA SSD upgrade when I heard that the company had finally started supporting third-party NVMe as of macOS High Sierra. My iMac (old as it is) replaced a Windows PC with an NVMe SSD, and though the iMac is a far superior experience overall, I missed the speed of the NVMe SSD running the operating system. Boot Camp, Logic Pro X…great stuff, but if I could just add NVMe and run macOS from it.

In case you aren’t aware, NVMe is fast—two to six times the sustained throughput of SATA (depending on the drive), with, at times, one-tenth the seek latency. If you’ve used a MacBook Pro from the last couple of years, you’ll understand: Apps pop open, files load and save in and instant, and boot times are ridiculous.

The thing for me is, covering storage for PCWorld has its perks. Just lying around were an Akitio Thunder3 PCIe enclosure, as well as several PCIe NVMe drives left over from reviews. All I needed was a Thunderbolt 3 to Thunderbolt 1/2 adapter for NVMe rapture, right? Hah! After updating to High Sierra, Murphy’s law caught up with me big time.

Don’t do what I did

The first issue was the adapter. Seemed to me that any old Thunderbolt 1/2 to Thunderbolt 3 adapter should work. This is what they’re for, right? Apparently not. As it turns out, only one of the several on the market will do the job, and that’s Apple’s $50 adapter shown below.

The $50 units from Akitio and Startech I tried initially were uni-directional in the wrong direction—they would only handle Thunderbolt 1 or 2 devices attached to a Thunderbolt 3 port. This experiment was the exact opposite, a Thunderbolt 3 device to a Thunderbolt 1 port. The bi-directional Apple adapter does both. It’s what you want.

After the Apple adapter was in place, the next issue was Thunderbolt PCIe enclosure compatibility. The early-to-market models I tried initially—the aforementioned Aktio Thunder3 PCIe and Akitio Node—work fine on Windows PCs, but they use TI’s 65982 communications chip, which predates Apple’s adoption of Thunderbolt 3. Any enclosure using this chip with throw the message you see below. Murphy was laughing.

Finally, Akitio sent a Node Lite which uses TI’s newer 65983 (as does Akitio’s Node Pro) chip that Apple does support, and I was sure that NVMe bliss was nearly upon me. Wrong again. Shut up Murphy!

I should point out that I could’ve avoided all this with the Akitio Thunder2 PCIe, which I later acquired for testing. It’s completely compatible with Thunderbolt and Thunderbolt 2 Macs. The issue here is that many Thunderbolt 2 boxes are reaching end of life. Also, Thunderbolt is the future and performs much better on newer Macs. And there’s only a minor performance penalty with the adapter.

NVMe: Modern implementations required

In my testing for PCWorld’s Akitio Thunder3 PCIe review with a 1.2TB Intel 750 NVMe PCIe SSD inside, the combo read at about 2 GBps and wrote at well over 1 GBps. Nice, but I certainly wasn’t expecting the same results on a Thunderbolt 1 port, with an adapter, and older 2nd generation Intel CPU in play. Neither was I expecting the 350 MBps reading and writing that the Node Lite/Intel 750 combo initially delivered on my iMac. Slower than SATA? Seriously Murphy?

After some head-scratching and other physical manifestations of frustration I won’t discuss, it occurred to me that the Intel 750, being an older enterprise product from the days of NVMe 1.1 (the NVMe spec is now up to 1.3) might need a firmware upgrade. It did. There was one.

Performance jumped to a much more respectable 566 MBps reading, and 644 MBps writing. That’s better than the 529 MBps and 499 MBps my iMac’s internal Samsung 860 Pro delivers, but hardly the 800 MBps to 900 MBps I was hoping for. Seek times were halved, but again, not the 10-times reduction that I’d hoped for.

As it turned out, the 750 is a bit of an outlier. Both the Intel Optane 900P and the Samsung 970 EVO I tried next bumped those numbers up a good 100 MBps. I tried the drives on a slightly newer iMac Core i7-3770, and there was a gain of about 50 MBps. While the price versus result ratio of this upgrade is debatable—the improvement is noticeable, but not earth-shaking—on my iMac, you’ll see at better throughput/value with faster CPUs and chipsets

I also did a quick test on a friend’s 2015 MacBook Air with 20 Gbps Thunderbolt 2, and things really got moving with a healthy 1.2 GBps read/write rate. Thunderbolt 3 proved the panacea, with read/write rates that roughly equal what I’ve seen from the same drives mounted internally on Windows PCs.

Numbers

To flesh out the performance picture I used the two aforementioned enclosures: Akitio’s Thunder2 PCIe (a native Thunderbolt 2 product), and the Thunderbolt 3-based Akitio Node Lite. I tested the three drives I mentioned above in each enclosure: Intel’s 750 PCIe, Intel’s Optane 900P, and Samsung’s far more affordable 970 EVO.

I tested under macOS on the aforementioned Core i7-2600 iMac, a 2016 MacBook Pro Core i7-6700, and a 2015 MacBook Air i7-5650KK using Blackmagic’s DiskSpeed with the drives formatted to HFS+. I tested using AS SSD and CrystalDiskMark under Windows 10 from the same dual-booting Core i7-2600 iMac, as well as PCWorld’s Core i7-5820K storage testbed. I reformatted the drives to NTFS for the Windows 10 tests.

The MacBook Pro and storage testbed are native Thunderbolt 3 and required the Apple adapter when testing the Akitio Thunder2 PCIe. The MacBook Air was Thunderbolt 2, and iMac is native Thunderbolt 1 and required the Apple adapter when using the Node Lite. Below are summarized some of the most pertinent findings.

NVMe over Thunderbolt: Killer external storage with caveats 2
This chart shows the performance of an internal hard drive, an internal SATA SSD, and the best performances of an Intel 900P inside a native Thunderbolt enclosure on the three flavors of Thunderbolt. If your recent Thunderbolt Mac has a hard drive, hybrid drive, or SATA SSD, the performance will astound you.

Aside from a few earlier product that are probably already end-of-life, all the other NVMe SSDs I lightly tested delivered a nice boost. Most were M.2 types mounted on a $25 Adata PCIe adapter. Slower NVMe drives will do almost as well as the faster ones under Thunderbolt 1, but their weaker performance will definitely show up with 40 Gbps Thunderbolt 3 and to a lesser extent, 20 Gbps Thunderbolt 2. Take a look at PCWorld’s SSD coverage for relative performance numbers.

If you’re interested in the specific results—and there were significant differences between the various combos—you can find them in the spreadsheet capture seen below. The spreadsheet also shows the seek results from AS SSD. Note that these were noticeable slower on the iMac than our Windows test bed, for reasons yet to be determined.

NVMe over Thunderbolt: Killer external storage with caveats 3
Here are the test results for all the enclosures, drives, and computers I tried. Just about any x4 PCIe NVMe drive should come close to the Intel 900P’s times.

A couple of notes. As I said earlier, the Apple adapter impacted performance only slightly, and the difference was not apparent to the naked eye. You may also notice that combos tested faster under Windows in terms of throughput. There was also a noticeable difference in seek times between the iMac and our Windows test bed. Whether this is the difference in the hardware, drivers, or the fault of my test methodologies, I was unable to determine at the time of this writing.

Also, the Boot Camp utility crashed sometimes when I tried to create a Windows partition when macOS was run from an external NVMe drive. The solution to that is also under investigation, but Boot Camp won’t let you create the Windows partition on an external drive at this point anyway.

NVMe over Thunderbolt: Killer external storage with caveats 4
The improvements offered by NVMe over Thunderbolt for these Macs is marginal, and probably not worth the money. However, these are top of the line models with already fast SSDs inside. Many Macs might benefit greatly from the upgrade.

In the graphic above, there’s improvement using NVMe over Thunderbolt in most cases, but it might not be enough to justify the expense. Note however, the models listed are top of the line Macs with fast SSDs. If your internal storage is less capable, that’s a different story.

Other vendors that produce Thunderbolt PCIe enclosures are Sonnet, Highpoint, and Startech. Some are for external graphics cards and Macworld hasn’t tested them. Ask the appropriate questions before buying.

The upshot

Upgrading your Mac’s hard drive to an SSD of any type is a transformative performance experience. NVMe over Thunderbolt is faster than an internal SATA SSD, and an easier upgrade, but that 30 to 40 percent bonus and halved seek times will cost you $400. A DIY internal SATA SSD upgrade is around $125. Your choice.

If you already have a fast SSD in your system, then the upgrade to Thunderbolt NVMe isn’t going to rock your world. Everything depends on the vintage and type of storage in your Mac. If you’re not getting the types of numbers you see above, it’s a pretty sweet upgrade. The upgrade to my own iMac wasn’t “gotta have it” though now that it’s up and running, it’s hard to go back. Just sayin’.

This article first appeared at Macworld on August 3, 2018.


No Comments