See the Intel white paper: Thunderbolt 3 Technology Brief.
See also: Understanding Thunderbolt 3 Daisy Chaining.
I thought I understood Thunderbolt 3 bandwidth, but it turns out I was way off. Here I’ll capture some critical points from Thunderbolt 3 Technology Brief.
Thunderbolt 3 is often claimed to be 40 Gbps (bidirectional), which it is. However, this is misleading in regards to data transfer speeds for SSDs and hard drives because it lumps together bandwidth which cannot be used for that kind of data transfer, and does not account for the large bandwidth consumption of an external 4K display (or two, or 5K display).
40 Gbps is really 32 Gbps for non-video data
Of the 40 Gbps = 5 GB/sec, 8 Gbps can be used ONLY for video data. Thus the maximum theoretical bandwidth for non-video data is 32 Gbps = 4 GB/sec, because at most 4 X 8 Gbps as PCI Express 3 lanes can be marshalled onto the TB3 bus. And that’s assuming no external 4K or 5K display eating bandwidth.
...the silicon extracts and routes up to 4 lanes of PCI Express Gen 3 (4 x 8 Gbps) and up to two full (4 lane) links of DisplayPort out over the Thunderbolt cable and connector to the device(s) attached downstream from the host system.
Using a 4K display or two or a 5K display throttles data bandwidth
If more than 8Gbps of video data is used, that video bandwidth eats into the available bandwidth for regular data. This isn’t an issue for a 2560X 1600 display which falls into the 8Gbps video-only portion without eating into the regular data bandwidth.
But consider a 4K display, which with encoding uses at least 14 Gbps. That leaves ~26 Gbps = 3.25 GB/sec for regular data—ample for most uses and users.
If it is a 4K display, perhaps 14 Gbps will be consumed depending on the exact pixel count, color depth, and refresh rate.
Next consider dual 4K displays: that eats up at least 28 Gbps and as much as 32 Gbps, leaving only 8 Gbps to 12 Gbps = 1.0 GB/sec to 1.5 GB/sec for regular data. A single 5K display will eat ~22 Gbps, leaving 18 Gpbs = 2.25 GB/sec.
PCI Express devices will continue to function with a variable rate of bandwidth, but certain performance levels may not be achieved if two high resolution displays are being used on the same port.
These figures are for outbound traffic (e.g., writes) and aside from flow control, inbound traffic (reads) are not so-limited. Still, it means that machines with a single Thunderbolt 3 bus like the 2017 iMac 5K are a poor choice for users needing multiple displays and high bandwidth I/O. Users running dual 4K displays or a 5K display should thus go right to the iMac Pro, which has dual Thunderbolt 3 busses.