See the Intel white paper: Thunderbolt 3 Technology Brief.
See also: Understanding Thunderbolt 3 Daisy Chaining.
I thought I understood Thunderbolt 3 bandwidth, but it turns out I was way off. Here I’ll capture some critical points from Thunderbolt 3 Technology Brief.
Thunderbolt 3 is often claimed to be 40 Gbps (bidirectional), which it is. However, this is misleading in regards to data transfer speeds for SSDs and hard drives because it lumps together bandwidth which cannot be used for that kind of data transfer, and does not account for the large bandwidth consumption of an external display (or two), particularly a 5K or 6K display where the bandwidth requirements are much higher than for a 4K display.
Bandwidth is per bus, *not* per port
The total 40 Gbps bandwidth of Thunderbolt 3 is shared among the two ports on the bus. That is why it is a priority to have at least two Thunderbolt 3 busses on any computer, since one bus serving a high performance SSD along with a high-res display lacks adequate bandwidth for full performance.
40 Gbps is really 32.4 Gbps for non-video data
Of the ~40 Gbps = 5 GB/sec bandwidth of Thunderbolt 3, ~8 Gbps can be used ONLY for video data, leaving a maximum theoretical bandwidth for non-video data of 32.4 Gbps = 4.05 GB/sec.
Thus the maximum theoretical bandwidth for non-video data is 32.4 Gbps = 4.05 GB/sec. However, that data is 8b/10b encoded, so the true *usable* bandwidth drops to 25.92 Gbps ~= 3.24 GB/sec. Good luck with that—MPG has never been able to test any Thunderbolt 3 SSD at faster than about 2.645 GiB/sec= 2.84GB/sec. So much for specifications versus achievable real-world data throughput.
...the silicon extracts and routes up to 4 lanes of PCI Express Gen 3 (4 x 8 Gbps) and up to two full (4 lane) links of DisplayPort out over the Thunderbolt cable and connector to the device(s) attached downstream from the host system.
* At most 4 X 8 Gbps as PCI Express 3 lanes can be marshalled onto the TB3 bus.
Using a 4K display or two or a 5K /6K display throttles data bandwidth
Update,August 2020: the use of Display Stream Compression (DSC) can reduce bandwidth requirements of video by 3X (up to 3:1 compression). The DSC standard is new enough that even top-flight video cards in the 2020 Mac Pro do *not* support DSC.
Figures below are for outbound traffic (e.g., writes) and aside from flow control, inbound traffic (reads) are not reduced significantly (the display only receives data).
The bandwidth losses mean that machines with a single Thunderbolt 3 bus like the 2019 iMac 5K are a poor choice for users needing multiple displays and high bandwidth I/O.
Figures assume 10-bit color (30-bit) at 60 Hz, and account for Thunderbolt 3 transfer efficiencies for video data, which reduce the nominal bandwith required.
Approx bandwidth, 2.5K: ~8 Gbps (2560 X 1600~= 4.1MP)
Approx bandwidth, 4K: ~14 to 15 Gbps (3840 X 2160 ~= 8.3MP or 4096 X 2160 ~= 8.85MP )
Approx bandwidth, 5K: ~22 Gbps (5120 X 2880 ~= 14.7MP)
Approx bandwidth, 6K: ~31 Gbps (6016 X 3384 ~= 20.3MP)
If more than 8Gbps of video bandwidth is used, that video bandwidth eats into the available bandwidth for regular data. This isn’t an issue for a 2.5K display (2560X 1600), because its bandwidth usage falls into the 8Gbps video-only portion thus leaving regular data bandwidth undiminished.
Using a single 4K display per Thunerbolt 3 bus has little impact on data transfer speeds.
Consider a 4K display, which with transport encoding uses around 14 Gbps. That leaves ~26 Gbps = 3.25 GB/sec for regular data—ample for most uses and users.
If it is a 4K display, perhaps 14 Gbps will be consumed depending on the exact pixel count, color depth, and refresh rate.
Next consider dual 4K displays on a single bus: that eats up 28 Gbps and as much as 32 Gbps, leaving only 8 Gbps to 12 Gbps = 1.0 GB/sec to 1.5 GB/sec for regular data. Which is why a computer with dual Thunderbolt 3 busses should be used.
Using a 5K display on a computer with a single Thunderbolt 3 means impaired data transfer speeds but the spee is still ample for most all uses.
A single 5K display will eat ~22 Gbps, leaving 18 Gpbs = 2.25 GB/sec.
PCI Express devices will continue to function with a variable rate of bandwidth, but certain performance levels may not be achieved if two high resolution displays are being used on the same port.
Using a 6K display on a computer with a single Thunderbolt 3 bus guarantees USB speeds for data transfer.