In ordinary networking the CPU runs the protocol stack, and data is copied between application and kernel memory. At hundreds of gigabits per second, that consumes a large amount of CPU and memory bandwidth.

RDMA moves that work into the adapter. Once memory is registered and a connection exists, the application posts requests directly and the adapter does the rest. The CPU still pays for the setup, and registering memory is slow, which is why libraries reuse registered regions.

Training traffic is bursty and synchronised: many GPUs send at the same moment, and queues fill up. RoCE’s transport handles loss poorly compared with TCP, so a little loss can cost a lot of throughput.

A job finishes only when its slowest transfer does, so the fabric’s worst moments, not its average, set the job’s speed. This is what fails first when a RoCE fabric is scaled up without careful tuning.

PFC lets a switch tell its neighbour to pause one traffic class when a buffer is nearly full. It is a hop-by-hop safeguard, intended to prevent drops. NVIDIA’s RoCE documentation recommends it.

The trade-off is that pauses spread. A single slow receiver can push back across several switches and hold up traffic that was not even going there, and badly configured networks can deadlock. Treat PFC as a backstop, not a way of running normally.

Instead of dropping a packet, a congested switch marks it. The receiver tells the sender, and the sender slows down. Adapters typically do this in hardware, using an algorithm such as DCQCN.

It acts end to end and early, while PFC acts hop by hop and late. A healthy fabric relies on ECN to slow senders first, so PFC rarely has to act. Getting those thresholds right takes tuning for the actual workload.

MTU (maximum transmission unit) is the largest packet a link will carry. VXLAN adds 50 bytes over IPv4, and RFC 7348 says VTEPs must not fragment VXLAN packets. So the underlay must allow more than the tenant’s largest frame.

This fails in a misleading way: small packets, such as pings and control messages, pass while large ones vanish. The connection looks fine but real traffic stalls. Data-centre fabrics often avoid this by setting jumbo frames well above any tenant size, consistently, on every hop.

NVLink joins a fixed group of GPUs with very high bandwidth per GPU. It is fast and tightly coupled, but it stops at the edge of its domain.

RoCE reaches every server in the cluster, at a small fraction of that bandwidth per GPU. Large jobs use both: NVLink inside the domain, and the network between domains. Schedulers therefore try to keep tightly-coupled work in one domain.

Inside the server the two are the same: GPUDirect RDMA, the same libraries, the same CPU role. The decision is about the network and who runs it. InfiniBand brings flow control and central route management built in. RoCE runs on Ethernet, with its wider choice of suppliers and the tools and skills most network teams already have, but the fabric tuning falls on you.

InfiniBand is its own technology to buy, staff and operate, usually alongside Ethernet for storage and management: NVIDIA’s DGX GB200 SuperPOD design does exactly that. Claims that one is simply faster depend on the workload, scale and tuning, so test with your own traffic before deciding.

A GPU runs an application’s mathematics. A DPU runs infrastructure software: virtual switching, encryption, storage services. Moving work to a DPU frees CPU cores, not GPU cycles.

DPUs add a second operating system per server to patch, monitor and secure. They pay off where isolation has real value, such as a multi-tenant GPU cloud. In a single-owner cluster, they may not be worth it on the compute path.