NETWORK CONGESTION AVOIDANCE OVER RING-BASED COLLECTIVE COMMUNICATION

摘要

In a distributed training, in order to avoid network congestion, a first computing node may determine a routing identifier for routing data from a first process to a second process based at least in part on whether a first network interface controller associated with the first process and a second network interface controller associated with the second process are located in a same computing node or are linked to a same leaf switch, the first process and the second process belonging to a particular inter-node ring that connects a plurality of different nodes under a network topology. The first computing node may then route the data from the first process to the second process according to the routing identifier.