torch_simple_timing.utils

Module Contents

Functions

get_world_size()	Returns the number of processes in the current distributed training.
initialized()	Whether or not distributed training is initialized.
synchronize()	Synchronizes:

torch_simple_timing.utils.get_world_size()

Returns the number of processes in the current distributed training. Defaults to 1 in the case of non-distributed training.

Returns:

number of processes

Return type:

int

torch_simple_timing.utils.initialized()

Whether or not distributed training is initialized. False when not initialized or not available.

Returns:

Distributed training is initialized.

Return type:

bool

torch_simple_timing.utils.synchronize()

Synchronizes:

• nothing on CPU

• per-GPU CUDA streams with torch.cuda.synchronize()

• across all processes in distributed training with torch.distributed.barrier()

Return type:

None