can sparse all-reduce keep efficiency with large number of gpu workers？

[Eiji911]

in my opinion, when the gpu cluster scaled up to several hundred workers, high sparsification ratios still generate significant communication overheads, which even worst than DenseAllReduce.