Scientific and practical basis for implementing machine learning on a parallel computing cluster

The rapid expansion of machine learning (ML) applications in high-performance computing has motivated extensive research on theoretical models and practical challenges in deploying ML on parallel computing clusters. This PRISMA-guided systematic mapping review synthesized 21 peer-reviewed, open-access studies indexed in IEEE Xplore and published between 2023 and 2024. The review examines distributed optimization frameworks, asynchronous training methods, and hybrid parallelism strategies. It also explores practical issues such as data partitioning, synchronization overhead, and load balancing. Reported performance improvements—including communication time reductions of up to 93.15% and throughput gains of up to 530%—indicate promising advances within the experimental settings and baselines used in the individual studies; these values should not be interpreted as directly comparable or universally generalizable across cluster environments. However, many evaluations were simulation-based and assumed homogeneous hardware, suggesting that real-world scalability requires further validation. Because the evidence base was restricted to one database and a recent two-year publication window, the findings should be interpreted as a bounded map of recent evidence rather than an exhaustive review of the full field

Asynchronous communication · Computer cluster · Homogeneous · Parallel processing · Scalability · Throughput · Cloud Computing and Resource Management · Distributed and Parallel Computing Systems · Parallel Computing and Optimization Techniques

• The PRISMA 2020 statement

  Open Access•Matthew J Page, Joanne E McKenzie et al.•Systematic Reviews•2021

  Add to list