A Semantic-Aware Reinforcement Learning Scheduler for Parameterized Deep Learning Jobs on Heterogeneous Multi-GPU Clusters

Abstract

Efficient scheduling of deep learning workloads in heterogeneous GPU clusters is increasingly challenging due to diverse workload characteristics and stringent memory constraints. Existing approaches often rely on coarse-grained metrics such as runtime or resource demand, which fail to capture workload semantics and lead to resource misallocation, where high-memory GPUs are occupied by low-demand jobs while memory-intensive workloads remain blocked. To address this limitation, we first develop a semantic-aware workload simulator that models deep learning jobs using intrinsic attributes such as model type, dataset size, and training configuration, and estimates execution behavior based on hardware-agnostic computational workloads. This enables more realistic modeling of heterogeneous performance and system-level behaviors. Building on this foundation, we propose SARL, a Semantic-Aware Reinforcement Learning scheduler that incorporates workload semantics and hardware heterogeneity into scheduling decisions. We formulate the scheduling problem as a Markov Decision Process and introduce a model-based GPU allocation mechanism to enable efficient decision making under large action spaces. Experimental results demonstrate that SARL consistently outperforms strong baselines, achieving a 29.6% reduction in deadline miss rate and a 14.4% reduction in job completion time (JCT).