Workshops Archive

RL4Sys: A Lightweight System-driven RL Framework for Drop-in Integration in System Optimization

Workshop: PDSW'25: The 10th International Parallel Data Systems Workshop

Authors: Jiaxin Dong and Md. Hasanur Rashid (University of Delaware), Helen Xu (Georgia Institute of Technology), and Dong Dai (University of Delaware)

Abstract: Reinforcement learning (RL) has achieved notable success in complex decision-making tasks. Motivated by these advances, systems researchers have explored RL for optimizing system behavior. However, practical deployment remains uncommon, as existing RL frameworks are ill-suited for system-oriented use cases. To address this gap, we present \textbf{RL4Sys}, a lightweight RL framework designed specifically for seamless system-level integration. RL4Sys includes a minimal client that embeds easily within target systems to record trajectories and run inference from locally cached deep policies. RL4Sys's remote RL trainers executed asynchronously and distributed across servers leverage zero-copy gRPC and adaptive batching to update policies without blocking the original system. Our evaluation shows that RL4Sys matches the convergence behavior of conventional RL frameworks and achieves up to 220% higher throughput in environment-oriented settings compared to state-of-the-art systems such as RLlib, while incurring less than 6% runtime overhead relative to the original non-RL system.

Back to PDSW'25: The 10th International Parallel Data Systems Workshop Archive Listing Back to Full Workshop Archive Listing