Reinforcement Learning and Policy Optimization for Stock Trading: A Graph-Based State Space Approach to Sequential Decision Making

Abstract

Stock trading is inherently a sequential decision-making problem wherein agents must balance immediate rewards against long-term portfolio growth while managing risk in an uncertain market environment. Traditional approaches to algorithmic trading rely on supervised learning for price prediction combined with heuristic rules for position sizing and risk management, failing to jointly optimize the complete trading pipeline in an end-to-end manner. This paper proposes a novel framework, RL-S3G (Reinforcement Learning Stock State Space Graph), that integrates reinforcement learning policy optimization with the Stock State Space Graph architecture for end-to-end learning of optimal trading policies. Our approach leverages the S3G model introduced by Lu, Hu, and Zhang as the environment representation within a Markov Decision Process framework, enabling an RL agent to learn trading strategies that directly optimize risk-adjusted returns. We adapt explanation-based bias decoupling regularization principles from Zang and Liu's work on natural language inference to improve the robustness of learned policies to market regime changes. Through extensive experiments on benchmark financial datasets, we demonstrate that RL-S3G substantially outperforms both supervised learning baselines and existing reinforcement learning trading agents in terms of Sharpe ratio, maximum drawdown, and risk-adjusted performance. Our work contributes to the growing body of research on deep reinforcement learning in quantitative finance, providing a principled framework for learning adaptive trading policies in complex market environments.