Investor Preference-Aware Portfolio Optimization with Deep Reinforcement Learning

Abstract

<p>Recently, deep reinforcement learning (RL) approaches have been adopted to optimize financial portfolios with the objective of maximizing total profits while reducing potential risks by spreading investment capital across a variety of assets. Despite achieving great advances in the trade-off between profits and risks, the existing deep RL-based frameworks rarely consider practical trading constraints when making decisions in real-world financial markets, which cannot fulfill the customized requirements of specific users and may violate market regulations. Accordingly, Multi-Agent and Self-Adaptive trading framework for Constrained portfolio optimization, namely the MASAC, is proposed in which the deep RL-based agent dynamically explores profit-maximization policies while the heuristic-based agent conducts min-conflict search to ensure the generated trading strategies satisfying all concerned constraints. Through sharing knowledge within the trading system, the agents of the proposed framework cooperatively produce new portfolios that can maximize overall profits while satisfying all investor requirements throughout the trading period. The experimental results reveal the advantages of the MASAC framework against many state-of-the-art approaches in investment performance and trading constraint satisfaction on the challenging data sets of real-world markets.<br></p>