STOCHASTIC OPERATORS OF ACTION IMPACT: FORMALISATION AND MULTI-STEP REGULARISED LEARNING

Abstract

This paper proposes a formalisation of action impact in stochastic dynamical systems as a dedicated stochastic operator acting on system states. Accurate modelling of action impact is an important problem in sequential decision-making under uncertainty, since in many real-world systems actions are applied repeatedly and their consequences propagate through system dynamics over time. While modern machine learning approaches, including reinforcement learning and conditional density estimation, can approximate short-term transitions, the behaviour of learned models under recursive multi-step application remains insufficiently studied. In most existing frameworks, transition dynamics are embedded within policy optimisation or trajectory prediction objectives and are rarely treated as independent modelling entities. In the proposed approach, the action impact operator maps the current system state and applied action to a conditional distribution of future states and is defined with explicit compositional structure. This enables the analysis of recursive operator application across multiple time steps. A learning objective is introduced that combines one-step negative log-likelihood with a multi-step consistency term derived from operator composition. The central hypothesis of the study is that one-step maximum likelihood training does not guarantee stable long-horizon behaviour when the learned operator is recursively applied. To investigate this hypothesis, empirical evaluation is conducted in a fully observable stochastic dynamical system using a minimal realisable linear Gaussian model. The empirical results show that purely one-step training leads to long-horizon degradation, including accumulation of trajectory error and systematic underestimation of predictive uncertainty. Introducing explicit multi-step regularisation significantly improves long-horizon stability and uncertainty calibration, and the improvement persists beyond the training horizon. The proposed formulation establishes a basis for modelling action impact in stochastic dynamical systems and provides a machine-learning framework for robust modelling of recursively applied transitions. This provides a foundation for further research in partially observable environments, nonlinear architectures, and decision-support systems.