MODELING AND OPTIMIZATION OF DYNAMIC CONTROL IN STOCHASTIC SYSTEMS BASED ON MACHINE LEARNING

Abstract

The problem of optimizing the functioning of the entry group of a container terminal operating under conditions of stochastic uncertainty of transport flows caused by global logistics trends and random external factors is considered. The relevance of the chosen direction is due to the need to reduce truck waiting time, minimize operating costs and level the negative environmental impact from excess emissions during transport downtime in queues. Obviously, a simple expansion of the physical infrastructure is often economically impractical. A transition to an intelligent hybrid dynamic control model based on the reinforcement learning (RL) paradigm is proposed, which allows the system to adaptively regulate the number of active service channels. The developed model is based on a Markov decision-making process. To adequately reproduce the real dynamics of truck arrivals, a Poisson distribution is used. The entry group is represented through a discrete approximation of the classical mass service model. The use of Q-learning ensures finding the optimal control policy even in the absence of exhaustive a priori information, allowing the agent to «learn» directly during interaction with the environment. The study illustrates the evolution of agent learning and confirms its convergence to a theoretically justified optimal strategy. The modeling results show that the implementation of RL methods contributes to effective smoothing of peak loads, a significant reduction in queue length and an overall increase in terminal throughput. The possibilities of scaling the model through the integration of deep neural networks are considered, which allows operating with large data sets and complex state spaces. The Hamilton–Jacobi–Bellman equation, which determines the optimality limits in continuous control problems, is a theoretical verification of the obtained strategies. The proposed approach has practical significance for the development of logistics systems, as it allows integrating hybrid intelligent algorithms into infrastructure management and ensures optimization of economic indicators.