RELIABILITY-DRIVEN STREAMING MODELLING OF MULTIMODAL TIME SERIES FOR ROBUST DECISION SUPPORT UNDER DRIFT AND MODALITY DEGRADATION
Abstract
Streaming decision support systems that process multimodal time series must remain robust under simultaneous concept drift and temporary modality degradation. Existing approaches usually treat multimodal fusion, drift adaptation, anomaly detection, and probability calibration as separate problems, which makes it difficult to distinguish whether a loss of predictive quality is caused by a real change in the process or by a temporary failure of one modality. This paper presents a unified online pipeline in which online reliability estimation is used as a single control interface for reliability-adaptive dynamic fusion, driftinitiated budgeted micro-adaptation, and reliability-constrained anomaly detection under a false alarm rate budget. Reliability is modeled as a causal probability of the current non-degraded modality state, post-hoc calibrated, and reused in all downstream control rules. Evaluation follows the prequential protocol on controlled streams with deterministic injections of modality degradation, concept drift, and anomalies, and on real data from UCI Appliances Energy Prediction and UCI Air Quality. The calibrated reliability model retains high degradation-separation ability (ROC - AUC = 0.8624) and improves calibration to ECE = 0.0845 versus 0.1840 without calibration. RADF preserves clean-regime quality (MAE = 0.5557; RMSE = 0.6952) and improves degraded segments, for example MAE = 0.6402 versus 0.6820 for early fusion under alternating missingness. Budgeted micro-adaptation improves post-drift forecasting relative to no adaptation (MAE = 0.6613 versus 0.7046; average recovery 157 versus 800 steps) while updating only a three-parameter head within fixed budgets. RC-AD increases Recall@FAR at all tested budgets, including 0.335 versus 0.103 at a false alarm rate budget of 0.05 on controlled streams. In the integrated stress scenario, the system achieves ROC - AUC = 0.983, ECE = 0.029, Recall@FAR=0.311, and PR - AUC = 0.287; in robust application protocols with severe segment missingness, it reduces MAE by about 35 % in the energy domain and about 93 % in the BTC/USD domain relative to naive fusion. These results present one coherent streaming decision support result rather than isolated local methods.
References
2. Lahat D., Adali T., Jutten C. Multimodal Data Fusion: An Overview of Methods, Challenges, and Prospects // Proceedings of the IEEE. 2015. Vol. 103, no. 9. P. 1449–1477. DOI: 10.1109/JPROC.2015.2460697
3. Hall D. L., Llinas J. An Introduction to Multisensor Data Fusion // Proceedings of the IEEE. 1997. Vol. 85, no. 1. P. 6–23. DOI: 10.1109/5.554205
4. Gama J., Žliobaitė I., Bifet A., Pechenizkiy M., Bouchachia A. A Survey on Concept Drift Adaptation // ACM Computing Surveys. 2014. Vol. 46, no. 4. Art. 44. DOI: 10.1145/2523813
5. Lu J., Liu A., Dong F., Gu F., Gama J., Zhang G. Learning under Concept Drift: A Review // IEEE Transactions on Knowledge and Data Engineering. 2019. Vol. 31, no. 12. P. 2346–2363. DOI: 10.1109/TKDE.2018.2876857
6. Dawid A. P. Present Position and Potential Developments: Some Personal Views: Statistical Theory: The Prequential Approach // Journal of the Royal Statistical Society. Series A (General). 1984. Vol. 147, no. 2. P. 278–292. DOI: 10.2307/2981683
7. Bifet A., Gavaldà R., Holmes G., Pfahringer B. Machine Learning for Data Streams: With Practical Examples in MOA. Cambridge: MIT Press, 2018. 336 p.
8. Rubin D. B. Inference and Missing Data // Biometrika. 1976. Vol. 63, no. 3. P. 581–592. DOI: 10.1093/biomet/63.3.581
9. Guo C., Pleiss G., Sun Y., Weinberger K. Q. On Calibration of Modern Neural Networks // Proceedings of the 34th International Conference on Machine Learning. 2017. P. 1321–1330. URL: https://proceedings.mlr.press/v70/guo17a.html
10. Zadrozny B., Elkan C. Transforming Classifier Scores into Accurate Multiclass Probability Estimates // Proceedings of the Eighth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 2002. P. 694–699. DOI: 10.1145/775047.775151
11. Niculescu-Mizil A., Caruana R. Predicting Good Probabilities with Supervised Learning // Proceedings of the 22nd International Conference on Machine Learning. 2005. P. 625–632. DOI: 10.1145/1102351.1102430
12. Chandola V., Banerjee A., Kumar V. Anomaly Detection: A Survey // ACM Computing Surveys. 2009. Vol. 41, no. 3. Art. 15. DOI: 10.1145/1541880.1541882
13. Fawcett T. An Introduction to ROC Analysis // Pattern Recognition Letters. 2006. Vol. 27, no. 8. P. 861–874. DOI: 10.1016/j.patrec.2005.10.010
14. Hyndman R. J., Koehler A. B. Another Look at Measures of Forecast Accuracy // International Journal of Forecasting. 2006. Vol. 22, no. 4. P. 679–688. DOI: 10.1016/j.ijforecast.2006.03.001
15. UCI Machine Learning Repository. Appliances Energy Prediction. 2017. URL: https://archive.ics.uci.edu/dataset/374/appliances+energy+prediction
16. Candanedo L. M., Feldheim V., Deramaix D. Data Driven Prediction Models of Energy Use of Appliances in a Low-Energy House // Energy and Buildings. 2017. Vol. 140. P. 81–97. DOI: 10.1016/j.enbuild.2017.01.083
17. UCI Machine Learning Repository. Air Quality. 2008. DOI: 10.24432/C5060Z. URL: https://archive.ics.uci.edu/dataset/387/air+quality
18. De Vito S., Massera E., Piga M., Martinotto L., Di Francia G. On Field Calibration of an Electronic Nose for Benzene Estimation in an Urban Pollution Monitoring Scenario // Sensors and Actuators B: Chemical. 2008. Vol. 129, no. 2. P. 750–757. DOI: 10.1016/j.snb.2007.09.060
19. Uzun I. Multimodal DSS: Models and Methods for Multimodal Time-Series Analysis in Intelligent Decision Support Systems – Open-Source Repository. 2026. URL: https://github.com/illiaUzun-pub/PhD-CS-UA
ISSN 
