Optimal Bayesian regression with a non-informative prior: Order-recursive and incremental learning
Reihanian S. Zollanvari A. Fazli S.
1 December 2025Elsevier B.V.
Neurocomputing
2025#656
Optimal Bayesian regression (OBR) for data generated from a multidimensional vector autoregressive process of order p, denoted as VAR(p), has a closed-form analytic expression that has been previously obtained. Despite the closed-form expressions to compute the “OBR-VAR”, in certain practical scenarios the computational cost involved in training OBR-VAR is a bottleneck. From a computational perspective, two common scenarios that incur excessive computational cost are: 1) given a set of training data, estimating the unknown model order p generally entails computing the OBR-VAR from scratch for every p in a range from 1 to a maximum value; and 2) in dynamic environments where data arrives sequentially, currently one must recompute OBR-VAR from scratch for every new upcoming observation. To address the first issue, in this paper, an order-recursive OBR-VAR regressor using QR decomposition is proposed. This method efficiently updates the regressor without recalculating it from scratch for each p, significantly reducing computational complexity while preserving model accuracy. Analytical results demonstrate that the proposed order-recursive method achieves a computational complexity reduction by a factor proportional to p, making it scalable to larger datasets and higher model orders. To address the second issue, an incremental version of the OBR-VAR algorithm is developed for real-time data processing. This method updates the regressor incrementally as new data points arrive, maintaining accuracy without the need for costly recomputation of key matrices. Its capability makes it well-suited for continuous-time data acquisition and streaming applications, where timely and accurate responses are critical. In all cases we assume an improper non-informative prior to model the case of having no prior knowledge about the problem. Theoretical analysis and empirical evaluations using synthetic and real datasets demonstrate that both methods significantly outperform the standard OBR-VAR algorithm in terms of computational complexity while preserving accuracy.
Incremental learning , Non-informative prior , Optimal Bayesian regression , Order-recursive , Vector autoregressive process
Text of the article Перейти на текст статьи
Department of Electrical and Computer Engineering, School of Engineering and Digital Sciences, Nazarbayev University, Kabanbay batyr 53, Astana, 010000, Kazakhstan
Department of Electrical and Computer Engineering, Utah Valley University, 800 West University Parkway, Orem, UT 84058, United States
Department of Computer Science, School of Engineering and Digital Sciences, Nazarbayev University, Kabanbay batyr 53, Astana, 010000, Kazakhstan
Department of Electrical and Computer Engineering
Department of Electrical and Computer Engineering
Department of Computer Science
10 лет помогаем публиковать статьи Международный издатель
Книга Публикация научной статьи Волощук 2026 Book Publication of a scientific article 2026