Mathematical Optimization of Machine Learning Models Under Uncertainty: A Robust Framework for Intelligent Decision Systems

Abstract

In intelligent decision systems, the use of machine learning models often comes across with uncertain inputs that may affect the stability of prediction and reliability of decision-making. This study presented a powerful mathematical optimization framework to enhance the performance of machine learning systems when uncertainty is explicitly modelled. The German Credit dataset was chosen for the evaluation of Logistics Regression, Support Vector Machine, Random Forest, Gradient Boosting and Multilayer Perceptron classifiers as it consists of 1000 observations and 20 predictors. Errors in model behavior were evaluated for controlled amounts of uncertainty of 0%, 5%, 10%, 15% and 20%. The average performance of Gradient Boosting was the highest conventional baseline with 78.5% while its worst case relative degradation was 2.55%. However, the degradation was decreased to 1.28% with uncertainty-aware training and there were also robustness improvements for Support Vector Machine and Random Forest. The accuracy of the uncertainty-aware Multilayer Perceptron remained at 77.5-78.5% under all perturbation conditions. The results illustrate that explicit consideration of uncertainty in model optimization can be helpful in improving the worst-case performance and decision stability with respect to more reliable intelligent decision systems, which go beyond  the traditional focus on accuracy in model selection.