Discovery and inference beyond linearity by integrating Bayesian regression, tree ensembles and Shapley values

Abstract

Machine Learning (ML) is gaining popularity for hypothesis-free discovery of risk and protective factors in healthcare studies. ML is strong at discovering nonlinearities and interactions, but this power is compromised by a lack of reliable inference. Although Shapley values provide local measures of features' effects, valid uncertainty quantification for these effects is typically lacking, thus precluding statistical inference. We propose RuleSHAP, a framework that addresses this limitation by combining a dedicated Bayesian sparse regression model with a new tree-based rule generator and Shapley value attribution. RuleSHAP provides detection of nonlinear and interaction effects with uncertainty quantification at the individual level. We derive an efficient formula for computing marginal Shapley values within this framework. We demonstrate the validity of our framework on simulated data. Finally, we apply RuleSHAP to data from an epidemiological cohort to detect and infer several effects for high cholesterol and blood pressure, such as nonlinear interaction effects between features like age, sex, ethnicity, BMI and glucose level.