Classical Model Selection via Simulated Annealing

Stephen P. Brooks, Nial Friel and Ruth King

University of Cambridge

Summary

The classical approach to statistical analysis is usually based upon finding values for model parameters that maximise the likelihood function. Model choice in this context is often also based upon the likelihood function, but with the addition of a penalty term for the number of parameters. Though models may be compared pairwise using likelihood ratio tests for example, various criteria such as the AIC have been proposed as alternatives when multiple models need to be compared. In practical terms, the classical approach to model selection usually involves maximising the likelihood function associated with each competing model and then calculating the corresponding criteria value(s). However, when large numbers of models are possible, this quickly becomes infeasible unless a method that simultaneously maximises over both parameter and model space is available. In this paper we propose an extension to the traditional simulated annealing algorithm that allows for moves that not only change parameter values but that also move between competing models. This trans-dimensional simulated annealing algorithm can therefore be used to locate models and parameters that minimise criteria such as the AIC, but within a single algorithm, removing the need for large numbers of simulations to be run. We discuss the implementation of the trans-dimensional simulated annealing algorithm and use simulation studies to examine their performance in realistically complex modelling situations. We illustrate our ideas with a pedagogic example based upon the analysis of an autoregressive time series and two more detailed examples: one on variable selection for logistic regression and the other on model selection for the analysis of integrated recapture/recovery data.

Keywords:

Autoregressive time series; Capture-recapture; Classical statistics; Information criteria; Logistic Regression; Markov chain Monte Carlo; Optimisation; Reversible jump MCMC; Variable selection.


Appeared as Brooks, S. P., Friel, N. and King, R. (2003) "Classical Model Selection via Simulated Annealing". Royal Statistical Society, Series B 65 pp 503-520.