Applied Bayesian Modelling

Peter Congdon is Research Professor of Quantitative Geography and Health Statistics at Queen Mary University of London. He has written three earlier books on Bayesian modelling and data analysis techniques with Wiley, and has a wide range of publications in statistical methodology and in application areas. His current interests include applications to spatial and survey data relating to health status and health service research.

This book provides an accessible approach to Bayesian computing and data analysis, with an emphasis on the interpretation of real data sets. Following in the tradition of the successful first edition, this book aims to make a wide range of statistical modeling applications accessible using tested code that can be readily adapted to the reader s own applications. The second edition has been thoroughly reworked and updated to take account of advances in the field. A new set of worked examples is included. The novel aspect of the first edition was the coverage of statistical modeling using WinBUGS and OPENBUGS. This feature continues in the new edition along with examples using R to broaden appeal and for completeness of coverage.

This book provides an accessible approach to Bayesian computing and data analysis, with an emphasis on the interpretation of real data sets.

Preface xi
 
1 Bayesian methods and Bayesian estimation 1
 
1.1 Introduction 1
 
1.1.1 Summarising existing knowledge: Prior densities for parameters 2
 
1.1.2 Updating information: Prior, likelihood and posterior densities 3
 
1.1.3 Predictions and assessment 5
 
1.1.4 Sampling parameters 6
 
1.2 MCMC techniques: The Metropolis-Hastings algorithm 7
 
1.2.1 Gibbs sampling 8
 
1.2.2 Other MCMC algorithms 9
 
1.2.3 INLA approximations 10
 
1.3 Software for MCMC: BUGS, JAGS and R-INLA 11
 
1.4 Monitoring MCMC chains and assessing convergence 19
 
1.4.1 Convergence diagnostics 20
 
1.4.2 Model identifiability 21
 
1.5 Model assessment 23
 
1.5.1 Sensitivity to priors 23
 
1.5.2 Model checks 24
 
1.5.3 Model choice 25
 
References 28
 
2 Hierarchical models for related units 34
 
2.1 Introduction: Smoothing to the hyper population 34
 
2.2 Approaches to model assessment: Penalised fit criteria, marginal likelihood and predictive methods 35
 
2.2.1 Penalised fit criteria 36
 
2.2.2 Formal model selection using marginal likelihoods 37
 
2.2.3 Estimating model probabilities or marginal likelihoods in practice 38
 
2.2.4 Approximating the posterior density 40
 
2.2.5 Model averaging from MCMC samples 42
 
2.2.6 Predictive criteria for model checking and selection: Cross-validation 46
 
2.2.7 Predictive checks and model choice using complete data replicate sampling 50
 
2.3 Ensemble estimates: Poisson-gamma and Beta-binomial hierarchical models 53
 
2.3.1 Hierarchical mixtures for poisson and binomial data 54
 
2.4 Hierarchical smoothing methods for continuous data 61
 
2.4.1 Priors on hyperparameters 62
 
2.4.2 Relaxing normality assumptions 63
 
2.4.3 Multivariate borrowing of strength 65
 
2.5 Discrete mixtures and dirichlet processes 69
 
2.5.1 Finite mixture models 69
 
2.5.2 Dirichlet process priors 72
 
2.6 General additive and histogram smoothing priors 78
 
2.6.1 Smoothness priors 79
 
2.6.2 Histogram smoothing 80
 
Exercises 83
 
Notes 86
 
References 89
 
3 Regression techniques 97
 
3.1 Introduction: Bayesian regression 97
 
3.2 Normal linear regression 98
 
3.2.1 Linear regression model checking 99
 
3.3 Simple generalized linear models: Binomial, binary and Poisson regression 102
 
3.3.1 Binary and binomial regression 102
 
3.3.2 Poisson regression 105
 
3.4 Augmented data regression 107
 
3.5 Predictor subset choice 110
 
3.5.1 The g-prior approach 114
 
3.5.2 Hierarchical lasso prior methods 116
 
3.6 Multinomial, nested and ordinal regression 126
 
3.6.1 Nested logit specification 128
 
3.6.2 Ordinal outcomes 130
 
Exercises 136
 
Notes 138
 
References 144
 
4 More advanced regression techniques 149
 
4.1 Introduction 149
 
4.2 Departures from linear model assumptions and robust alternatives 149
 
4.3 Regression for overdispersed discrete outcomes 154
 
4.3.1 Excess zeroes 157
 
4.4 Link selection 160
 
4.5 Discrete mixture regressions for regression and outlier status 161
 
4.5.1 Outlier accommodation 163
 
4.6 Modelling non-linear regression effects 167
 
4.6.1 Smoothness priors for non-linear regression 167
 
4.6.2 Spline regression and other basis functions 169
 
4.6.3 Priors on basis coefficients 171
 
4.7 Quantile