Approximate Methods For Otherwise Intractable Problems

Recent Monte Carlo methods have expanded the scope of the Bayesian statistical approach. In some situations however, computational methods are often impractically burdensome. We present new methods which reduce this burden and aim to extend the Bayesian toolkit further. This thesis is partitioned into three parts. The first part builds on the Approximate Bayesian Computation (ABC) method. Existing ABC methods often suffer from a local trapping problem which causes inefficient sampling. We present a new ABC framework which overcomes this problem and additionally allows for model selection as a by-product. We demonstrate that this framework conducts ABC inference with an adaptive ABC kernel and extend the framework to specify this kernel in a completely automated way. Furthermore, the ABC part of the thesis also presents a novel methodology for multifidelity ABC. This method constructs a computationally efficient sampler that minimises the approximation error induced by performing early acceptance with a low fidelity model. The second part of the thesis extends the Reversible Jump Monte Carlo method. Reversible Jump methods often suffer from poor mixing. It is possible to construct a “bridge” of intermediate models to facilitate the model transition. However, this scales poorly to big datasets because it requires many evaluations of the model likelihoods. Here we present a new method which greatly improves the scalability at the cost of some approximation error. However, we show that under weak conditions this error is well controlled and convergence is still achieved. The third part of the thesis introduces a multifidelity spatially clustered Gaussian process model. This model enables cheap modelling of nonstationary spatial statistical problems. The model outperforms existing methodology which perform poorly when predicting output at new spatial locations.