Home / Research /

Split-and-augmented Gibbs sampler

Recently, a new class of Markov chain Monte Carlo (MCMC) algorithms took advantage of convex optimization to build efficient and fast sampling schemes from high-dimensional distributions. Variable splitting methods have become classical in optimization to divide difficult problems into simpler ones and have proven their efficiency in solving high-dimensional inference problems encountered in machine learning and signal processing. This paper derives two new optimization-driven sampling schemes inspired from variable splitting and data augmentation. In particular, the formulation of one of the proposed approaches is closely related to the alternating direction method of multipliers (ADMM) main steps. The proposed framework enables to derive faster and more efficient sampling schemes than the current state-of-the-art methods and can embed the latter. By sampling efficiently the parameter to infer as well as the hyperparameters of the problem, the generated samples can be used to approximate maximum a posteriori (MAP) and minimum mean square error (MMSE) estimators of the parameters to infer. Additionally, the proposed approach brings credibility intervals at a low cost contrary to optimization methods. Simulations on two often-studied signal processing problems illustrate the performance of the two proposed samplers. All results are compared to those obtained by recent state-of-the-art optimization and MCMC algorithms used to solve these problems.
DAG associated with the SPA distribution
Fig. 1. Directed acyclic graph associated with the usual and proposed hierarchical Bayesian models.

The proposed split-and-augmented Gibbs sampler is detailed in the submitted paper:

As an illustration, the proposed split-and-augmented Gibbs sampler has been implemented to conduct sparse Bayesian logistic regression. The results are reported in the paper presented at IEEE Workshop on Machine Learning for Signal Processing (MLSP 2018):

More results are also available here. The corresponding Matlab codes are available on Maxime Vono's GitHub.

sitemeter stats