Home / Research /

Split-and-augmented Gibbs sampler

Recently, a new class of Markov chain Monte Carlo (MCMC) algorithms took advantage of convex optimization to build efficient and fast sampling schemes from high-dimensional distributions. Variable splitting methods have become classical in optimization to divide difficult problems into simpler ones and have proven their efficiency in solving high-dimensional inference problems encountered in machine learning and signal processing. We derive two new optimization-driven sampling schemes inspired from variable splitting and data augmentation. In particular, the formulation of one of the proposed approaches is closely related to the alternating direction method of multipliers (ADMM) main steps. The proposed framework enables to derive faster and more efficient sampling schemes than the current state-of-the-art methods and can embed the latter. By sampling efficiently the parameter to infer as well as the hyperparameters of the problem, the generated samples can be used to approximate maximum a posteriori (MAP) and minimum mean square error (MMSE) estimators of the parameters to infer. Additionally, the proposed approach brings credibility intervals at a low cost contrary to optimization methods. Simulations on two often-studied signal processing problems illustrate the performance of the two proposed samplers. All results are compared to those obtained by recent state-of-the-art optimization and MCMC algorithms used to solve these problems.
DAG associated with the SPA distribution
Fig. 1. Directed acyclic graph associated with the usual and proposed hierarchical Bayesian models.

The proposed split-and-augmented Gibbs sampler is detailed in the paper published in IEEE Trans. Signal Processing:

Asymptotically exact data augmentation

Actually, the SPA Gibbs sampler targets a probability density derived from an asymptotically exact data augmentation (AXDA). This framework is described here and in the paper published in the Journal of Computational and Graphical Statistics:

Application to sparse logistic regression

As an illustration, the proposed split-and-augmented Gibbs sampler has been implemented to conduct sparse Bayesian logistic regression. The results are reported in the paper presented at IEEE Workshop on Machine Learning for Signal Processing (MLSP 2018):

More results are also available here.

Application to image restoration under Poisson noise and log-concave prior

Another instance of the the proposed split-and-augmented Gibbs sampler has been implemented to conduct Bayesian image restoration under Poisson noise with a log-concave prior (e.g., TV regularization or sparse frame-based synthesis regularization). The results are reported in the paper presented at IEEE Int. Conf. Acoust., Speech, and Signal Processing (ICASSP 2019):

sitemeter stats