Inferring Sparsity: Compressed Sensing using Generalized Restricted Boltzmann Machines

E. W. Tramel, A. Manoel, F. Caltagirone, M. Gabrié, & F. Krzakala

Inferring Sparsity: Compressed Sensing using Generalized Restricted Boltzmann Machines
(Left) CS reconstruction performance over first 1,000 digit images from the MNIST test partition. Results for non-i.i.d. AMP, support-based BRBM-AMP, and GRBM-AMP are on the left, center, and right, respectively. The \(M = K\) oracle support transition is indicated by the black dotted line, and the spinodal transition [Krzakala et al. 2012] by the solid one. Top: Average reconstruction accuracy in MSE measured in dB. Bottom: Average reconstruction correlation with original digit image. (Right) Visual comparison of reconstructions for a single digit image \((\rho = 0.25)\) for small values of \(\alpha\).

Astract

In this work, we consider compressed sensing reconstruction from \(M\) measurements of \(K\)-sparse structured signals which do not possess a writable correlation model. Assuming that a generative statistical model, such as a Boltzmann machine, can be trained in an unsupervised manner on example signals, we demonstrate how this signal model can be used within a Bayesian framework of signal reconstruction. By deriving a message-passing inference for general distribution restricted Boltzmann machines, we are able to integrate these inferred signal models into approximate message passing for compressed sensing reconstruction. Finally, we show for the MNIST dataset that this approach can be very effective, even for \(M < K\).

BibTeX

    article{tdk2016,
    Author = {Eric W. Tramel and Andre Manoel and Francesco Caltagirone and Marylou Gabri{\'e} and Florent Krzakala},
    Journal = {Proc. IEEE Info. Theory Workshop},
    Title = Inferring Sparsity: Compressed Sensing using Generalized Restricted {B}oltzmann Machines},
    Year = {2016}}
    
rss facebook twitter github gitlab youtube mail spotify lastfm instagram linkedin google google-plus pinterest medium googlescholar cv vimeo stackoverflow reddit quora quora