Approximate Message Passing with Restricted Boltzmann Machine Priors

E. W. Tramel, A. Dremeau, & F. Krzakala

Link
PDF

Approximate Message Passing with Restricted Boltzmann Machine Priors — Visual comparison of reconstructions for four test digits across \(\alpha\) for the same experimental settings. The rows of each box, from top to bottom, correspond to the reconstructions providied by i.i.d. AMP-GB, non-i.i.d. AMP-GB, the proposed approach with NMF RBM factorization, and the proposed approach with TAP RBM factorization, respectively. The columns of each box, from left to right, represent the values \(\alpha = 0.025, 0.074, 0.123, 0.172, 0.222, 0.271, 0.320, 0.369, 0.418, 0.467\). The advantages provided by the proposed approach are clearly seen by comparing the last row to the first one. The digits shown have \(\rho = 0.342\) (top left), \(\rho = 0.268\) (bottom left), \(\rho = 0.214\) (top right), and \(\rho = 0.162\) (bottom right). The vertical blue line represents the \(\alpha = \rho\) oracle exact-reconstruction boundary for each reconstruction task.

Astract

Approximate Message Passing (AMP) has been shown to be an excellent statistical approach to signal inference and compressed sensing problem. The AMP framework provides modularity in the choice of signal prior; here we propose a hierarchical form of the Gauss-Bernouilli prior which utilizes a Restricted Boltzmann Machine (RBM) trained on the signal support to push reconstruction performance beyond that of simple iid priors for signals whose support can be well represented by a trained binary RBM. We present and analyze two methods of RBM factorization and demonstrate how these affect signal reconstruction performance within our proposed algorithm. Finally, using the MNIST handwritten digit dataset, we show experimentally that using an RBM allows AMP to approach oracle-support performance.

BibTeX

    article{tdk2016,
    Author = {Eric W. Tramel and Ang{\'e}lique Dr{\'e}meau and Florent Krzakala},
    Journal = {Journal of Statistical Mechanics: Theory and Experiment},
    Title = {Approximate Message Passing with Restricted {B}oltzmann Machine Priors},
    Year = {2016}}

Approximate Message Passing with Restricted Boltzmann Machine Priors

E. W. Tramel, A. Dremeau, & F. Krzakala

Astract

BibTeX

Related Posts