Restricted Boltzmann Machine (RBM)
M.ZARADZKI - 2016
Loading MNIST data, will take a few seconds ...
Unsupervided learning with bi-partite (visible-hidden) graph
A binary-states Boltzmann Machine is a graph of whose nodes are (on/off) random variables with activation probability, conditionally on their relatives, that are logistic functions.
In a Restricted Boltzmann Machine, the nodes can be split in 2 sets (denoted $V$ and $H$) such that $V$ nodes can only be connected to $H$ nodes, and conversely.
This implies that conditionally on $V$, two nodes from the $H$ set, say $h1$ and $h2$, are independent from each other.
In the context of machine learning, the two sets are denoted $V$ and $H$ as one will encode Visible inputs and the other will encode Hidden features.
Denoting $\sigma$ the logistic function :
$$\sigma\left(x\right)=\frac{1}{1+e^{-x}}$$ Conditionaly on all the visible units, the probability of hidden unit $h_{j}$ being active is :
$$p_{V}\left(h_{j}=1\right)=\sigma\left(b_{j}+\sum_{i=1}^{n}W_{ij}v_{i}\right)$$ Conditionaly on all the hidden units, the probability of visible unit $v_{i}$ being active is :
$$p_{H}\left(v_{i}=1\right)=\sigma\left(a_{i}+\sum_{j=1}^{p}W_{ij}h_{j}\right)$$
In a Restricted Boltzmann Machine, the nodes can be split in 2 sets (denoted $V$ and $H$) such that $V$ nodes can only be connected to $H$ nodes, and conversely.
This implies that conditionally on $V$, two nodes from the $H$ set, say $h1$ and $h2$, are independent from each other.
In the context of machine learning, the two sets are denoted $V$ and $H$ as one will encode Visible inputs and the other will encode Hidden features.
Denoting $\sigma$ the logistic function :
$$\sigma\left(x\right)=\frac{1}{1+e^{-x}}$$ Conditionaly on all the visible units, the probability of hidden unit $h_{j}$ being active is :
$$p_{V}\left(h_{j}=1\right)=\sigma\left(b_{j}+\sum_{i=1}^{n}W_{ij}v_{i}\right)$$ Conditionaly on all the hidden units, the probability of visible unit $v_{i}$ being active is :
$$p_{H}\left(v_{i}=1\right)=\sigma\left(a_{i}+\sum_{j=1}^{p}W_{ij}h_{j}\right)$$