# 受限玻尔兹曼机

## 结构

${\displaystyle E(v,h)=-\sum _{i}a_{i}v_{i}-\sum _{j}b_{j}h_{j}-\sum _{i}\sum _{j}h_{j}w_{i,j}v_{i))$

${\displaystyle E(v,h)=-a^{\mathrm {T} }v-b^{\mathrm {T} }h-h^{\mathrm {T} }Wv}$

${\displaystyle P(v,h)={\frac {1}{Z))e^{-E(v,h)))$

${\displaystyle P(v)={\frac {1}{Z))\sum _{h}e^{-E(v,h)))$

${\displaystyle P(v|h)=\prod _{i=1}^{m}P(v_{i}|h)}$.

${\displaystyle P(h|v)=\prod _{j=1}^{n}P(h_{j}|v)}$.

${\displaystyle P(h_{j}=1|v)=\sigma \left(b_{j}+\sum _{i=1}^{m}w_{i,j}v_{i}\right)\,}$${\displaystyle \,P(v_{i}=1|h)=\sigma \left(a_{i}+\sum _{j=1}^{n}w_{i,j}h_{j}\right)}$

## 训练算法

${\displaystyle \arg \max _{W}\prod _{v\in V}P(v)}$

${\displaystyle \arg \max _{W}\mathbb {E} \left[\sum _{v\in V}\log P(v)\right]}$

1. 取一个训练样本v，计算隐层节点的概率，在此基础上从这一概率分布中获取一个隐层节点激活向量的样本h
2. 计算vh外积，称为“正梯度”；
3. h获取一个重构的可见层节点的激活向量样本v'，此后从v'再次获得一个隐层节点的激活向量样本h'
4. 计算v'h'的外积，称为“负梯度”；
5. 使用正梯度和负梯度的差以一定的学习率更新权重${\displaystyle w_{i,j))$${\displaystyle \Delta w_{i,j}=\epsilon (vh^{\mathsf {T))-v'h'^{\mathsf {T)))}$

## 参考资料

1. ^ Smolensky, Paulgengxin. Chapter 6: Information Processing in Dynamical Systems: Foundations of Harmony Theory. (编) Rumelhart, David E.; McLelland, James L. Parallel Distributed Processing: Explorations in the Microstructure of Cognition, Volume 1: Foundations (PDF). MIT Press. 1986: 194–281. ISBN 0-262-68053-X. （原始内容 (PDF)存档于2013-06-13）.
2. ^ G. E. Hinton, R. R. Salakhutdinov. Reducing the Dimensionality of Data with Neural Networks. Science. 2006-07-28, 313 (5786): 504–507 [2018-04-02]. ISSN 0036-8075. doi:10.1126/science.1127647 （英语）.
3. ^ Hugo Larochelle, Yoshua Bengio. Classification using discriminative restricted Boltzmann machines. ACM: 536–543. 2008-07-05 [2018-04-02]. ISBN 9781605582054. doi:10.1145/1390156.1390224.
4. ^ Salakhutdinov, R.; Mnih, A.; Hinton, G. Restricted Boltzmann machines for collaborative filtering. Proceedings of the 24th international conference on Machine learning - ICML '07: 791. 2007. ISBN 9781595937933. doi:10.1145/1273496.1273596.
5. ^ Coates, Adam; Lee, Honglak; Ng, Andrew Y. An analysis of single-layer networks in unsupervised feature learning (PDF). International Conference on Artificial Intelligence and Statistics (AISTATS). 2011. （原始内容 (PDF)存档于2013-05-10）.
6. ^ Ruslan Salakhutdinov and Geoffrey Hinton (2010). Replicated softmax: an undirected topic model. Neural Information Processing Systems 23.
7. Miguel Á. Carreira-Perpiñán and Geoffrey Hinton (2005). On contrastive divergence learning. Artificial Intelligence and Statistics.
8. ^ Hinton, G. Deep belief networks. Scholarpedia. 2009, 4 (5): 5947. Bibcode:2009SchpJ...4.5947H. doi:10.4249/scholarpedia.5947.
9. Geoffrey Hinton (2010). A Practical Guide to Training Restricted Boltzmann Machines. UTML TR 2010–003, University of Toronto.
10. Sutskever, Ilya; Tieleman, Tijmen. On the convergence properties of contrastive divergence (PDF). Proc. 13th Int'l Conf. on AI and Statistics (AISTATS). 2010. （原始内容 (PDF)存档于2015-06-10）.
11. Asja Fischer and Christian Igel. Training Restricted Boltzmann Machines: An Introduction 页面存档备份，存于互联网档案馆. Pattern Recognition 47, pp. 25-39, 2014
12. ^ María Angélica Cueto; Jason Morton; Bernd Sturmfels. Geometry of the restricted Boltzmann machine (PDF). Algebraic Methods in Statistics and Probability (American Mathematical Society). 2010, 516. arXiv:0908.4425.[永久失效链接]
13. ^ Geoffrey E. Hinton. Training Products of Experts by Minimizing Contrastive Divergence. Neural Computation. 2006-03-30, 14 (8): 1771–1800 [2018-04-02]. doi:10.1162/089976602760128018 （英语）.