search for




 

Discriminative restricted Boltzmann machine using average contrastive divergence algorithm
Journal of the Korean Data & Information Science Society 2018;29:229-40
Published online January 31, 2018
© 2018 Korean Data and Information Science Society.

Jooyong Shim1 · Won-Yong Shin2 · Changha Hwang3

1Department of Statistics, Inje University
2Department of Mobile Systems Engineering, Dankook University
3Department of Applied Statistics, Dankook University
Correspondence to: Professor, Department of Applied Statistics, Dankook University, Yongin, Gyeonggido 16890, Korea. E-mail: chwang@dankook.ac.kr
Received September 11, 2017; Revised September 28, 2017; Accepted October 11, 2017.
This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/3.0) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.
Abstract
Restricted Boltzmann machine (RBM) is a generative model that can learn probability distributions over input vectors. It has been mainly used to extract features for other learning algorithms or to obtain initial values of weights of deep neural networks. However, a discriminative RBM has been recently devised by adding a label layer to ordinary RBM. Contrastive divergence (CD) algorithm is mainly used for training discriminative RBM. In this paper, we propose an average CD algorithm for discriminative RBM and show that classification performance can be improved. The proposed method is evaluated through numerical studies based on real data sets available from UCI machine learning repository and MNIST database.
Keywords : Average contrastive divergence, backpropagation algorithm, contrastive divergence, discriminative restricted Boltzmann machine, restricted Boltzmann machine.