The Role of Diversity in Ensemble Learning
Ensemble learning is a machine learning paradigm that achieves the state-of-the-art performance. Diversity was believed to be a key to a good performance of an ensemble approach, which, however, previously served only as a heuristic idea. We show that diversity can play the role of regularization.
Papers
:
Diversity regularized machine:
In the
IJCAI'11 (PDF)
paper, we showed that diversity plays a role of regularization as in popular statistical learning approaches.
Diversity regularized ensemble pruning:
In the
ECML'12 (PDF)
paper, we proved that diversity defined on hypothesis output space plays a role of regularization, and use this principle to prune Bagging classifiers.
Codes
:
Diversity-regularized SVM
(codes in Matlab)
Diversity-regularized ensemble pruning
(codes in Matlab)