Ensemble learning is a machine learning paradigm that achieves the state-of-the-art performance. Diversity was believed to be a key to a good performance of an ensemble approach, which, however, previously served only as a heuristic idea. We show that diversity can play the role of regularization.
Papers:
- Diversity regularized machine: In the IJCAI'11 (PDF) paper, we showed that diversity plays a role of regularization as in popular statistical learning approaches.
- Diversity regularized ensemble pruning: In the ECML'12 (PDF) paper, we proved that diversity defined on hypothesis output space plays a role of regularization, and use this principle to prune Bagging classifiers.
Codes: