Multi-Layered Gradient Boosting Decision Trees
J. Feng, Y. Yu, and Z.-H. Zhou.
In:Advances in Neural Information Processing Systems 31 (NIPS'18), Montreal, Canada, 2018.

Multi-layered representation is believed to be the key ingredient of deep neural networks especially in cognitive tasks like computer vision. In this work, we propose the multi-layered GBDT forest (mGBDTs), with an explicit emphasis on exploring the ability to learn hierarchical distributed representations by stacking several layers of regression GBDTs as its building block. The model can be jointly trained by a variant of target propagation across layers, without the need to derive back-propagation nor differentiability.
[code on github]

Distributed Deep Forest and its Application to Automatic Detection of Cash-out Fraud
Y-L Zhang, J. Zhou, W.H. Zheng, J. Feng, L.F Li, Z.Q. Liu, M. Li, Z.Q. Zhang, C.C. Chen, X.L Li, and Z.-H. Zhou.
arXiv preprint: 1805.04234.

This is a joint work with Ant Financial. We implemented a distributed version of deep forest and successfully applied it on the detection of cash-out fraud with more than 100 millions of training samples. An easy-to-use GUI was also deployed inside Ant Financial and Alibaba. Now data scientists can build the deep forest model with only few mouse clicked.

AutoEncoder by Forest
J. Feng and Z.-H. Zhou.
In: Proceedings of the 32nd AAAI Conference on Artificial Intelligence (AAAI'18), New Orleans, Lousiana, USA, 2018.

We proposed the first forest based auto-encoder by enabling tree ensembles to perform autoencoding tasks. The key motivation is that can we extract useful information along decision paths(rather than the information stored in the leaf node)? Our idea is that by traversing from leaf nodes back to the root and caculating the maximum compatible rules, we can then get an amazingly good estimate of the input pattern.
[code on github]

Deep forest: Towards an alternative to deep neural networks.
Z.-H. Zhou and J. Feng.
In: Proceedings of the 26th International Joint Conference on Artificial Intelligence (IJCAI'17), Melbourne, Australia, 2017, pp.3553-3559.

Deep learning models were usually achieved by constructing multi-layered differentiable modules and been trained via back-propagation. In this work, we proposed the first deep model called gcForest via non-differentiable components without the need for backprop. Compared with deep neural networks, gcForest achieves highly competitive performance with much fewer hyper-parameters.
[code on github]

Deep MIML Network.
J. Feng and Z.-H. Zhou.
In: Proceedings of the 31st AAAI Conference on Artificial Intelligence (AAAI'17), San Francisco, CA, 2017,pp.1884-1890.

We proposed an unified DNN framework called DeepMIML network for Multi-Instance Multi-Label learning. By introducing the sub-concept tensor layer, the network is able to explore the instance-label relationships and can automatically detect key instances for a particular label.
[code on github]