Fan Wenshu LAMDA Group School of Artificial Intelligence Nanjing University, Nanjing 210023, China. Email: fanws [at] lamda.nju.edu.cn

Short Bio

• Fan Wenshu received his B.Sc. degree from Northwestern Polytechnical University, China in June 2019.

• After that, he became a first year Ph.D. student in the LAMDA Group led by professor Zhi-Hua Zhou in Nanjing University, under the supervision of Prof. De-Chuan Zhan.

Main Research Interests

Wenshu mainly focuses on machine learning, especially:

Knowledge Distillation

• Knoweldge distillation, which utilizes a well-trained teacher model to assist student model, often accelerates training process or improves model performance. This technology can be also used in model compression and other applications.

Publications - Conference Papers

Wen-Shu Fan, Su Lu, Xin-Chun Li, De-Chuan Zhan, Le Gan. Revisit the Essence of Distilling Knowledge through Calibration. In: Proceedings of the 41st International Conference on Machine Learning (ICML'24), Vienna, Austria, 2024. [Paper] A counter-intuitive phenomenon known as capacity mismatch in knowledge distillation has been identified, wherein KD performance may not be good when a better teacher instructs the student. In this paper, we propose a unifying analytical framework to pinpoint the core of capacity mismatch based on calibration.

Xin-Chun Li , Wen-Shu Fan, Shaoming Song, Yinchuan Li, Bingshuai Li, Yunfeng Shao, De-Chuan Zhan. Asymmetric Temperature Scaling Makes Larger Networks Teach Well Again. In: Advances in Neural Information Processing Systems 35 (NeurIPS'22), New Orleans, Louisiana, USA, 2022. [Paper] We find traditional temperature scaling limits the efficacy of class discriminability and hinders large teachers from teaching well. We propose a method which enriches class discriminability by separately applying a higher/lower temperature to the correct/wrong class.

Academic Service

• Conference PC Member/Reviewer: CCML'21, ACML'22, ICML'23, KDD'23

• Journal Reviewer: IEEE Transactions on Pattern Analysis and Machine Intelligence (IEEE TPAMI)

Teaching Assistant

• Advanced Machine Learning. (For undergraduate and graduate students, Autumn, 2021)

• Discrete Mathematics. (For undergraduate students, Autumn, 2019)

Correspondence

Email: fanws [at] lamda.nju.edu.cn or dz1937001@smail.nju.edu.cn or 1005448166 [at] qq.com
Office: Room 113, Computer Science Building, Xianlin Campus of Nanjing University
Address: Fan Wenshu
                 National Key Laboratory for Novel Software Technology
                 Nanjing University, Xianlin Campus Mailbox 603
                 163 Xianlin Avenue, Qixia District, Nanjing 210046, China