Zongzhang Zhang @ NJU-AI

Zongzhang Zhang Ph.D., Associate Professor LAMDA Group School of Artificial Intelligence National Key Laboratory for Novel Software Technology Nanjing University, P. R. China Office: Room A503, Yi Fu Building, Xianlin Campus Email: zzzhang@nju.edu.cn, zhangzongzhang@gmail.com

Short Bio

I am now an associate professor at the School of Artificial Intelligence, Nanjing University. I am also a member of the LAMDA group, led by Prof. Zhi-Hua Zhou. From July 2014 to June 2019, I worked as an associate professor at the School of Computer Science and Technology, Soochow University. I received my Ph.D. degree from the School of Computer Science and Technology, University of Science and Technology of China, advised by Prof. Xiaoping Chen, in 2012. I worked with Prof. Mykel J. Kochenderfer as a visiting scholar at the Stanford Intelligent Systems Laboratory (SISL) from September 2018 to March 2019 and worked as a research fellow at the School of Computing, National University of Singapore, from November 2012 to June 2014, under Prof. David Hsu and Prof. Wee Sun Lee. Before that, I visited the Rutgers Laboratory for Real-Life Reinforcement Learning (RL³), directed by Prof. Michael L. Littman, as a research visiting student, from October 2010 to October 2011. I also briefly worked as a research engineer at the Noah's Ark Lab in the Huawei Company in 2012.

[Curriculum Vitae][中文简历]

Research Interests

• Reinforcement learning, including deep reinforcement learning and multi-agent reinforcement learning
• Probabilistic planning, particularly in partially observable Markov decision processes
• Imitation learning based on generative adversarial nets

Projects >>>

Selected Publications

• Surfing Information: The Challenge of Intelligent Decision-Making [PDF]
  Chenyang Wu and Zongzhang Zhang
  Intelligent Computing, 2023, 2: Article 0041.

• Communication-Robust Multi-Agent Learning by Adaptable Auxiliary Multi-Agent Adversary Generation [Online]
  Lei Yuan, Feng Chen, Zongzhang Zhang, and Yang Yu
  Frontiers of Computer Science, 2023.

• Robust Cooperative Multi-agent Reinforcement Learning via Multi-view Message Certification [Online]
  Lei Yuan, Tao Jiang, Lihe Li, Feng Chen, Zongzhang Zhang, and Yang Yu
  SCIENCE CHINA Information Sciences, 2023.

• Multi-Agent Policy Transfer via Task Relationship Modeling [Online]
  Rongjun Qin, Feng Chen, Tonghan Wang, Lei Yuan, Xiaoran Wu, Yipeng Kang, Zongzhang Zhang, Chongjie Zhang, and Yang Yu
  SCIENCE CHINA Information Sciences, 2023.

• Internal Logical Induction for Pixel-Symbolic Reinforcement Learning [PDF]
  Jiacheng Xu, Chao Chen, Fuxiang Zhang, Lei Yuan, Zongzhang Zhang, and Yang Yu
  In: Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD-2023), pages 2825–2837, Long Beach, CA, USA, 2023.

• Policy Regularization with Dataset Constraint for Offline Reinforcement Learning [PDF + Supplementary] [Code]
  Yuhang Ran, Yi-Chen Li, Fuxiang Zhang, Zongzhang Zhang, and Yang Yu
  In: Proceedings of the 40th International Conference on Machine Learning (ICML-2023), pages 28701-28717, Honolulu, Hawaii, USA, 2023.

• Retrosynthetic Planning with Dual Value Networks [PDF + Supplementary]
  Guoqing Liu, Di Xue, Shufang Xie, Yingce Xia, Austin Tripp, Krzysztof Maziarz, Marwin Segler, Tao Qin, Zongzhang Zhang, and Tie-Yan Liu
  In: Proceedings of the 40th International Conference on Machine Learning (ICML-2023), pages 22266-22276, Honolulu, Hawaii, USA, 2023.

• Discovering Generalizable Multi-agent Coordination Skills from Multi-task Offline Data [PDF + Supplementary] [Code]
  Fuxiang Zhang, Chengxing Jia, Yi-Chen Li, Lei Yuan, Yang Yu, and Zongzhang Zhang
  In: Proceedings of the 11th International Conference on Learning Representations (ICLR-2023), Kigali, Rwanda, 2023.

• How To Guide Your Learner: Imitation Learning with Active Adaptive Expert Involvement [PDF] [Supplementary] [Code]
  Xuhui Liu, Feng Xu, Xinyu Zhang, Tianyuan Liu, Shengyi Jiang, Ruifeng Chen, Zongzhang Zhang, and Yang Yu
  In: Proceedings of the 22nd International Conference on Autonomous Agents and Multiagent Systems (AAMAS-2023), pages 1276-1284, London, United Kingdom, 2023.

• Policy-Independent Behavioral Metric-Based Representation for Deep Reinforcement Learning [PDF] [Supplementary]
  Weijian Liao, Zongzhang Zhang, and Yang Yu
  In: Proceedings of the 37th AAAI Conference on Artificial Intelligence (AAAI-2023), pages 8746-8754, Washington, DC, USA, 2023.

• Anti-Drifting Feature Selection via Deep Reinforcement Learning (Student Abstract) [PDF]
  Aoran Wang, Hongyang Yang, Feng Mao, Zongzhang Zhang, Yang Yu, and Xiaoyang Liu
  In: Proceedings of the 37th AAAI Conference on Artificial Intelligence (AAAI-2023), pages 16356-16357, Washington, DC, USA, 2023.     (Best Student Abstract Award - Honorable Mention)

• Bayesian Optimistic Optimization: Optimistic Exploration for Model-based Reinforcement Learning [PDF] [Supplementary]
  Chenyang Wu, Tianci Li, Zongzhang Zhang, and Yang Yu
  In: Advances in Neural Information Processing Systems 35 (NeurIPS-2022), pages 14210-14223, New Orleans, USA, 2022.

• Efficient Multi-agent Communication via Self-supervised Information Aggregation [PDF] [Supplementary] [Code] [Demo]
  Cong Guan, Feng Chen, Lei Yuan, Chenghe Wang, Hao Yin, Zongzhang Zhang, and Yang Yu
  In: Advances in Neural Information Processing Systems 35 (NeurIPS-2022), pages 1020-1033, New Orleans, USA, 2022.

• Multi-agent Dynamic Algorithm Configuration [PDF] [Supplementary] [Code] [Demo]
  Ke Xue, Jiacheng Xu, Lei Yuan, Miqing Li, Chao Qian, Zongzhang Zhang, and Yang Yu
  In: Advances in Neural Information Processing Systems 35 (NeurIPS-2022), pages 20147-20161, New Orleans, USA, 2022.

• Efficient Multi-Agent Communication via Shapley Message Value [PDF] [Code] [Demo]
  Di Xue, Lei Yuan, Zongzhang Zhang, and Yang Yu
  In: Proceedings of the 31st International Joint Conference on Artificial Intelligence (IJCAI-2022), pages 578-584, Vienna, Austria, 2022.

• Multi-Agent Concentrative Coordination with Decentralized Task Representation [PDF] [Code] [Demo]
  Lei Yuan, Chenghe Wang, Jianhao Wang, Fuxiang Zhang, Feng Chen, Cong Guan, Zongzhang Zhang, Chongjie Zhang, and Yang Yu
  In: Proceedings of the 31st International Joint Conference on Artificial Intelligence (IJCAI-2022), pages 599-605, Vienna, Austria, 2022.

• Multi-Agent Incentive Communication via Decentralized Teammate Modeling [PDF] [Code] [Demo]
  Lei Yuan, Jianhao Wang, Fuxiang Zhang, Chenghe Wang, Zongzhang Zhang, Yang Yu, and Chongjie Zhang
  In: Proceedings of the 36th AAAI Conference on Artificial Intelligence (AAAI-2022), pages 9466-9474, Virtual Conference, 2022.

• Adapt to Environment Sudden Changes by Learning a Context Sensitive Policy [PDF] [Code]
  Fan-Ming Luo, Shengyi Jiang, Yang Yu, Zongzhang Zhang, and Yi-Feng Zhang
  In: Proceedings of the 36th AAAI Conference on Artificial Intelligence (AAAI-2022), pages 7637-7646, Virtual Conference, 2022.

• Adaptive Online Packing-guided Search for POMDPs [PDF] [Supplementary] [Code]
  Chenyang Wu, Guoyu Yang, Zongzhang Zhang, Yang Yu, Dong Li, Wulong Liu, and Jianye Hao
  In: Advances in Neural Information Processing Systems 34 (NeurIPS-2021), pages 28419-28430, Virtual Conference, 2021.

• Cross-Modal Domain Adaptation for Cost-Efficient Visual Reinforcement Learning [PDF] [Supplementary] [Code]
  Xiong-Hui Chen, Shengyi Jiang, Feng Xu, Zongzhang Zhang, and Yang Yu
  In: Advances in Neural Information Processing Systems 34 (NeurIPS-2021), pages 12520-12532, Virtual Conference, 2021.

• Efficient Policy Detecting and Reusing for Non-Stationarity in Markov Games [PDF]
  Yan Zheng, Jianye Hao, Zongzhang Zhang, Zhaopeng Meng, Tianpei Yang, Yanran Li, and Changjie Fan
  Autonomous Agents and Multi-Agent Systems, 2021, 35(2): 1-29.

• Triple-GAIL: A Multi-Modal Imitation Learning Framework with Generative Adversarial Nets [PDF]
  Cong Fei, Bing Wang, Yuzheng Zhuang, Zongzhang Zhang, Jianye Hao, Hongbo Zhang, Xuewu Ji, and Wulong Liu
  In: Proceedings of the 29th International Joint Conference on Artificial Intelligence (IJCAI-2020), pages 2929-2935, Yokohama, Japan, 2020.

• Efficient Deep Reinforcement Learning via Adaptive Policy Transfer [PDF]
  Tianpei Yang, Jianye Hao, Zhaopeng Meng, Zongzhang Zhang, Weixun Wang, Yujing Hu, Yingfeng Chen, Changjie Fan, Wulong Liu, Zhaodong Wang, and Jiajie Peng
  In: Proceedings of the 29th International Joint Conference on Artificial Intelligence (IJCAI-2020), pages 3094-3100, Yokohama, Japan, 2020.
  

• Efficient Multiagent Policy Optimization Based on Weighted Estimators in Stochastic Environments [PDF]
  Yan Zheng, Jianye Hao, Zongzhang Zhang, Zhaopeng Meng, and Xiaotian Hao
  Journal of Computer Science and Technology, 2020, 35(2): 268-280.

• A Survey of Imitation Learning Based on Generative Adversarial Nets [PDF]
  Jiahao Lin, Zongzhang Zhang, Chong Jiang, and Jianye Hao
  Chinese Journal of Computers, 2020, 43(2): 326-351.

• Monte-Carlo Tree Search for Policy Optimization [PDF]
  Xiaobai Ma, Katherine R. Driggs-Campbell, Zongzhang Zhang, and Mykel J. Kochenderfer
  In: Proceedings of the 28th International Joint Conference on Artificial Intelligence (IJCAI-2019), pages 3116-3122, Macao, China, 2019.

• Efficient Reinforcement Learning in Continuous State and Action Spaces with Dyna and Policy Approximation [PDF]
  Shan Zhong, Quan Liu, Zongzhang Zhang, and Qiming Fu
  Frontiers of Computer Science, 2019, 13(1): 106-126.

• A Deep Bayesian Policy Reuse Approach Against Non-Stationary Agents [PDF] [Supplementary]
  Yan Zheng, Zhaopeng Meng, Jianye Hao, Zongzhang Zhang, Tianpei Yang, and Changjie Fan
  In: Advances in Neural Information Processing Systems 31 (NeurIPS-2018), pages 960-970, Montreal, Canada, 2018.

• A Survey on Deep Reinforcement Learning [PDF]
  Quan Liu, Jianwei Zhai, Zongzhang Zhang, Shan Zhong, Qian Zhou, Peng Zhang, and Jin Xu
  Chinese Journal of Computers, 2018, 41(1): 1-27.     (2017-2021's Best Paper Award)

• Weighted Double Q-learning [PDF]
  Zongzhang Zhang, Zhiyuan Pan, and Mykel J. Kochenderfer
  In: Proceedings of the 26th International Joint Conference on Artificial Intelligence (IJCAI-2017), pages 3455-3461, Melbourne, Australia, 2017.

• Reasoning and Predicting POMDP Planning Complexity via Covering Numbers [PDF]
  Zongzhang Zhang, Qiming Fu, Xiaofang Zhang, and Quan Liu
  Frontiers of Computer Science, 2016, 10(4): 726-740.

• PLEASE: Palm Leaf Search for POMDPs with Large Observation Spaces [PDF]
  Zongzhang Zhang, David Hsu, Wee Sun Lee, Zhan Wei Lim, and Aijun Bai
  In: Proceedings of the 25th International Conference on Automated Planning and Scheduling (ICAPS-2015), pages 249-257, Jerusalem. Israel, 2015.

• Covering Number for Efficient Heuristic-Based POMDP Planning [PDF] [Supplementary]
  Zongzhang Zhang, David Hsu, and Wee Sun Lee
  In: Proceedings of the 31st International Conference on Machine Learning (ICML-2014), pages 28-36, Beijing, China, 2014.

• Thompson Sampling based Monte-Carlo Planning in POMDPs [PDF]
  Aijun Bai, Feng Wu, Zongzhang Zhang, and Xiaoping Chen
  In: Proceedings of the 24th International Conference on Automated Planning and Scheduling (ICAPS-2014), pages 28-36, Portsmouth, USA, 2014.

• Covering Number as a Complexity Measure for POMDP Planning and Learning [PDF]
  Zongzhang Zhang, Michael L. Littman, and Xiaoping Chen
  In: Proceedings of the 26th AAAI Conference on Artificial Intelligence (AAAI-2012), pages 1853-1859, Toronto, Ontario, Canada, 2012.

• FHHOP: A Factored Hybrid Heuristic Online Planning Algorithm for Large POMDPs [PDF]
  Zongzhang Zhang and Xiaoping Chen
  In: Proceedings of the 28th Conference on Uncertainty in Artificial Intelligence (UAI-2012), pages 934-943, Catalina Island, CA, USA, 2012.

Professional Services

• Editorial Board Member: Intelligent Computing (AAAS/Science Partner Journal, 2022-2024)
• Young Associate Editor: Frontiers of Computer Science (2019-2025)
• Senior Program Committee Member: AAMAS 2024; IJCAI 2020-2021; AAAI 2019; ICAPS 2021; ECAI 2020
• Member of the Novel Program Committee Board: IJCAI 2022-2024
• Program Committee Member/Reviewer: AAAI 2018, 2020, 2022-2023; ICML 2019-2023; IJCAI 2013, 2017-2019; NeurIPS 2018-2023; ICLR 2021-2022; AISTATS 2022; DAI 2019-2022; AAMAS 2021; ICAPS 2020; ECML-PKDD 2020; CoRL 2020; IJCNN 2020; CCDM 2020; ACML 2017-2019; PRICAI 2018-2019; ICA 2017-2019; ADPRL 2018; SSCI 2019; CCFAI 2019
• Journal Reviewer: Transactions on Pattern Analysis and Machine Intelligence; SCIENCE CHINA Information Sciences; Journal of Artificial Intelligence Research; IEEE Transactions on Neural Networks and Learning Systems; IEEE Transactions on Cybernetics; IEEE Transactions on Transportation Electrification; ACM Transactions on Intelligent Systems and Technology; Machine Learning; Pattern Recognition; IEEE Computational Intelligence Magazine; Robotics and Autonomous Systems; Information Sciences; Frontiers of Computer Science; Neurocomputing; Knowledge-Based Systems; Applied Intelligence; Expert Systems with Applications; Chinese Journal of Computers; Journal of Software; IEEE/CAA Journal of Automatica Sinica; Journal of Computer Research and Development
• Workshop Co-chair: Asian Workshop on Reinforcement Learning (AWRL) 2016-2018, PRICAI 2018's Workshop on Methods and Applications of Reinforcement Learning
• Local Organizing Committee Chair: DAI 2020; MLA 2020, 2022
• Professional Organization Membership: CCF Senior Member; AAAI Member; IEEE Member
• Reviewer Award: ICLR 2021's Outstanding Reviewer; NeurIPS 2019, 2022's Top Reviewer
• Consultant/Visiting Scholar: Polixir Technologies; Alibaba Group (2021-2022); Netease (2017-2020)

Teaching

• Control Theory and Methods (for undergraduate and graduate students, Fall 2020-2023) [textbook]
• Multi-Agent Systems (for undergraduate students, Spring 2021-2023) [textbook]
• Reinforcement Learning (for undergraduate and graduate students, Fall 2020-2023, with Prof. Yang Yu) [textbook][course home]
• Introduction to Artificial Intelligence (for undergraduate students, Fall 2021-2023, with Prof. Yang Yu) [textbook][course home]
• Intelligent Systems: Design and Application (for undergraduate and graduate students, Spring 2020-2021) [textbook]
• Intelligent Application Modeling (for undergraduate students, July 2019) [a summer course co-constructed with Tencent]

Students

Ph.D. Students:

• 2020 - : Feng Xu 徐峰, Weijian Liao 廖沩健 (co-supervised with Prof. Ming Li)
• 2022 - : Aoran Wang 王傲然 (co-supervised with Prof. Yang Yu)
• 2023 - : Chenyang Wu 吴晨阳, Di Xue 薛迪

Master Students:

• 2021 - : Fuxiang Zhang 张福翔, Chenghe Wang 王铖鹤, Tianchi Li 李天赐, Yichen Li 李逸尘, Yuhang Ran 冉雨杭, Jiacheng Xu 徐嘉诚
• 2022 - : Feng Chen 陈烽, Hao Ding 丁豪, Chenxiao Gao 高辰潇, Fuguang Han 韩馥光, Rui Kong 孔锐, Renzhe Zhou 周韧哲, Chao Chen 陈超, Xinyu Zhang 张鑫钰
• 2023 - : Ningjing Chao 巢凝静, Tao Jiang 江涛, Tianyuan Liu 刘恬远, Tianyi Zhang 张天一, Wenjie Qiu 邱文杰, Yujian Zhu 祝宇健

To prospective students:

I am in a LAMDA's reinforcement learning team with Prof. Yang Yu.

I am looking for self-driven, diligent, adaptable and resourceful students to work on exciting research in machine learning, including topics of reinforcement learning, probabilistic planning, imitation learning, multi-agent learning, etc. If you are passionate about research, you are welcome to contact me.