李逸尘 Yi-Chen Li Ph.D. Student, LAMDA Group School of Artificial Intelligence National Key Laboratory for Novel Software Technology Nanjing University, Nanjing 210023, China Email: liyc@lamda.nju.edu.cn Laboratory: Room A201, Shaoyifu Building, Nanjing University Xianlin Campus

Biography

• Currently I am a Ph.D. student of School of Artificial Intelligence in Nanjing University, advised by Professor Yang Yu and Associate Professor Zongzhang Zhang. I am a member of LAMDA Group, led by professor Zhi-Hua Zhou.
• I received my B.Eng. degree in Automation from School of Management and Engineering, Nanjing University in June 2021. In the same year, I was admitted to study for a M.Sc. degree in Computer Science and Technology.

Research Interests

I am interested in theoretically justified algorithms and real-world applications of Reinforcement Learning (RL). When RL ventures beyond game environments, numerous challenging problems may arise, such as the absence of suitable reward functions, the lack of high-fidelity simulators, and the continually evolving environments, to name a few. To deal with the above issues, I am currenly working on

• Reward Function Learning (incl. Adversarial Imitation Learning, RLHF, etc.)

• Offline RL and Sim2Real Transfer

• Decision Making in Non-stationary Environments

Inspired by the impressive success of ChatGPT, I am also interested in decision making via Large Language Models (LLMs). Please feel free to drop me an email if you would like to discuss or collaborate with me.

Publications

Preprints

Yi-Chen Li*, Fuxiang Zhang*, Wenjie Qiu, Lei Yuan, Chengxing Jia, Zongzhang Zhang, Yang Yu, Bo An. Q-Adapter: Customizing Pre-trained LLMs to New Preferences with Forgetting Mitigation. arXiv preprint arXiv: 2407.03856, 2024. [PDF] [Code]

Conference Papers

2024

Ruiqi Xue, Ziqian Zhang, Lihe Li, Feng Chen, Yi-Chen Li, Yang Yu, Lei Yuan. Dynamics Adaptive Safe Reinforcement Learning with a Misspecified Simulator. In: Proceedings of the 35th European Conference on Machine Learning (ECML 2024), Vilnius, Lithuania, 2024. [PDF] [Code]

Xiong-Hui Chen*, Junyin Ye*, Hang Zhao*, Yi-Chen Li, Haoran Shi, Yu-Yan Xu, Zhihao Ye, Si-Hang Yang, Yang Yu, Anqi Huang, Kai Xu, Zongzhang Zhang. Deep Demonstration Tracing: Learning Generalizable Imitator for Runtime One-Shot Imitation. In: Proceedings of the 41th International Conference on Machine Learning (ICML 2024), Vienna, Austria, 2024. [PDF] [Code]

Xinyu Zhang*, Wenjie Qiu*, Yi-Chen Li*, Lei Yuan, Chengxing Jia, Zongzhang Zhang, Yang Yu. Debiased Offline Representation Learning for Fast Online Adaptation in Non-stationary Dynamics. In: Proceedings of the 41th International Conference on Machine Learning (ICML 2024), Vienna, Austria, 2024. [PDF] [Code]

Lihe Li, Ruotong Chen, Ziqian Zhang, Zhichao Wu, Yi-Chen Li, Cong Guan, Yang Yu, and Lei Yuan. Continual Multi-Objective Reinforcement Learning via Reward Model Rehearsal. In: Proceedings of the 33rd International Joint Conference on Artificial Intelligence (IJCAI 24), Jeju, South Korea, 2024. [PDF] [Code]

Cong Guan, Lichao Zhang, Chunpeng Fan, Yi-Chen Li, Feng Chen, Lihe Li, Yunjia Tian, Lei Yuan, and Yang Yu. Efficient Human-AI Coordination via Preparatory Language-based Convention. In: ICLR 2024 Workshop on LLM Agents, Vienna, Austria. [PDF]

Chengxing Jia*, Fuxiang Zhang*, Yi-Chen Li, Chenxiao Gao, Xu-Hui Liu, Lei Yuan, Zongzhang Zhang, Yang Yu. Disentangling Policy from Offline Task Representation Learning via Adversarial Data Augmentation. In: Proceedings of the 23rd International Conference on Autonomous Agents and Multi-Agent Systems (AAMAS 2024), Auckland, New Zealand, 2024. [PDF] [Code]

Cong Guan, Ruiqi Xue, Ziqian Zhang, Lihe Li, Yi-Chen Li, Lei Yuan, Yang Yu. Cost-aware Offline Safe Meta Reinforcement Learning with Robust In-distribution Online Task Adaptation. In: Proceedings of the 23rd International Conference on Autonomous Agents and Multi-Agent Systems (AAMAS 2024), Auckland, New Zealand, 2024. [PDF] [Code]

2023

Yuhang Ran*, Yi-Chen Li*, Fuxiang Zhang, Zongzhang Zhang, Yang Yu. Policy Regularization with Dataset Constraint for Offline Reinforcement Learning. In: Proceedings of the 40th International Conference on Machine Learning (ICML 2023), Honolulu, HA, 2023. [PDF] [Code]

Fuxiang Zhang*, Chengxing Jia*, Yi-Chen Li, Lei Yuan, Yang Yu, Zongzhang Zhang. Discovering Generalizable Multi-agent Coordination Skills from Multi-task Offline Data. In: Proceedings of the 11th International Conference on Learning Representations (ICLR 2023), 2023. [PDF] [Code]

Yi-Chen Li*, Wen-Jie Shen*, Boyu Zhang, Feng Mao, Yang Yu. Learning Generalizable Batch Active Learning Strategies via Deep Q-Networks (Student Abstract). In: Proceedings of the 37th AAAI Conference on Artificial Intelligence (AAAI 2023), Washington, DC, USA, 2023. [PDF] [Code]

Journal Papers

Cong Guan, Tao Jiang, Yi-Chen Li, Zongzhang Zhang, Lei Yuan, Yang Yu. Constraining an Unconstrained Multi-agent Policy with Offline Data. Neural Networks, in press, 2024. [PDF] [Code]

* Co-first Authors

Experience

• 2022/7-2022/9: Internship at Polixir, working for autonomous driving via imitation learning and reinforcement learning.

Projects

• RL-pytorch, A clean code base for deep reinforcment learning , written in Pytorch.

Awards and Honors

• Yingcai Scholarship of Nanjing University, Second Prize, 2024

• Huawei Scholarship, 2023

• Outstanding Graduate of Nanjing University, 2021

Teaching Assistant

• Control Theory and Method. (with Assoc.Prof. Zongzhang Zhang; for undergraduate students, Fall, 2023)

Service

• Contributor of d3rlpy , ALiPy , RLAssistant .
• Reviewer for NeurIPS (2023), ICLR (2024, 2025), ICML (2024), AAAI (2025), Neural Networks.

Correspondence

Email: liyc {AT} lamda.nju.edu.cn

Laboratory: Room A201, Shaoyifu Building, Xianlin Campus of Nanjing University

Address: National Key Laboratory for Novel Software Technology, Nanjing University, Xianlin Campus, 163 Xianlin Avenue, Qixia District, Nanjing 210023, China
(南京市栖霞区仙林大道163号, 南京大学仙林校区, 软件新技术国家重点实验室, 210023.)