Lihe Li @ LAMDA, NJU-AI

We review multi-agent cooperation from closed environment to open environment settings, and provide prospects for future development and research directions of cooperative MARL in open environments.

we propose employing the large language model (LLM) to develop an action plan (or equivalently, a convention) that effectively guides both human and AI.

We propose Multi-agent Compatible Policy Learning (MACOP) to continually and alternatively (1) generate teammates incompatible with the controllable agents and (2) train the controllable agents to coordinate with the generated teammates. This approrach generates diverse teammates that cover the teammate policy space and controllable agents that can coordinate with anyone.

We formulate Open Dec-POMDP and propose Fast teammate adaptation (Fastap) to enable controllable agents in a multi-agent system to fast adapt to the uncontrollable teammates, whose policy could be changed with one episode.

We formulate Limited Policy Adversary Dec-POMDP and propose ROMANCE to enable the trained agents to encounter diversified and strong auxiliary adversarial attacks during training, achieving high robustness under various policy perturbations.

We propose CroMAC to enable agents to obtain guaranteed lower bounds on state-action values to identify and choose the optimal action under a worst-case deviation when the received messages are perturbed.

We formulate the continual coordination framework and propose MACPro to enable agents to continually coordinate with each other when the dynamic of the training task and the multi-agent system itself changes over time. [code]