Exploring into the Unseen: Enhancing Language-Conditioned Policy Generalization with Behavioral Information

Abstract

Generalizing policies learned by agents in known environments to unseen domains is an essential challenge in advancing the development of reinforcement learning. Lately, language-conditioned policies have underscored the pivotal role of linguistic information in the context of cross-environments. Integrating both environmental and textual information into the observation space enables agents to accomplish similar tasks across different scenarios. However, for entities with varying forms of motion but the same name present in observations (e.g., immovable mage and fleeing mage), existing methods are unable to learn the motion information the entities possess well. They face the problem of ambiguity caused by motion. In order to tackle this challenge, we propose the entity mapper with multi-modal attention based on behavior prediction (EMMA-BBP) framework, comprising modules for predicting motion behavior and text matching. The behavioral prediction module is used to determine the motion information of the entities present in the environment to eliminate the semantic ambiguity of the motion information. The role of the text-matching module is to match the text given in the environment with the information about the entity’s behavior under observation, thus eliminating false textual information. EMMA-BBP has been tested in the demanding environment of MESSENGER, doubling the generalization ability of EMMA.