Cost-Sensitive Action Model Learning

Abstract

Action model learning can relieve people from writing planning domain descriptions from scratch. Real-world learners need to be sensitive to all kinds of expenses which it will spend in the learning. However, most of previous studies in this research line only considered the running time as the learning cost. In real-world applications, we will spend extra expense when we carry out actions or get observations, particularly for online learning. The learning algorithm should apply more techniques for saving the total cost when keeping a high rate of accuracy. The cost of carrying out actions and getting observations is the dominated expense in online learning. Therefore, we design a cost-sensitive algorithm to learn action models under partial observability. It combines three techniques to lessen the total cost: constraints, filtering and active learning. These techniques are used in observation reduction in action model learning. First, the algorithm uses constraints to confine the observation space. Second, it removes unnecessary observations by belief state filtering. Third, it actively picks up observations based on the results of the previous two techniques. This paper also designs strategies to reduce the amount of plan steps used in the learning. We performed experiments on some benchmark domains. It shows two results. For one thing, the learning accuracy is high in most cases. For the other, the algorithm dramatically reduces the total cost according to the definition of cost in this paper. Therefore, it is significant for real-world learners, especially, when long plans are unavailable or observations are expensive.