Finite-Memory Strategies in POMDPs with Long-Run Average Objectives-Reference-Cited by-同舟云学术

Finite-Memory Strategies in POMDPs with Long-Run Average Objectives

Published:2021-04-06 Issue: Volume: Page:
ISSN:0364-765X
Container-title:Mathematics of Operations Research
language:en
Short-container-title:Mathematics of OR

Author:

Chatterjee Krishnendu¹^ORCID,Saona Raimundo¹^ORCID,Ziliotto Bruno²^ORCID

Affiliation:

1. Institute of Science and Technology Austria, 3400 Klosterneuburg, Austria;

2. Centre de Recherche en Mathématiques de la Décision, Centre National de la Recherche Scientifique, Université Paris Dauphine, Université PSL, 75016 Paris, France

Abstract

Partially observable Markov decision processes (POMDPs) are standard models for dynamic systems with probabilistic and nondeterministic behaviour in uncertain environments. We prove that in POMDPs with long-run average objective, the decision maker has approximately optimal strategies with finite memory. This implies notably that approximating the long-run value is recursively enumerable, as well as a weak continuity property of the value with respect to the transition function.

Publisher

Institute for Operations Research and the Management Sciences (INFORMS)

Subject

Management Science and Operations Research,Computer Science Applications,General Mathematics

Reference24 articles.

1. Discrete-Time Controlled Markov Processes with Average Cost Criterion: A Survey

2. Probabilistic ω-automata

3. Bellman R (1957) A Markovian decision process. J. Math. Mech. 6(5):679–684.

4. Discrete Dynamic Programming

5. Bonet B, Geffner H (2009) Solving POMDPs: RTDP-bel vs. point-based algorithms. Proc. 21st Internat. Joint Conf. Artificial Intelligence (Morgan Kaufmann, San Francisco), 1641–1646.

Cited by 4 articles. 订阅此论文施引文献订阅此论文施引文献，注册后可以免费订阅5篇论文的施引文献，订阅后可以查看论文全部施引文献

1. Artificial enactive inference in three-dimensional world;Cognitive Systems Research;2024-08

2. Bounded-Memory Strategies in Partial-Information Games;Proceedings of the 39th Annual ACM/IEEE Symposium on Logic in Computer Science;2024-07-08

3. Strong Simple Policies for POMDPs;International Journal on Software Tools for Technology Transfer;2024-06

4. Dynamic Random Access Without Observation Under Deadline-Constrained Periodic Traffic;IEEE Transactions on Vehicular Technology;2024-01