1. Sutton, R. S., & Barto, A. G. (2018). Reinforcement Learning: An Introduction. Massachusetts: MIT press.
2. Laurent, G. J., Matignon, L., Fort-Piat, L., et al. (2011). The world of independent learners is not markovian. International Journal of Knowledge-based and Intelligent Engineering Systems, 15(1), 55–64.
3. Even-Dar, E., Kakade, S. M., & Mansour, Y. (2009). Online markov decision processes. Mathematics of Operations Research, 34(3), 726–736.
4. Dick, T., Gyorgy, A., & Szepesvari, C. (2014). Online learning in markov decision processes with changing cost sequences. In ICML (pp. 512–520).
5. Neu, G., Antos, A., György, A., & Szepesvári, C. (2010). Online markov decision processes under bandit feedback. In NeurIPS (pp. 1804–1812).