**G. Banjac and J. Lygeros**

BibTeX

@inproceedings{BL:2019, author = {G. Banjac and J. Lygeros}, title = {A data-driven policy iteration scheme based on linear programming}, booktitle = {IEEE Conference on Decision and Control (CDC)}, year = {2019} }

We consider the problem of learning discounted-cost optimal control policies for unknown deterministic discrete-time systems with continuous state and action spaces. We show that a policy evaluation step of the well-known policy iteration (PI) algorithm can be characterized as a solution to an infinite dimensional linear program (LP). However, when approximating such an LP with a finite dimensional program, the PI algorithm loses its nominal properties. We propose a data-driven PI scheme that ensures a certain monotonic behavior and allows for incorporation of expert knowledge on the system. A numerical example illustrates effectiveness of the proposed algorithm.