Optimization for value function approximation
Many problems in control theory are theoretically addressable by dynamic programming. The theory of dynamic programming revolves around the notion of a value function—a function that evaluates the performance of the best possible controller for a given system. Dynamic programming gives methods for computing the value function and for finding an optimal controller once the value function is known. However, these methods break down for many real-world problems; due to their size and complexity, the computation of a value function becomes intractably difficult. As a result, there is growing interest in efficient methods for finding an approximation to the value function. Commonly the value function is approximated by a weighted summation of basis functions. The basis functions are chosen by the practitioner who relies on his intuition to generate a set of functions whose span is likely to contain a good approximation to the value function. The weights are then chosen using computational algorithms that incorporate elements of dynamic programming and statistical regression. 
In this thesis, two new algorithms are proposed for value function approximation. These new algorithms borrow from optimization theory to extend the state-of-the-art in value function approximation. The first algorithm uses elements of the Kalman filter to speed up the generation of the weights. The second algorithm uses elements of the simplex method to automate the selection of basis functions. The contributions of the thesis include a theoretical analysis (a convergence proof and error bounds) of the first algorithm, and computation experiments for both algorithms. The experiments consist of problems in queueing networks, including networks that can be used to model problems such as management of a call center for a business. In the experimental results, the first algorithm successfully accelerates the computation of weights, while the second algorithm successfully chooses basis functions to approximate the value function.