On the convergence of a two timescale stochastic optimisation algorithm for optimal observer trajectory planning
Sensor scheduling has been a topic of interest to the target tracking community for some years now and more recently, it has enjoyed fresh impetus with the current importance and popularity of applications in Sensor Networks and Robotics. The sensor scheduling problem can be formulated as a controlled Hidden Markov Model. In the paper [7], we addressed precisely this problem and considered the case in which the state, observation and action spaces are continuous. We presented a novel simulation-based method that used a two timescale stochastic approximation algorithm to find optimal actions. In this report, we study the convergence of the proposed stochastic approximation algorithm under general assumptions and for the specific observer trajectory planning application of [7].