Stochastic approximation algorithms and applications

the long version

The book presents a comprehensive development of the modern theory of stochastic approximation, or recursive stochastic algorithms, for both constrained and unconstrained problems, with step sizes that either go to zero or are constant and small (and perhaps random). The general motivation arises from the new challenges in applications that have arisen in recent years. There is a thorough treatment of both probability one and weak convergence methods for very general noise models. The convergence proofs are built around the powerful ODE (ordinary, differential equation) method, which characterizes the limit behavior of the algorithm in terms of the asymptotics of a "mean limit ODE" or an analogous dynamical system. Not only is the method particularly convenient for dealing with complicated noise and dynamics, but also greatly simplifies the treatment of the more classical cases. There is a thorough treatment of rate of convergence, iterate averaging, high-dimensional problems, ergodic cost problems, stability methods for correlated noise, and decentralized and asynchronous algorithms.