Stochastic Averaging and Stochastic Extremum Seeking
Shu-Jun Liu and Miroslav Krstic

Springer, 2012

224 pages, hardcover

ISBN 978-1-4471-4086-3


Preface and Contents





1. Introduction to Averaging

2. Introduction to Extremum Seeking

3. Stochastic Averaging for Asymptotic Stability

4. Stochastic Averaging for Practical Stability

5. Single-parameter Stochastic Extremum Seeking

6. Stochastic Source Seeking for Nonholonomic Vehicles

7. Stochastic Source Seeking with Tuning of Forward Velocity

8. Multi-parameter Stochastic Extremum Seeking and Slope Seeking

9. Stochastic Nash Equilibrium Seeking for Games with General Nonlinear Payoffs

10. Nash Equilibrium Seeking for Quadratic Games and Application to Oligopoly Markets and Vehicle Deployment

11. Newton-based Stochastic Extremum Seeking.






Order from

Order from Springer


Publisher's description:


Stochastic Averaging and Stochastic Extremum Seeking develops methods of mathematical analysis inspired by the interest in reverse engineering  and analysis of bacterial  convergence by chemotaxis and to apply similar stochastic optimization techniques in other environments.


The first half of the text presents significant advances in stochastic averaging theory, necessitated by the fact that existing theorems are restricted to systems with linear growth, globally exponentially stable average models, vanishing stochastic perturbations, and prevent analysis over infinite time horizon.


The second half of the text introduces stochastic extremum seeking algorithms for model-free optimization of systems in real time using stochastic perturbations for estimation of their gradients. Both gradient- and Newton-based algorithms are presented, offering the user the choice between the simplicity of implementation (gradient) and the ability to achieve a known, arbitrary convergence rate (Newton).


The design of algorithms for non-cooperative/adversarial games is described. The analysis of their convergence to Nash equilibria is provided. The algorithms are illustrated on models of economic competition and on problems of the deployment of teams of robotic vehicles. Bacterial locomotion, such as chemotaxis in E. coli, is explored with the aim of identifying two simple feedback laws for climbing nutrient gradients. Stochastic extremum seeking is shown to be a biologically plausible interpretation for chemotaxis. For the same chemotaxis-inspired stochastic feedback laws, the book also provides a detailed analysis of convergence for models of nonholonomic robotic vehicles operating in GPS-denied environments.