The Cross-Entropy Method

The cross-entropy (CE) method is one of the most significant developments in randomized optimization and simulation in recent years. This book explains in detail how and why the CE method works. The simplicity and versatility of the method is illustrated via a diverse collection of optimization and estimation problems. The book is aimed at a broad audience of engineers, computer scientists, mathematicians, statisticians and in general anyone, theorist and practitioner, who is interested in smart simulation, fast optimization, learning algorithms, and image processing.

This book is a comprehensive and accessible introduction to the cross-entropy (CE) method. The CE method started life around 1997 when the first author proposed an adaptive algorithm for rare-event simulation using a cross-entropy minimization technique. It was soon realized that the underlying ideas had a much wider range of application than just in rare-event simulation; they could be readily adapted to tackle quite general combinatorial and multi-extremal optimization problems, including many problems associated with the field of learning algorithms and neural computation. The book is based on an advanced undergraduate course on the CE method, given at the Israel Institute of Technology (Technion) for the last three years. It is aimed at a broad audience of engineers, computer scientists, mathematicians, statisticians and in general anyone, theorist or practitioner, who is interested in smart simulation, fast optimization, learning algorithms, image processing, etc. Our aim was to write a book on the CE method which was accessible to advanced undergraduate students and engineers who simply want to apply the CE method in their work, while at the same time accentu ating the unifying and novel mathematical ideas behind the CE method, so as to stimulate further research at a postgraduate level.

Preliminaries A Tutorial Introduction to the Cross-Entropy Method Efficient Simulation via Cross-Entropy Combinatorial Optimization via Cross-Entropy Continuous Optimization and Modifications Noisy Optimization with CE Applications of CE to COPs Applications of CE to Machine Learning Appendix: Example Programs