Entropy, Large Deviations, and Statistical Mechanics

This book has two main topics: large deviations and equilibrium statistical mechanics. I hope to convince the reader that these topics have many points of contact and that in being treated together, they enrich each other. Entropy, in its various guises, is their common core. The large deviation theory which is developed in this book focuses upon convergence properties of certain stochastic systems. An elementary example is the weak law of large numbers. For each positive e, P{ISn/nl 2: e} con- verges to zero as n --+ 00, where Sn is the nth partial sum of indepen- dent identically distributed random variables with zero mean. Large deviation theory shows that if the random variables are exponentially bounded, then the probabilities converge to zero exponentially fast as n --+ 00. The exponen- tial decay allows one to prove the stronger property of almost sure conver- gence (Sn/n --+ 0 a.s.). This example will be generalized extensively in the book. We will treat a large class of stochastic systems which involve both indepen- dent and dependent random variables and which have the following features: probabilities converge to zero exponentially fast as the size of the system increases; the exponential decay leads to strong convergence properties of the system. The most fascinating aspect of the theory is that the exponential decay rates are computable in terms of entropy functions. This identification between entropy and decay rates of large deviation probabilities enhances the theory significantly.

This book has two main topics: large deviations and equilibrium statistical mechanics. I hope to convince the reader that these topics have many points of contact and that in being treated together, they enrich each other. Entropy, in its various guises, is their common core. The large deviation theory which is developed in this book focuses upon convergence properties of certain stochastic systems. An elementary example is the weak law of large numbers. For each positive e, P{ISn/nl 2: e} con- verges to zero as n --+ 00, where Sn is the nth partial sum of indepen- dent identically distributed random variables with zero mean. Large deviation theory shows that if the random variables are exponentially bounded, then the probabilities converge to zero exponentially fast as n --+ 00. The exponen- tial decay allows one to prove the stronger property of almost sure conver- gence (Sn/n --+ 0 a.s.). This example will be generalized extensively in the book. We will treat a large class of stochastic systems which involve both indepen- dent and dependent random variables and which have the following features: probabilities converge to zero exponentially fast as the size of the system increases; the exponential decay leads to strong convergence properties of the system. The most fascinating aspect of the theory is that the exponential decay rates are computable in terms of entropy functions. This identification between entropy and decay rates of large deviation probabilities enhances the theory significantly.