Introduction to Stochastic Models

This book provides a pedagogical examination of the way in which stochastic models are encountered in applied sciences and techniques such as physics, engineering, biology and genetics, economics and social sciences. It covers Markov and semi-Markov models, as well as their particular cases: Poisson, renewal processes, branching processes, Ehrenfest models, genetic models, optimal stopping, reliability, reservoir theory, storage models, and queuing systems. Given this comprehensive treatment of the subject, students and researchers in applied sciences, as well as anyone looking for an introduction to stochastic models, will find this title of invaluable use.

This book provides a pedagogical examination of the way in which stochastic models are encountered in applied sciences and techniques such as physics, engineering, biology and genetics, economics and social sciences. It covers Markov and semi-Markov models, as well as their particular cases: Poisson, renewal processes, branching processes, Ehrenfest models, genetic models, optimal stopping, reliability, reservoir theory, storage models, and queuing systems. Given this comprehensive treatment of the subject, students and researchers in applied sciences, as well as anyone looking for an introduction to stochastic models, will find this title of invaluable use.

Preface ixChapter 1. Introduction to Stochastic Processes 11.1. Sequences of random variables 11.2. The notion of stochastic process 101.3. Martingales 131.4. Markov chains 171.5. State classification 241.6. Continuous-time Markov processes 271.7. Semi-Markov processes 33Chapter 2. Simple Stochastic Models 372.1. Urn models 372.2. Random walks 392.3. Brownian motion 442.4. Poisson processes 502.5. Birth and death processes 59Chapter 3. Elements of Markov Modeling 613.1. Markov models: ideas, history, applications 613.2. The discrete-time Ehrenfest model 633.3. Markov models in genetics 793.4. Markov storage models 1103.5. Reliability of Markov models 124Chapter 4. Renewal Models 1494.1. Fundamental concepts and examples 1494.2. Waiting times 1554.3. Modified renewal processes 1594.4. Replacement models 1614.5. Renewal reward processes 1654.6. The risk problem of an insurance company 1684.7. Counter models 1714.8. Alternating renewal processes 1804.9. Superposition of renewal processes 1824.10. Regenerative processes 186Chapter 5. Semi-Markov Models 1895.1. Introduction 1895.2. Markov renewal processes 1905.3. First-passage times and state classification 1965.4. Reliability 2005.5. Reservoir models 2075.6. Queues 2185.7. Digital communication channels 222Chapter 6. Branching Models 2276.1. The Bienaymé-Galton-Watson model 2276.2. Generalizations of the B-G-W model 2716.3. Continuous-time models 302Chapter 7. Optimal Stopping Models 3157.1. The classic optimal stopping problem 3157.2. Renewal with binary decision 333Bibliography 343Notation 367Index 369