Introduction to Probability Theory and Stochastic Processes

A unique approach to stochastic processes that connects the mathematical formulation of random processes to their use in applications

This book presents an innovative approach to teaching probability theory and stochastic processes based on the binary expansion of the unit interval. Departing from standard pedagogy, it uses the binary expansion of the unit interval to explicitly construct an infinite sequence of independent random variables (of any given distribution) on a single probability space. This construction then provides the framework to understand the mathematical formulation of probability theory for its use in applications.

Features include:

• The theory is presented first for countable sample spaces (Chapters 1-3) and then for uncountable sample spaces (Chapters 4-18)
• Coverage of the explicit construction of i.i.d. random variables on a single probability space to explain why it is the distribution function rather than the functional form of random variables that matters when it comes to modeling random phenomena
• Explicit construction of continuous random variables to facilitate the "digestion" of random variables, i.e., how they are used in contrast to how they are defined
• Explicit construction of continuous random variables to facilitate the two views of expectation: as integration over the underlying probability space (abstract view) or as integration using the density function (usual view)
• A discussion of the connections between Bernoulli, geometric, and Poisson processes
• Incorporation of the Johnson-Nyquist noise model and an explanation of why (and when) it is valid to use a delta function to model its autocovariance

Comprehensive, astute, and practical, Introduction to Probability Theory and Stochastic Processes is a clear presentation of essential topics for those studying communications, control, machine learning, digital signal processing, computer networks, pattern recognition, image processing, and coding theory.

1 Coin Tossing 1

2 Countable Sample Spaces 61

3 Conditional Probability in Countable Sample Spaces 105

4 Uncountable Sample Spaces 151

5 Continuous Random Variables 213

6 Expectation 245

7 Modeling Random Phenomena 267

8 Functions of One Random Variables and Transforms 321

9 Functions of Two Random Variables 365

10 Two Functions of Two Random Variables 431

11 Conditional Probability for Continuous Random Variables 473

12 Random Vectors 549

13 Bernoulli, Geometric, and Poisson Processes 587

14 Brownian Motions and White Noise 645

15 Stationary Random Processes 703

16 Convergence of Random Variables 777

17 Statistics 839

18 Kalman Filter 905

Further Reading 933

Table of Common Distributions 935

References 941

Index 946

JOHN CHIASSON, PhD, is a Fellow of the IEEE and the author of Modeling and High-Performance Control of Electric Machines, published by Wiley-IEEE Press.