Mathematical Statistics and Stochastic Processes

Generally, books on mathematical statistics are restricted to the case of independent identically distributed random variables. In this book however, both this case AND the case of dependent variables, i.e. statistics for discrete and continuous time processes, are studied. This second case is very important for today?s practitioners.
Mathematical Statistics and Stochastic Processes is based on decision theory and asymptotic statistics and contains up-to-date information on the relevant topics of theory of probability, estimation, confidence intervals, non-parametric statistics and robustness, second-order processes in discrete and continuous time and diffusion processes, statistics for discrete and continuous time processes, statistical prediction, and complements in probability.
This book is aimed at students studying courses on probability with an emphasis on measure theory and for all practitioners who apply and use statistics and probability on a daily basis.

Preface xiii

PART 1. MATHEMATICAL STATISTICS 1

Chapter 1. Introduction to Mathematical Statistics 3

1.1. Generalities 3

1.2. Examples of statistics problems 4

Chapter 2. Principles of Decision Theory 9

2.1. Generalities 9

2.2. The problem of choosing a decision function 11

2.3. Principles of Bayesian statistics 13

2.4. Complete classes 17

2.5. Criticism of decision theory – the asymptotic point of view 18

2.6. Exercises 18

Chapter 3. Conditional Expectation 21

3.1. Definition 21

3.2. Properties and extension 22

3.3. Conditional probabilities and conditional distributions 24

3.4. Exercises 26

Chapter 4. Statistics and Sufficiency 29

4.1. Samples and empirical distributions 29

4.2. Sufficiency 31

4.3. Examples of sufficient statistics – an exponential model 33

4.4. Use of a sufficient statistic 35

4.5. Exercises 36

Chapter 5. Point Estimation 39

5.1. Generalities 39

5.2. Sufficiency and completeness 42

5.3. The maximum-likelihood method 45

5.4. Optimal unbiased estimators 49

5.5. Efficiency of an estimator 56

5.6. The linear regression model 65

5.7. Exercises 68

Chapter 6. Hypothesis Testing and Confidence Regions 73

6.1. Generalities 73

6.2. The Neyman–Pearson (NP) lemma 75

6.3. Multiple hypothesis tests (general methods) 80

6.4. Case where the ratio of the likelihoods is monotonic 84

6.5. Tests relating to the normal distribution 86

6.6. Application to estimation: confidence regions 86

6.7. Exercises 90

Chapter 7. Asymptotic Statistics 101

7.1. Generalities 101

7.2. Consistency of the maximum likelihood estimator 103

7.3. The limiting distribution of the maximum likelihood estimator 104

7.4. The likelihood ratio test 106

7.5. Exercises 108

Chapter 8. Non-Parametric Methods and Robustness 113

8.1. Generalities 113

8.2. Non-parametric estimation 114

8.3. Non-parametric tests 117

8.4. Robustness 121

8.5. Exercises 124

PART 2. STATISTICS FOR STOCHASTIC PROCESSES 131

Chapter 9. Introduction to Statistics for Stochastic Processes 133

9.1. Modeling a family of observations 133

9.2. Processes 134

9.3. Statistics for stochastic processes 137

9.4. Exercises 138

Chapter 10. Weakly Stationary Discrete-Time Processes 141

10.1. Autocovariance and spectral density 141

10.2. Linear prediction and Wold decomposition 144

10.3. Linear processes and the ARMA model 146

10.4. Estimating the mean of a weakly stationary process 149

10.5. Estimating the autocovariance 151

10.6. Estimating the spectral density 151

10.7. Exercises 155

Chapter 11. Poisson Processes – A Probabilistic and Statistical Study 163

11.1. Introduction 163

11.2. The axioms of Poisson processes 164

11.3. Interarrival time 166

11.4. Properties of the Poisson process 168

11.5. Notions on generalized Poisson processes 170

11.6. Statistics of Poisson processes 172

11.7. Exercises 177

Chapter 12. Square-Integrable Continuous-Time Processes 183

12.1. Definitions 183

12.2. Mean-square continuity 183

12.3. Mean-square integration 184

12.4. Mean-square differentiation 187

12.5. The Karhunen–Loeve theorem 188

12.6. Wiener processes 189

12.7. Notions on weakly stationary continuous-time processes 195

12.8. Exercises 197

Chapter 13. Stochastic Integration and Diffusion Processes 203

13.1. Itô integral 203

13.2. Diffusion processes 206

13.3. Processes defined by stochastic differential equations and stochastic integrals 212

13.4. Notions on statistics for diffusion processes 215

13.5. Exercises 216

Chapter 14. ARMA Processes 219

14.1. Autoregressive processes 219

14.2. Moving average processes 223

14.3. General ARMA processes 224

14.4. Non-stationary models 226

14.5. Statistics of ARMA processes 228

14.6. Multidimensional processes 232

14.7. Exercises 233

Chapter 15. Prediction 239

15.1. Generalities 239

15.2. Empirical methods of prediction 240

15.3. Prediction in the ARIMA model 242

15.4. Prediction in continuous time 244

15.5. Exercises 245

PART 3. SUPPLEMENT 249

Chapter 16. Elements of Probability Theory 251

16.1. Measure spaces: probability spaces 251

16.2. Measurable functions: real random variables 253

16.3. Integrating real random variables 255

16.4. Random vectors 259

16.5. Independence 261

16.6. Gaussian vectors 262

16.7. Stochastic convergence 264

16.8. Limit theorems 265

Appendix. Statistical Tables 267

A1.1. Random numbers 267

A1.2. Distribution function of the standard normal distribution 268

A1.3. Density of the standard normal distribution 269

A1.4. Percentiles (tp) of Student’s distribution 270

A1.5. Ninety-fifth percentiles of Fisher–Snedecor distributions 271

A1.6. Ninety-ninth percentiles of Fisher–Snedecor distributions 272

A1.7. Percentiles (χ2 p) of the χ2 distribution with n degrees of freedom 273

A1.8. Individual probabilities of the Poisson distribution 274

A1.9. Cumulative probabilities of the Poisson distribution 275

A1.10. Binomial coefficients Ck n for n ≤ 30 and 0 ≤ k ≤ 7 276

A1.11. Binomial coefficients Ck n for n ≤ 30 and 8 ≤ k ≤ 15 277

Bibliography 279

Index 281