Methods of Multivariate Analysis (3^rd Ed.)
Wiley Series in Probability and Statistics Series

Praise for the Second Edition

"This book is a systematic, well-written, well-organized text on multivariate analysis packed with intuition and insight . . . There is much practical wisdom in this book that is hard to find elsewhere."
—IIE Transactions

Filled with new and timely content, Methods of Multivariate Analysis, Third Edition provides examples and exercises based on more than sixty real data sets from a wide variety of scientific fields. It takes a "methods" approach to the subject, placing an emphasis on how students and practitioners can employ multivariate analysis in real-life situations.

This Third Edition continues to explore the key descriptive and inferential procedures that result from multivariate analysis. Following a brief overview of the topic, the book goes on to review the fundamentals of matrix algebra, sampling from multivariate populations, and the extension of common univariate statistical procedures (including t-tests, analysis of variance, and multiple regression) to analogous multivariate techniques that involve several dependent variables. The latter half of the book describes statistical tools that are uniquely multivariate in nature, including procedures for discriminating among groups, characterizing low-dimensional latent structure in high-dimensional data, identifying clusters in data, and graphically illustrating relationships in low-dimensional space. In addition, the authors explore a wealth of newly added topics, including:

• Confirmatory Factor Analysis
• Classification Trees
• Dynamic Graphics
• Transformations to Normality
• Prediction for Multivariate Multiple Regression
• Kronecker Products and Vec Notation

New exercises have been added throughout the book, allowing readers to test their comprehension of the presented material. Detailed appendices provide partial solutions as well as supplemental tables, and an accompanying FTP site features the book's data sets and related SAS® code.

Requiring only a basic background in statistics, Methods of Multivariate Analysis, Third Edition is an excellent book for courses on multivariate analysis and applied statistics at the upper-undergraduate and graduate levels. The book also serves as a valuable reference for both statisticians and researchers across a wide variety of disciplines.

Acknowledgments xxi

1 Introduction 1

1.1 Why Multivariate Analysis? 1

1.2 Prerequisites 3

1.3 Objectives 3

1.4 Basic Types of Data And Analysis 4

2 Matrix Algebra 7

2.1 Introduction 7

2.2 Notation and Basic Definitions 8

2.3 Operations 11

2.4 Partitioned Matrices 22

2.5 Rank 23

2.6 Inverse 25

2.7 Positive Definite Matrices 26

2.8 Determinants 28

2.9 Trace 31

2.10 Orthogonal Vectors and Matrices 31

2.11 Eigenvalues and Eigenvectors 32

2.12 Kronecker and VEC Notation 37

Problems 39

3 Characterizing and Displaying Multivariate Data 47

3.1 Mean and Variance of a Univariate Random Variable 47

3.2 Covariance and Correlation Of Bivariate Random Variables 49

3.3 Scatter Plots of Bivariate Samples 55

3.4 Graphical Displays for Multivariate Samples 56

3.5 Dynamic Graphics 58

3.6 Mean Vectors 63

3.7 Covariance Matrices 66

3.8 Correlation Matrices 69

3.9 Mean Vectors and Covariance Matrices for Subsets of Variables 71

3.9.1 Two Subsets 71

3.9.2 Three or More Subsets 73

3.10 Linear Combinations of Variables 75

3.10.1 Sample Properties 75

3.10.2 Population Properties 81

3.11 Measures of Overall Variability 81

3.12 Estimation of Missing Values 82

3.13 Distance Between Vectors 84

Problems 85

4 The Multivariate Normal Distribution 91

4.1 Multivariate Normal Density Function 91

4.2 Properties of Multivariate Normal Random Variables 94

4.3 Estimation in the Multivariate Normal 99

4.4 Assessing Multivariate Normality 101

4.5 Transformations to Normality 108

4.6 Outliers 111

Problems 117

5 Tests on One or Two Mean Vectors 125

5.1 Multivariate Versus Univariate Tests 125

5.2 Tests on µ With ??Known 126

5.3 Tests on µ When ??is Unknown 130

5.4 Comparing two Mean Vectors 134

5.5 Tests on Individual Variables Conditional on Rejection of H0 by the T2-test

139

5.6 Computation of T2 143

5.7 Paired Observations Test 145

5.8 Test for Additional Information 149

5.9 Profile Analysis 152

Profile Analysis 154

Problems 161

6 Multivariate Analysis of Variance 169

6.1 One-way Models 169

6.2 Comparison of the Four Manova Test Statistics 189

6.3 Contrasts 191

6.4 Tests on Individual Variables Following Rejection of H0 by the Overall Manova Test 195

6.5 Two-Way Classification 198

6.6 Other Models 207

6.7 Checking on the Assumptions 210

6.8 Profile Analysis 211

6.9 Repeated Measures Designs 215

6.10 Growth Curves 232

6.11 Tests on a Subvector 241

Problems 244

7 Tests on Covariance Matrices 259

7.1 Introduction 259

7.2 Testing a Specified Pattern for ∑ 259

7.3 Tests Comparing Covariance Matrices 265

7.4 Tests of Independence 269

Problems 276

8 Discriminant Analysis: Description of Group Separation 281

8.1 Introduction 281

8.2 The Discriminant Function for two Groups 282

8.3 Relationship Between two-group Discriminant Analysis and Multiple Regression 286

8.4 Discriminant Analysis for Several Groups 288

8.5 Standardized Discriminant Functions 292

8.6 Tests of Significance 294

8.7 Interpretation of Discriminant Functions 298

8.8 Scatter Plots 301

8.9 Stepwise Selection of Variables 303

Problems 306

9 Classification Analysis: Allocation of Observations to Groups309

9.1 Introduction 309

9.2 Classification into two Groups 310

9.3 Classification into Several Groups 314

9.4 Estimating Misclassification Rates 318

9.5 Improved Estimates of Error Rates 320

9.6 Subset Selection 322

9.7 Nonparametric Procedures 326

Problems 336

10 Multivariate Regression 339

10.1 Introduction 339

10.2 Multiple Regression: Fixed X’s 340

10.3 Multiple Regression: Random X’s 354

10.4 Multivariate Multiple Regression: Estimation 354

10.5 Multivariate Multiple Regression: Hypothesis Tests 364

10.6 Multivariate Multiple Regression: Prediction 370

10.7 Measures of Association Between the Y’s and the X’s 372

10.8 Subset Selection 374

10.9 Multivariate Regression: Random X’s 380

Problems 381

11 Canonical Correlation 385

11.1 Introduction 385

11.2 Canonical Correlations and Canonical Variates 385

11.3 Properties of Canonical Correlations 390

11.4 Tests of Significance 391

11.5 Interpretation 395

11.6 Relationships of Canonical Correlation Analysis to Other Multivariate

Problems 402

12 Principal Component Analysis 405

12.1 Introduction 405

12.2 Geometric and Algebraic Bases of Principal Components 406

12.3 Principal Components and Perpendicular Regression 412

12.4 Plotting of Principal Components 414

12.5 Principal Components from the Correlation Matrix 419

12.6 Deciding How Many Components to Retain 423

12.7 Information in the Last Few Principal Components 427

12.8 Interpretation of Principal Components 427

12.9 Selection of Variables 430

Problems 432

13 Exploratory Factor Analysis 435

13.1 Introduction 435

13.2 Orthogonal Factor Model 437

13.3 Estimation of Loadings and Communalities 442

13.4 Choosing the Number of Factors, m 453

13.5 Rotation 457

13.6 Factor Scores 466

13.7 Validity of the Factor Analysis Model 470

13.8 Relationship of Factor Analysis to Principal Component Analysis 475

Problems 476

14 Confirmatory Factor Analysis 479

14.1 Introduction 479

14.2 Model Specification and Identification 480

14.3 Parameter Estimation and Model Assessment 487

14.4 Inference for Model Parameters 492

14.5 Factor Scores 495

Problems 496

15 Cluster Analysis 501

15.1 Introduction 501

15.2 Measures of Similarity or Dissimilarity 502

15.3 Hierarchical Clustering 505

15.4 Nonhierarchical Methods 531

15.5 Choosing the Number of Clusters 544

15.6 Cluster Validity 546

15.7 Clustering Variables 547

Problems 548

16 Graphical Procedures 555

16.1 Multidimensional Scaling 555

16.2 Correspondence Analysis 565

16.3 Biplots 580

Problems 588

Appendix A: Tables 597

Appendix B: Answers and Hints to Problems 637

Appendix C: Data Sets and SAS Files 727

References 729

Index 747

ALVIN C. RENCHER is Professor Emeritus in the Department of Statistics at Brigham Young University. A Fellow of the American Statistical Association, he is the author of Linear Models in Statistics, Second Edition and Multivariate Statistical Inference and Applications, both published by Wiley.

WILLIAM F. CHRISTENSEN is Professor in the Department of Statistics at Brigham Young University. He has been published extensively in his areas of research interest, which include multivariate analysis, resampling methods, and spatial and environmental statistics.