Large-Scale Inference
Empirical Bayes Methods for Estimation, Testing, and Prediction
Institute of Mathematical Statistics Monographs Series

We live in a new age for statistical inference, where modern scientific technology such as microarrays and fMRI machines routinely produce thousands and sometimes millions of parallel data sets, each with its own estimation or testing problem. Doing thousands of problems at once is more than repeated application of classical methods. Taking an empirical Bayes approach, Bradley Efron, inventor of the bootstrap, shows how information accrues across problems in a way that combines Bayesian and frequentist ideas. Estimation, testing and prediction blend in this framework, producing opportunities for new methodologies of increased power. New difficulties also arise, easily leading to flawed inferences. This book takes a careful look at both the promise and pitfalls of large-scale statistical inference, with particular attention to false discovery rates, the most successful of the new statistical techniques. Emphasis is on the inferential ideas underlying technical developments, illustrated using a large number of real examples.

Introduction and foreword; 1. Empirical Bayes and the James–Stein estimator; 2. Large-scale hypothesis testing; 3. Significance testing algorithms; 4. False discovery rate control; 5. Local false discovery rates; 6. Theoretical, permutation and empirical null distributions; 7. Estimation accuracy; 8. Correlation questions; 9. Sets of cases (enrichment); 10. Combination, relevance, and comparability; 11. Prediction and effect size estimation; A. Exponential families; B. Programs and data sets; Bibliography; Index.

Bradley Efron is Max H. Stein Professor of Statistics and Biostatistics at the Stanford University School of Humanities and Sciences, and the Department of Health Research and Policy with the School of Medicine.