Optimal Estimation of Parameters

This book presents a comprehensive and consistent theory of estimation. The framework described leads naturally to a generalized maximum capacity estimator. This approach allows the optimal estimation of real-valued parameters, their number and intervals, as well as providing common ground for explaining the power of these estimators. Beginning with a review of coding and the key properties of information, the author goes on to discuss the techniques of estimation and develops the generalized maximum capacity estimator, based on a new form of Shannon's mutual information and channel capacity. Applications of this powerful technique in hypothesis testing and denoising are described in detail. Offering an original and thought-provoking perspective on estimation theory, Jorma Rissanen's book is of interest to graduate students and researchers in the fields of information theory, probability and statistics, econometrics and finance.

1. Introduction; 2. Coding; 3. Basics of information; 4. Modeling problem; 5. Other optimality properties; 6. Interval estimation; 7. Hypothesis testing; 8. Denoising; 9. Sequential models; Appendix A. Elements of algorithmic information; Appendix B. Universal prior for integers.

Jorma Rissanen was a member of research staff in IBM Almaden Research Center from 1965 to 2001 and is currently Professor Emeritus at Technical University of Tampere, Finland. Among his main achievements are the introduction of the MDL principle for statistics, the invention of arithmetic coding and the introduction of variable-length Markov chains with the associated Algorithm Context. He has received many awards, including the 2007 Kolmogorov medal from the CLRC, University of London, and the 2009 Shannon Award from the Information Theory Society. He received two Outstanding Innovation Awards from IBM in 1980 and 1988 and a Corporate Award in 1991.