Beyond the Worst-Case Analysis of Algorithms

There are no silver bullets in algorithm design, and no single algorithmic idea is powerful and flexible enough to solve every computational problem. Nor are there silver bullets in algorithm analysis, as the most enlightening method for analyzing an algorithm often depends on the problem and the application. However, typical algorithms courses rely almost entirely on a single analysis framework, that of worst-case analysis, wherein an algorithm is assessed by its worst performance on any input of a given size. The purpose of this book is to popularize several alternatives to worst-case analysis and their most notable algorithmic applications, from clustering to linear programming to neural network training. Forty leading researchers have contributed introductions to different facets of this field, emphasizing the most important models and results, many of which can be taught in lectures to beginning graduate students in theoretical computer science and machine learning.

Forward; Preface; 1. Introduction Tim Roughgarden; Part I. Refinements of Worst-Case Analysis: 2. Parameterized algorithms Fedor Fomin, Daniel Lokshtanov, Saket Saurabh, and Meirav Zehavi; 3. From adaptive analysis to instance optimality Jérémy Barbay; 4. Resource augmentation Tim Roughgarden; Part II. Deterministic Models of Data: 5. Perturbation resilience Konstantin Makarychev and Yury Makarychev; 6. Approximation stability and proxy objectives Avrim Blum; 7. Sparse recovery Eric Price; Part III. Semi-Random Models: 8. Distributional analysis Tim Roughgarden; 9. Introduction to semi-random models Uriel Feige; 10. Semi-random stochastic block models Ankur Moitra; 11. Random-order models Anupam Gupta and Sahil Singla; 12. Self-improving algorithms C. Seshadhri; Part IV. Smoothed Analysis: 13. Smoothed analysis of local search Bodo Manthey; 14. Smoothed analysis of the simplex method Daniel Dadush and Sophie Huiberts; 15. Smoothed analysis of Pareto curves in multiobjective optimization Heiko Röglin; Part V. Applications in Machine Learning and Statistics: 16. Noise in classification Maria-Florina Balcan and Nika Haghtalab; 17. Robust high-dimensional statistics Ilias Diakonikolas and Daniel Kane; 18. Nearest-neighbor classification and search Sanjoy Dasgupta and Samory Kpotufe; 19. Efficient tensor decomposition Aravindan Vijayaraghavan; 20. Topic models and nonnegative matrix factorization Rong Ge and Ankur Moitra; 21. Why do local methods solve nonconvex problems? Tengyu Ma; 22. Generalization in overparameterized models Moritz Hardt; 23. Instance-optimal distribution testing and learning Gregory Valiant and Paul Valiant; Part VI. Further Applications: 24. Beyond competitive analysis Anna R. Karlin and Elias Koutsoupias; 25. On the unreasonable effectiveness of satisfiability solvers Vijay Ganesh and Moshe Vardi; 26. When simple hash functions suffice Kai-Min Chung, Michael Mitzenmacher and Salil Vadhan; 27. Prior-independent auctions Inbal Talgam-Cohen; 28. Distribution-free models of social networks Tim Roughgarden and C. Seshadhri; 29. Data-driven algorithm design Maria-Florina Balcan; 30. Algorithms with predictions Michael Mitzenmacher and Sergei Vassilvitskii.

Tim Roughgarden is a Professor of Computer Science at Columbia University. For his research, he has been awarded the ACM Grace Murray Hopper Award, the Presidential Early Career Award for Scientists and Engineers (PECASE), the Kalai Prize in Computer Science and Game Theory, the Social Choice and Welfare Prize, the Mathematical Programming Society's Tucker Prize, and the EATCS-SIGACT Gödel Prize. He was an invited speaker at the 2006 International Congress of Mathematicians, the Shapley Lecturer at the 2008 World Congress of the Game Theory Society, and a Guggenheim Fellow in 2017. His other books include Twenty Lectures on Algorithmic Game Theory (2016) and the Algorithms Illuminated book series (2017-2020).