Too Big to Ignore
The Business Case for Big Data
Wiley and SAS Business Series

Residents in Boston, Massachusetts are automatically reporting potholes and road hazards via their smartphones. Progressive Insurance tracks real-time customer driving patterns and uses that information to offer rates truly commensurate with individual safety. Google accurately predicts local flu outbreaks based upon thousands of user search queries. Amazon provides remarkably insightful, relevant, and timely product recommendations to its hundreds of millions of customers. Quantcast lets companies target precise audiences and key demographics throughout the Web. NASA runs contests via gamification site TopCoder, awarding prizes to those with the most innovative and cost-effective solutions to its problems. Explorys offers penetrating and previously unknown insights into healthcare behavior.

How do these organizations and municipalities do it? Technology is certainly a big part, but in each case the answer lies deeper than that. Individuals at these organizations have realized that they don't have to be Nate Silver to reap massive benefits from today's new and emerging types of data. And each of these organizations has embraced Big Data, allowing them to make astute and otherwise impossible observations, actions, and predictions.

It's time to start thinking big.

In Too Big to Ignore, recognized technology expert and award-winning author Phil Simon explores an unassailably important trend: Big Data, the massive amounts, new types, and multifaceted sources of information streaming at us faster than ever. Never before have we seen data with the volume, velocity, and variety of today. Big Data is no temporary blip of fad. In fact, it is only going to intensify in the coming years, and its ramifications for the future of business are impossible to overstate.

Too Big to Ignore explains why Big Data is a big deal. Simon provides commonsense, jargon-free advice for people and organizations looking to understand and leverage Big Data. Rife with case studies, examples, analysis, and quotes from real-world Big Data practitioners, the book is required reading for chief executives, company owners, industry leaders, and business professionals.

Preface xvii

Acknowledgments xxiii

Introduction This Ain't Your Father's Data 1

Better Car Insurance through Data 2

Potholes and General Road Hazards 5

Recruiting and Retention 8

How Big is Big? The Size of Big Data 10

Why Now? Explaining the Big Data Revolution 12

Central Thesis of Book 22

Plan of Attack 24

Who Should Read This Book? 25

Summary 25

Notes 26

Chapter 1 Data 101 and the Data Deluge 29

The Beginnings: Structured Data 30

Structure This! Web 2.0 and the Arrival of Big Data 33

The Composition of Data: Then and Now 39

The Current State of the Data Union 41

The Enterprise and the Brave New Big Data World 43

Summary 46

Notes 47

Chapter 2 Demystifying Big Data 49

Characteristics of Big Data 50

The Anti-Definition: What Big Data Is Not 71

Summary 72

Notes 72

Chapter 3 The Elements of Persuasion: Big Data Techniques 77

The Big Overview 79

Statistical Techniques and Methods 80

Data Visualization 84

Automation 88

Semantics 93

Big Data and the Gang of Four 98

Predictive Analytics 100

Limitations of Big Data 105

Summary 106

Notes 107

Chapter 4 Big Data Solutions 111

Projects, Applications, and Platforms 114

Other Data Storage Solutions 121

Websites, Start-ups, and Web Services 128

Hardware Considerations 133

The Art and Science of Predictive Analytics 136

Summary 137

Notes 137

Chapter 5 Case Studies: The Big Rewards of Big Data 141

Quantcast: A Small Big Data Company 141

Explorys: The Human Case for Big Data 147

NASA: How Contests, Gamification, and Open Innovation

Enable Big Data 152

Summary 158

Notes 158

Chapter 6 Taking the Big Plunge 161

Before Starting 161

Starting the Journey 165

Avoiding the Big Pitfalls 174

Summary 181

Notes 181

Chapter 7 Big Data: Big Issues and Big Problems 183

Privacy: Big Data = Big Brother? 184

Big Security Concerns 188

Big, Pragmatic Issues 189

Summary 195

Notes 196

Chapter 8 Looking Forward: The Future of Big Data 197

Predicting Pregnancy 198

Big Data Is Here to Stay 200

Big Data Will Evolve 201

Projects and Movements 203

Big Data Will Only Get Bigger…and Smarter 205

The Internet of Things: The Move from Active to Passive Data

Generation 206

Big Data: No Longer a Big Luxury 211

Stasis Is Not an Option 212

Summary 213

Notes 214

Final Thoughts 217

Spreading the Big Data Gospel 219

Notes 220

Selected Bibliography 221

About the Author 223

Index 225

PHIL SIMON is a frequent keynote speaker, a recognized technology authority, and the award-winning author of seven management books. His contributions have been featured in The Harvard Business Review, Wired, BusinessWeek, The Huffington Post, and The New York Times, and on CNN, NBC, and CNBC.