Parallel Computing Architectures and APIs
IoT Big Data Stream Processing

Parallel Computing Architectures and APIs: IoT Big Data Stream Processing commences from the point high-performance uniprocessors were becoming increasingly complex, expensive, and power-hungry. A basic trade-off exists between the use of one or a small number of such complex processors, at one extreme, and a moderate to very large number of simpler processors, at the other. When combined with a high-bandwidth, interprocessor communication facility leads to significant simplification of the design process. However, two major roadblocks prevent the widespread adoption of such moderately to massively parallel architectures: the interprocessor communication bottleneck, and the difficulty and high cost of algorithm/software development.

One of the most important reasons for studying parallel computing architectures is to learn how to extract the best performance from parallel systems. Specifically, you must understand its architectures so that you will be able to exploit those architectures during programming via the standardized APIs.

This book would be useful for analysts, designers and developers of high-throughput computing systems essential for big data stream processing emanating from IoT-driven cyber-physical systems (CPS).

This pragmatic book:

• Devolves uniprocessors in terms of a ladder of abstractions to ascertain (say) performance characteristics at a particular level of abstraction
• Explains limitations of uniprocessor high performance because of Moore?s Law
• Introduces basics of processors, networks and distributed systems
• Explains characteristics of parallel systems, parallel computing models and parallel algorithms
• Explains the three primary categorical representatives of parallel computing architectures, namely, shared memory, message passing and stream processing
• Introduces the three primary categorical representatives of parallel programming APIs, namely, OpenMP, MPI and CUDA
• Provides an overview of Internet of Things (IoT), wireless sensor networks (WSN), sensor data processing, Big Data and stream processing
• Provides introduction to 5G communications, Edge and Fog computing

Parallel Computing Architectures and APIs: IoT Big Data Stream Processing discusses stream processing that enables the gathering, processing and analysis of high-volume, heterogeneous, continuous Internet of Things (IoT) big data streams, to extract insights and actionable results in real time.Application domains requiring data stream management include military, homeland security, sensor networks, financial applications, network management, web site performance tracking, real-time credit card fraud detection, etc.

Preface..............................................................................................................................................xv

Acknowledgments....................................................................................................................... xxi

Author..........................................................................................................................................xxiii

1 Uniprocessor Computers........................................................................................................1

2 Processor Physics and Moore’s Law.................................................................................. 17

Section I Genesis of Parallel Computing

3 Processor Basics..................................................................................................................... 37

4 Networking Basics................................................................................................................ 51

5 Distributed Systems Basics.................................................................................................65

Section II Road to Parallel Computing

6 Parallel Systems.....................................................................................................................83

7 Parallel Computing Models................................................................................................99

8 Parallel Algorithms............................................................................................................. 115

Section III Parallel Computing Architectures

9 Parallel Computing Architecture Basics......................................................................... 131

10 Shared Memory Architecture........................................................................................... 141

11 Message-Passing Architecture.......................................................................................... 157

12 Stream Processing Architecture....................................................................................... 167

Section IV Parallel Computing Programming

13 Parallel Computing Programming Basics...................................................................... 181

14 Shared-memory Parallel Programming with OpenMP.............................................. 213

15 Message Passing Parallel Programming with MPI......................................................223

16 Stream Processing Programming with CUDA, OpenCL, and OpenACC............... 239

Section V Internet of Things Big Data Stream Processing

17 Internet of Things (IoT) Technologies............................................................................255

18 Sensor Data Processing...................................................................................................... 287

19 Big Data Computing........................................................................................................... 313

20 Big Data Stream Processing..............................................................................................335

Epilogue: Quantum Computing................................................................................................ 361

References.....................................................................................................................................365

Index.............................................................................................................................................. 367

Vivek Kale has more than two decades of professional IT experience during which he has handled and consulted on various aspects of enterprise-wide information modeling, enterprise architectures, business process re-design, and, e-business architectures. He has been Group CIO of Essar Group, the steel/oil & gas major of India, as well as, Raymond Ltd., the textile & apparel major of India. He is a seasoned practitioner in enhancing business agility through digital transformation of business models, enterprise architecture and business processes, and, enhancing IT-enabled enterprise intelligence (EQ). He has authored books on Cloud Computing and Big Data Computing. He is also author of Big Data Computing: A Guide for Business and Technology Managers(CRC Press, 2016),Agile Network Businesses: Collaboration, Coordination, and Competitive Advantage (CRC Press 2017), and, Digital Transformation of Enterprise Architecture (CRC Press 2020).