Handbook on Networked Multipoint Multimedia Conferencing and Multistream Immersive Telepresence using SIP Scalable Distributed Applications and Media Control over Internet

Handbook on Networked Multipoint Multimedia Conferencing and Multistream Immsersive Telepresence using SIP: Scalable Distributed Applications and Media Control over Internet is the first book to put together all IETF request for comments (RFCs), and the internet drafts standards related to the multipoint conferencing and immersive telepresence.

This book includes mandatory and optional texts of all standards in a chronological and systematic way almost with one-to-one integrity from the beginning to end, allowing the reader to understand all aspects of the highly complex real-time applications.

It is a book that network designers, software developers, product manufacturers, implementers, interoperability testers, professionals, professors, and researchers will find to be immensely useful. Practitioners and engineers in all spectrums who are concentrating on building the real-time, scalable, interoperable multipoint applications, can use this book to make informed choices based on technical standards in the market place, on all proprietary non-scalable and non-interposable products. This book will provide focus and foundation for these decision makers.

Contents

Preface...........................................................................................................................................xxxi

Author...........................................................................................................................................xxxv

Chapter 1 Scalable Networked Multipoint Multimedia Conferencing and Telepresence..............1

1.1 Networked Multipoint Multimedia Conferencing..............................................1

1.2 Basic Conferencing Protocol Architecture.........................................................4

1.3 Third-Party Multiparty Conferencing.............................................................. 12

1.4 Multistream Immersive Telepresence...............................................................23

1.5 Scalable Centralized Multipoint Multimedia Conferencing Systems

and Protocols Framework.................................................................................24

1.6 Terminology.....................................................................................................28

1.7 Summary..........................................................................................................28

1.8 Problems........................................................................................................... 41

References................................................................................................................... 42

Chapter 2 Centralized Conferencing System Architecture......................................................... 43

2.1 Introduction...................................................................................................... 43

2.2 Convention........................................................................................................ 43

2.3 Terminology..................................................................................................... 43

2.4 Overview..........................................................................................................44

2.5 Centralized Conferencing Data........................................................................ 45

2.6 Centralized Conferencing Constructs and Identifiers...................................... 47

2.7 Conferencing System Realization....................................................................50

2.8 Conferencing Mechanisms............................................................................... 55

2.9 Conference Scenario Realizations....................................................................56

2.10 Relationships between SIP and Centralized Conferencing System

Frameworks......................................................................................................69

2.11 Security Considerations....................................................................................69

2.12 Summary.......................................................................................................... 71

2.13 Problems........................................................................................................... 72

Chapter 3 Media Server Control Architecture............................................................................. 73

3.1 Introduction...................................................................................................... 73

3.2 Terminology..................................................................................................... 75

3.3 Architecture Overview..................................................................................... 75

3.4 NSIP Usage.......................................................................................................77

3.5 Media Control for IVR Services...................................................................... 78

3.6 Media Control for Conferencing Services........................................................80

3.7 Security Considerations....................................................................................85

3.8 Summary..........................................................................................................86

3.9 Problems...........................................................................................................87

Chapter 4 Conferencing Information Data Model for Centralized Conferencing.......................89

4.1 Introduction......................................................................................................89

4.2 Terminology.....................................................................................................89

4.3 Overview..........................................................................................................89

Comparison.......................................................................... 91

4.4 Data Model Definition......................................................................................92

4.5 RELAX NG Schema...................................................................................... 103

4.6 XML Schema Extensibility............................................................................ 103

4.7 XML Example................................................................................................ 103

4.8 Non-Normative RELAX NG Schema in XML Syntax (Appendix A of

RFC 6501)....................................................................................................... 103

4.9 Relax Schema in XML Syntax (Appendix B of RFC 6501).......................... 103

4.10 Security Considerations.................................................................................. 104

4.11 Summary........................................................................................................ 108

4.12 Problems......................................................................................................... 109

References................................................................................................................. 111

Chapter 5 Centralized Conferencing Manipulation Protocol.................................................... 129

5.1 Introduction.................................................................................................... 129

5.2 Conventions and Terminology........................................................................ 129

5.3 XCON Conference Control System Architecture.......................................... 129

5.4 Protocol Overview.......................................................................................... 131

5.5 CCMP Messages............................................................................................ 134

5.6 Example of CCMP in Action......................................................................... 153

5.7 Locating a Conference Server........................................................................ 162

5.8 Managing Notifications.................................................................................. 163

5.9 HTTP Transport............................................................................................. 165

5.10 Security Considerations.................................................................................. 166

5.11 XML Schema................................................................................................. 168

5.12 IANA Considerations..................................................................................... 169

5.13 Summary........................................................................................................ 176

5.14 Evaluation of Other Protocol Models and Transports Considered for

CCMP (Appendix A of RFC 6503)................................................................ 176

5.15 Problems......................................................................................................... 178

References................................................................................................................. 179

Chapter 6 Binary Floor Control Protocol.................................................................................. 181

6.1 Introduction.................................................................................................... 181

6.2 Terminology................................................................................................... 181

6.3 Scope.............................................................................................................. 181

6.4 Overview of Operation................................................................................... 183

6.5 Packet Format................................................................................................. 186

6.6 Transport........................................................................................................200

6.7 Lower-Layer Security..................................................................................... 201

6.8 Protocol Transactions..................................................................................... 201

6.9 Authentication and Authorization..................................................................202

6.10 Floor Participant Operations..........................................................................202

6.11 Chair Operations............................................................................................205

6.12.1 Requesting Information about Floors...............................................206

6.13 Floor Control Server Operations....................................................................208

6.14 Security Considerations.................................................................................. 214

6.15 BFCP Connection Establishment................................................................... 215

6.16 IANA Considerations..................................................................................... 218

6.17 Summary........................................................................................................ 221

6.18 Problems.........................................................................................................222

Chapter 7 XCON Notification Service.......................................................................................225

7.1 Introduction....................................................................................................225

7.2 Terminology...................................................................................................225

7.3 Notification Formats.......................................................................................225

7.4 Full Notifications............................................................................................225

7.5 Partial Notifications........................................................................................227

7.6 IANA Considerations.....................................................................................229

7.7 Security Considerations..................................................................................229

7.7.1 Connection Security..........................................................................230

7.7.2 Authorization Considerations ...........................................................230

7.8 Summary........................................................................................................230

7.9 Problems......................................................................................................... 231

Reference................................................................................................................... 231

Chapter 8 Media Channel Control Framework.......................................................................... 233

8.1 Introduction.................................................................................................... 233

8.2 CFW Capabilities........................................................................................... 233

8.3 Overview........................................................................................................ 235

8.4 Control Channel Setup................................................................................... 238

8.4.1 Control Client SIP UAC Behavior..................................................... 238

8.4.2 Control Server SIP UAS Behavior.................................................... 239

8.5 Establishing Media Streams – Control Client SIP UAC Behavior.................240

8.6 Control Framework Interactions..................................................................... 241

8.7 Response Code Descriptions.......................................................................... 247

8.8 Control Packages............................................................................................ 247

8.9 Formal Syntax................................................................................................249

8.10 Examples........................................................................................................ 251

8.11 Extensibility....................................................................................................254

8.12 Security Considerations..................................................................................254

8.12.1 Session Establishment.......................................................................254

8.13 IANA Considerations.....................................................................................256

8.14 Common Package Components (Appendix A of RFC 6230).........................256

8.15 Summary........................................................................................................ 257

8.16 Problems......................................................................................................... 257

Chapter 9 Mixer Package for Media Control Channel Framework........................................... 259

9.1 Introduction.................................................................................................... 259

9.2 Conventions and Terminology........................................................................260

9.3 Control Package Definition............................................................................260

9.4 Element Definitions........................................................................................ 262

9.5 Formal Syntax................................................................................................290

9.6 Examples........................................................................................................ 291

9.6.1 AS-MS Framework Interaction Examples........................................ 291

9.6.1.1 Creating a Conference Mixer and Joining a

Participant..........................................................................309

9.6.1.2 Receiving Active Talker Notifications...............................309

9.6.1.3 Conference Termination....................................................309

9.6.2 Mixing Examples.............................................................................. 310

9.6.2.1 Audio Conferencing........................................................... 310

9.6.2.2 Bridging Connections........................................................ 312

xvi Contents

9.6.2.3 Video Conferencing........................................................... 312

9.7 Security Considerations.................................................................................. 313

9.8 IANA Considerations..................................................................................... 315

9.9 Summary........................................................................................................ 315

9.10 Problems......................................................................................................... 315

References................................................................................................................. 316

Chapter 10 Media Session Recording.......................................................................................... 317

10.1 Use Cases and Requirements for SIP-Based Media Recording................... 317

10.2 Media Recording Architecture....................................................................324

10.3 Media Recording Metadata.......................................................................... 331

XML Body.................................................................344

10.3.9 XML Schema Definition for Recording Metadata....................... 345

10.3.10 Security Considerations................................................................348

10.3.11 IANA Considerations...................................................................349

10.4 Media Session Recoding Protocol...............................................................349

10.5 Summary...................................................................................................... 371

10.6 Problems....................................................................................................... 372

References................................................................................................................. 373

Chapter 11 Media Resource Brokering....................................................................................... 375

11.1 Introduction.................................................................................................. 375

11.2 Conventions and Terminology..................................................................... 375

11.3 Problem Discussion...................................................................................... 377

11.4 Deployment Scenario Options..................................................................... 378

11.5 MRB Interface Definitions.......................................................................... 381

11.6 MRB Acting as a B2BUA............................................................................405

11.7 Multimodal MRB Implementations.............................................................405

11.8 Relative Merits of Query Mode, IAMM, and IUMM.................................406

11.9 Examples......................................................................................................407

11.10 Media Service Resource Publisher Interface XML Schema....................... 425

11.11 Media Service Resource Consumer Interface XML Schema......................444

11.12 Security Considerations............................................................................... 461

11.13 IANA Considerations...................................................................................463

11.14 Summary......................................................................................................463

11.15 Problems.......................................................................................................464

References.................................................................................................................465

Chapter 12 Media Control Channel Framework for Interactive Voice Response ....................... 467

12.1 Introduction.................................................................................................. 467

12.2 Conventions and Terminology.....................................................................469

12.3 Control Package Definition..........................................................................469

12.4 Element Definitions...................................................................................... 471

12.5 Formal Syntax.............................................................................................. 511

12.6 Examples...................................................................................................... 534

12.8 IANA Considerations...................................................................................544

12.9 Using VoiceXML as a Dialog Language.....................................................544

12.10 Summary...................................................................................................... 549

12.11 Problems....................................................................................................... 550

Chapter 13 SIP Interface to VoiceXML Media Services............................................................ 551

13.1 Introduction.................................................................................................. 551

13.2 VoiceXML Session Establishment and Termination................................... 553

13.3 Media Support..............................................................................................560

13.4 Returning Data to the Application Server................................................... 563

13.5 Outbound Calling.........................................................................................565

13.6 Call Transfer................................................................................................. 565

13.7 Security Considerations............................................................................... 569

13.8 IANA Considerations...................................................................................569

13.9 Summary......................................................................................................569

13.10 Problems....................................................................................................... 570

References................................................................................................................. 571

Chapter 14 Media Resource Control Protocol Version 2............................................................. 573

14.1 Introduction.................................................................................................. 573

14.2 Conventions.................................................................................................. 574

14.3 Architecture................................................................................................. 575

14.4 MRCPv2 Basics........................................................................................... 576

14.5 MRCPv2 Specification................................................................................. 583

14.5.5 Events........................................................................................... 587

14.6 MRCPv2 Generic Methods, Headers, and Result Structure........................ 588

14.7 Resource Discovery..................................................................................... 597

14.8 Speech Synthesizer Resource...................................................................... 597

14.10 Recorder Resource....................................................................................... 656

14.11 Speaker Verification and Identification........................................................663

14.12 Security Considerations...............................................................................684

14.13 IANA Considerations...................................................................................686

14.14 Examples......................................................................................................686

14.15 ABNF Normative Definition........................................................................697

14.16 XML Schemas.............................................................................................706

14.17 Summary...................................................................................................... 711

14.18 Problems....................................................................................................... 713

References................................................................................................................. 714

Chapter 15 Media Control Channel Framework (CFW) Call Flow Examples........................... 715

15.1 Introduction.................................................................................................. 715

15.2 Conventions.................................................................................................. 715

15.3 Terminology................................................................................................. 716

15.4 A Practical Approach................................................................................... 716

15.4.1 State Diagrams............................................................................. 716

15.5 Control Channel Establishment................................................................... 718

15.6 Use-Case Scenarios and Examples.............................................................. 726

15.7 Media Resource Brokering..........................................................................792

15.8 Security Considerations...............................................................................820

15.9 Summary......................................................................................................827

15.10 Problems.......................................................................................................830

Reference................................................................................................................... 831

Chapter 16 Multistream Immersive Telepresence Conferencing Systems.................................. 833

16.1 Requirements for Telepresence Multistreams.............................................. 833

16.2 Use Cases for Telepresence Multistreams.................................................... 837

16.3 Framework for Telepresence Multistreams.................................................. 845

16.4 XML Schema for the CLUE Data Model....................................................885

16.5 CLUE Protocol............................................................................................. 933

16.6 CLUE Protocol Data Channel.....................................................................966

16.7 Mapping RTP Streams to CLUE Media Captures.......................................980

16.8 Session Signaling for Controlling Multiple Streams for Telepresence

(CLUE)......................................................................................................... 983

16.9 Summary......................................................................................................997

16.10 Problems..................................................................................................... 1001

References...............................................................................................................1002

Appendix....................................................................................................................................... 1005

Index............................................................................................................................................. 1013

Radhika Ranjan Roy has been an electronics engineer, United States Army Research, United States

Command, Control, Communications, Computers, Combat Systems, Intelligence, Surveillance,

and Reconnaissance (C5ISR)-S&TCD Laboratories (previously known as CERDEC), Aberdeen

Proving Ground (APG), Maryland, United States since 2009. Dr. Roy is leading research and development

efforts in the development of scalable large-scale SIP-based VoIP/Multimedia networks and

services, cybersecurity, artificial intelligence and machine/deep learning (AI/ML/DL) architecture,

mobile ad hoc networks (MANETs), Peer-to-Peer (P2P) networks, cybersecurity detecting application

software and network vulnerability, jamming detection, supporting an array of the Army/

Department of Defense’s Nationwide and Worldwide Warfighter Networking Architectures and

participating in technical standards development in Multimedia/Real-Time Services Collaboration,

IPv6, Radio Communications, Enterprise Services Management, and Information Transfer of

Department of Defense (DoD) Technical Working Groups (TWGs). He received his PhD in Electrical

Engineering with Major in Computer Communications from the City University of New York

(CUNY), NY, United States in 1984 and his MS in Electrical Engineering from the Northeastern

University, Boston, MA, United States in 1978. He received his BS in Electrical Engineering from

the Bangladesh University of Engineering & Technology (BUET), Dhaka, Bangladesh in 1967. He

was born in the renowned country town of Derai, Bangladesh.

Prior to joining CERDEC, Dr. Roy worked as the lead system engineer at CACI, Eatontown,

NJ from 2007 to 2009 and developed Army Technical Resource Model (TRM), Army Enterprise

Architecture (AEA), DoD Architecture Framework (DoDAF), and Army LandWarNet (LWN)

Capability Sets, and technical standards for Joint Tactical Radi