Data Analytics for IT Networks: Developing Innovative Use Cases

Use data analytics to drive innovation and value throughout your network infrastructure

Network and IT professionals capture immense amounts of data from their networks. Buried in this data are multiple opportunities to solve and avoid problems, strengthen security, and improve network performance. To achieve these goals, IT networking experts need a solid understanding of data science, and data scientists need a firm grasp of modern networking concepts. Data Analytics for IT Networks fills these knowledge gaps, allowing both groups to drive unprecedented value from telemetry, event analytics, network infrastructure metadata, and other network data sources.

Drawing on his pioneering experience applying data science to large-scale Cisco networks, John Garrett introduces the specific data science methodologies and algorithms network and IT professionals need, and helps data scientists understand contemporary network technologies, applications, and data sources.

After establishing this shared understanding, Garrett shows how to uncover innovative use cases that integrate data science algorithms with network data. He concludes with several hands-on, Python-based case studies reflecting Cisco Customer Experience (CX) engineers’ supporting its largest customers. These are designed to serve as templates for developing custom solutions ranging from advanced troubleshooting to service assurance.

• Understand the data analytics landscape and its opportunities in Networking
• See how elements of an analytics solution come together in the practical use cases
• Explore and access network data sources, and choose the right data for your problem
• Innovate more successfully by understanding mental models and cognitive biases
• Walk through common analytics use cases from many industries, and adapt them to your environment
• Uncover new data science use cases for optimizing large networks
• Master proven algorithms, models, and methodologies for solving network problems
• Adapt use cases built with traditional statistical methods
• Use data science to improve network infrastructure analysisAnalyze control and data planes with greater sophistication
• Fully leverage your existing Cisco tools to collect, analyze, and visualize data
  

Table of Contents

    Foreword xvii
    Introduction: Your future is in your hands! xviii
Chapter 1 Getting Started with Analytics 1
    What This Chapter Covers 1
        Data: You as the SME 2
        Use-Case Development with Bias and Mental Models 2
        Data Science: Algorithms and Their Purposes 3
    What This Book Does Not Cover 4
        Building a Big Data Architecture 4
        Microservices Architectures and Open Source Software 5
        R Versus Python Versus SAS Versus Stata 6
        Databases and Data Storage 6
        Cisco Products in Detail 6
    Analytics and Literary Perspectives 7
        Analytics Maturity 7
        Knowledge Management 8
        Gartner Analytics 8
        Strategic Thinking 9
        Striving for “Up and to the Right” 9
        Moving Your Perspective 10
        Hot Topics in the Literature 11
    Summary 12
Chapter 2 Approaches for Analytics and Data Science 13
    Model Building and Model Deployment 14
    Analytics Methodology and Approach 15
        Common Approach Walkthrough 16
    Distinction Between the Use Case and the Solution 18
    Logical Models for Data Science and Data 19
        Analytics as an Overlay 20
        Analytics Infrastructure Model 22
    Summary 33
Chapter 3 Understanding Networking Data Sources 35
    Planes of Operation on IT Networks 36
        Review of the Planes 40
    Data and the Planes of Operation 42
        Planes Data Examples 44
        A Wider Rabbit Hole 49
        A Deeper Rabbit Hole 51
    Summary 53
Chapter 4 Accessing Data from Network Components 55
    Methods of Networking Data Access 55
        Pull Data Availability 57
        Push Data Availability 61
        Control Plane Data 67
        Data Plane Traffic Capture 68
        Packet Data 70
        Other Data Access Methods 74
    Data Types and Measurement Considerations 76
        Numbers and Text 77
        Data Structure 82
        Data Manipulation 84
        Other Data Considerations 87
        External Data for Context 89
    Data Transport Methods 89
        Transport Considerations for Network Data Sources 90
    Summary 96
Chapter 5 Mental Models and Cognitive Bias 97
    Changing How You Think 98
    Domain Expertise, Mental Models, and Intuition 99
        Mental Models 99
        Daniel Kahneman’s System 1 and System 2 102
        Intuition 103
    Opening Your Mind to Cognitive Bias 104
        Changing Perspective, Using Bias for Good 105
        Your Bias and Your Solutions 106
        How You Think: Anchoring, Focalism, Narrative Fallacy, Framing, and Priming 107
        How Others Think: Mirroring 110
        What Just Happened? Availability, Recency, Correlation, Clustering, and Illusion of Truth 111
        Enter the Boss: HIPPO and Authority Bias 113
        What You Know: Confirmation, Expectation, Ambiguity, Context, and Frequency Illusion 114
        What You Don’t Know: Base Rates, Small Numbers, Group Attribution, and Survivorship 117
        Your Skills and Expertise: Curse of Knowledge, Group Bias, and Dunning-Kruger 119
        We Don’t Need a New System: IKEA, Not Invented Here, Pro-Innovation, Endowment, Status Quo, Sunk Cost, Zero Price, and Empathy 121
        I Knew It Would Happen: Hindsight, Halo Effect, and Outcome Bias 123
    Summary 124
Chapter 6 Innovative Thinking Techniques 127
    Acting Like an Innovator and Mindfulness 128
        Innovation Tips and Techniques 129
    Developing Analytics for Your Company 140
        Defocusing, Breaking Anchors, and Unpriming 140
        Lean Thinking 142
        Cognitive Trickery 143
        Quick Innovation Wins 143
    Summary 144
Chapter 7 Analytics Use Cases and the Intuition Behind Them 147
    Analytics Definitions 150
    How to Use the Information from This Chapter 151
        Priming and Framing Effects 151
        Analytics Rube Goldberg Machines 151
    Popular Analytics Use Cases 152
        Machine Learning and Statistics Use Cases 153
        Common IT Analytics Use Cases 170
        Broadly Applicable Use Cases 199
        Some Final Notes on Use Cases 214
    Summary 214
Chapter 8 Analytics Algorithms and the Intuition Behind Them 217
    About the Algorithms 217
        Algorithms and Assumptions 218
        Additional Background 219
    Data and Statistics 221
        Statistics 221
        Correlation 224
        Longitudinal Data 225
        ANOVA 227
        Probability 228
        Bayes’ Theorem 228
        Feature Selection 230
        Data-Encoding Methods 232
        Dimensionality Reduction 233
    Unsupervised Learning 234
        Clustering 234
        Association Rules 240
        Sequential Pattern Mining 243
        Collaborative Filtering 244
    Supervised Learning 246
        Regression Analysis 246
        Classification Algorithms 248
        Decision Trees 249
        Random Forest 250
        Gradient Boosting Methods 251
        Neural Networks 252
        Support Vector Machines 258
        Time Series Analysis 259
    Text and Document Analysis 262
        Natural Language Processing (NLP) 262
        Information Retrieval 263
        Topic Modeling 265
        Sentiment Analysis 266
    Other Analytics Concepts 267
        Artificial Intelligence 267
        Confusion Matrix and Contingency Tables 267
        Cumulative Gains and Lift 269
        Simulation 271
    Summary 271
Chapter 9 Building Analytics Use Cases 273
    Designing Your Analytics Solutions 274
    Using the Analytics Infrastructure Model 275
    About the Upcoming Use Cases 276
        The Data 276
        The Data Science 278
        The Code 280
    Operationalizing Solutions as Use Cases 281
        Understanding and Designing Workflows 282
    Tips for Setting Up an Environment to Do Your Own Analysis 282
    Summary 284
Chapter 10 Developing Real Use Cases: The Power of Statistics 285
    Loading and Exploring Data 286
    Base Rate Statistics for Platform Crashes 288
    Base Rate Statistics for Software Crashes 299
    ANOVA 305
    Data Transformation 310
        Tests for Normality 311
        Examining Variance 313
    Statistical Anomaly Detection 318
    Summary 321
Chapter 11 Developing Real Use Cases: Network Infrastructure Analytics 323
    Human DNA and Fingerprinting 324
    Building Search Capability 325
        Loading Data and Setting Up the Environment 325
        Encoding Data for Algorithmic Use 328
        Search Challenges and Solutions 331
    Other Uses of Encoded Data 336
    Dimensionality Reduction 337
    Data Visualization 340
    K-Means Clustering 344
    Machine Learning Guided Troubleshooting 350
    Summary 353
Chapter 12 Developing Real Use Cases: Control Plane Analytics Using Syslog Telemetry 355
    Data for This Chapter 356
    OSPF Routing Protocols 357
    Non-Machine Learning Log Analysis Using pandas 357
        Noise Reduction 360
        Finding the Hotspots 362
    Machine Learning—Based Log Evaluation 366
        Data Visualization 367
        Cleaning and Encoding Data 369
        Clustering 373
        More Data Visualization 375
        Transaction Analysis 379
    Task List 386
    Summary 387
Chapter 13 Developing Real Use Cases: Data Plane Analytics 389
    The Data 390
    SME Analysis 394
    SME Port Clustering 407
    Machine Learning: Creating Full Port Profiles 413
    Machine Learning: Creating Source Port Profiles 419
    Asset Discovery 422
    Investigation Task List 423
    Summary 424
Chapter 14 Cisco Analytics 425
    Architecture and Advisory Services for Analytics 426
    Stealthwatch 427
    Digital Network Architecture (DNA) 428
    AppDynamics 428
    Tetration 430
    Crosswork Automation 431
    IoT Analytics 432
    Analytics Platforms and Partnerships 433
    Cisco Open Source Platform 433
    Summary 434
Chapter 15 Book Summary 435
    Analytics Introduction and Methodology 436
    All About Networking Data 438
    Using Bias and Innovation to Discover Solutions 439
    Analytics Use Cases and Algorithms 439
    Building Real Analytics Use Cases 440
    Cisco Services and Solutions 442
    In Closing 442
Appendix A Function for Parsing Packets from pcap Files 443
9781587145131, TOC, 9/19/18

Submit Errata