NetFlow and Other Telemetry Sources for Big Data Analytics for Cyber Security
As discussed in Chapter 1, “Introduction to NetFlow and IPFIX,” NetFlow provides detailed network telemetry that allows the administrator to:
- See what is actually happening across your entire network
- Regain control of your network, in case of denial-of-service (DoS) attack
- Quickly identify compromised endpoints and network infrastructure devices
- Monitor network usage of employees, contractors, or partners
- Obtain network telemetry during security incident response and forensics
- Detect firewall misconfigurations and inappropriate access to corporate resources
As previously mentioned, NetFlow data can grow to tens of terabytes of data per day in large organizations, and it is expected to grow over the years to petabytes. However, many other telemetry sources can be used in conjunction with NetFlow to identify, classify, and mitigate potential threats in your network. Figure 5-1 shows examples of these telemetry sources and how they “feed” into a collection engine.
)
Figure 5-1 NetFlow and Other Telemetry Sources
As illustrated in Figure 5-1, NetFlow data, syslog, SNMP logs, server and host logs, packet captures, and files (such as executables, malware, exploits) can be parsed, formatted, and combined with threat intelligence information and other “enrichment data” (network metadata) to perform analytics. This process is not an easy one; this is why Cisco has created an open source framework for big data analytics called Open Security Operations Center (OpenSOC). The following section provides an in-depth look at the OpenSOC framework.