Ultra-High Performance Network Security Analytics

In the presence of cloud transformation and distributed virtualization across all enterprise networks, it is easy to forget that massive capacity growth continues across large segments of the global network infrastructure. And while many enterprise networks continue to scatter their perimeter chokepoints, the sheer volume of data traversing the global infrastructure continues to drive the need for not only high performance data capture and analytics, but ultra-high performance support for critical security functions. During my research for the TAG Cyber Security Annual – which many are now viewing as the un-Gartner Method of cyber security analysis, I had the opportunity to sit down with a longtime friend and colleague, Parag Pruthi of NIKSUN to discuss these topics. I’ve been friendly and familiar with Parag's team for many years, given our shared preference for IEEE meetings over the usual marketing gatherings. But after sitting down with Parag and his team in their offices in Princeton, it became clear to me that their deep understanding of networking technology and its relation to our rapidly changing cyber security market is worth some attention. Here is what I learned:

EA: Parag, does it surprise you that network capacity and performance needs have continued to grow in the presence of enterprise perimeter redesign?

PP: No, Ed, it doesn’t surprise me, because the vision of the designers of the Internet is only now being realized. Remember that the original idea of the Internet was to give users with terminals access to remote computing, and while service providers worked hard to handle high capacity needs, endpoints and servers in the local data center soon grew much more powerful. As a result, large-scale data rarely moved beyond the perimeter. As fiber optics unleashed bandwidth at the core, driving carrier backbones up to 100Gbps, redesigns started to occur where one was no longer limited to local computing, but computing could be spread across the Internet and this gave rise to a proliferation of thin mobile smart phones, cloud computing, and real-time services like video. The demand for more bandwidth in the last decade has led to a complete makeover of the structure of the Internet, from a pronounced hierarchy to a flat structure that blurs the line between network edge and core. During this evolution, the need for security has increased as today’s enterprises are faced with more sophisticated and damaging attacks to enterprise data centers, mobility, cloud, and IoT. And with the proliferation of Big Data, data centers handle more traffic, which fuels the need for ultra-high performance support for critical security functions such as loss-less packet capture and support for analytics.

EA: How fast can a packet-capturing engine go in trying to keep up with a massive network load?

PP: When our team first started developing solutions for high-speed capture, we met the challenge of economically supporting 100Mbps, even though most observers said it could not be done. We had similar success at 1Gbps and 10Gbps networks, despite skepticism that such packet rates could be handled. Our success was driven by a commitment to building a robust solution that would take full advantage of Moore’s Law and that would employ techniques such as stream computing, parallel processing, multi-threading, data base management, and Big Data analytics. The result was our Supreme Eagle product, which is a single-unit modular hardware platform, engineered with the latest high-performance processors, ultra-fast memory, and our next-generation core IP integrated within its own interface line cards. It ensures full line rate data capture and processing ranging from 20 Gbps to 100+ Gbps without dropping a single packet and supports storage up to 10 PB. In comparison with existing industry offerings, Supreme Eagle requires considerably less rack space and power consumption, and delivers more processing capacity and storage than comparable solutions.

EA: Do you see corresponding advances in the quality of network security analytics? That is, as more data is captured at higher rates, is the security analysis still any good?

PP: Security analytics can be compared to fishing. That is, if there are many fish per unit volume of water, then one simply casts a net and pulls out many fish. Similarly, if there are many security issues in a network, then it is easy to cast a security net and identify many threats. But if the density of fish – or attacks in a network – is relatively low, then more intelligent methods are required, and that is where network security analytics comes in handy. For example, advanced cyber attackers, like clever fish, know what traps and signatures are set, and can devise mechanisms to evade security. This can include taking advantage of the distributed nature of a typical large area network. So, we distribute the analytics, scaling it to big and small nodes across multiple passes, not to mention supporting analytics on virtual machines. An additional problem involves keeping up with attacks embedded in large networks in real time. The NIKSUN team approached this problem by not rushing into the market with a point solution, but rather developing real-time management solutions that focus on streaming data, distributed across multiple sensors. For such distributed streaming data, popular frameworks such as Apache Hadoop that require the different streams to be transferred to a single location for centralized storage and analysis of the data are simply no longer an option. Instead, we focused on supporting efficient analysis of the hyper data our product generates. The basic challenge posed by such distributed streaming data is how to mine and analyze highest-quality traffic data that is collected in different geographically-dispersed locations and is made available to the analysts in two basic forms — as high-velocity sets of streaming data to be used for real-time analytics and as high-volume, static and highly-structured datasets to be used for network forensics and back-in-time analysis.

EA: Do you see any differences in the way network service providers try to capture data as they shift their networks to SDN?

PP: With SDN, you can equip switches with predefined functions in hardware and let the controller select them for different measurement tasks. In this sense, there will be a role for SDN with respect to traffic monitoring, but this role will be likely quite limited because switch hardware remains prime real estate and severely limits the monitoring tasks that can be performed in the data plane. In particular, lossless packet capture at Gbps line rates for network forensics and back-in-time analysis is a monitoring task for which SDN is ill-suited. On the other hand, SDN can play a dominant role in data analysis for real-time control decisions. In fact, when coupled with an SDN controller, network data analytics can detect and identify nefarious traffic in (close-to) real time. For network security, such solutions enable the timely detection of network attacks, followed by swift and timely mitigation. It is in this role where I see SDN playing a critical role as network providers embrace SDN and reap the benefits offered by a programmable data plane, in general, and programmable switches, in particular.

EA: What are some trends that you are predicting in the coming years? For example, do you see enterprise security teams ever getting to the point where they can stop advanced attacks from nation-states?

PP: I see increased investment, but I also see considerable hype. This can include reports that autonomic defensive systems are just around the corner, or that deep learning-based artificial intelligence will revolutionize cyber security. The reality is that the holy grail of cyber security – namely, the detection and reverse engineering of attacks that have never been seen – will continue to require human involvement. NIKSUN’s Supreme Eagle supports automation as appropriate, but also enables the domain expert to interact with whatever data is needed to learn about the unknown. If the right balance is struck between using self-learning systems and domain expert-driven discovery and exploration, then future enterprise security teams will be able to detect attacks from nation-states and stop them in their tracks.