The growth of cloud-based over-the-top (OTT) applications and IoT traffic are having a profound effect on the way networks are being designed and used. A modern network architecture is a fluid and dynamic mesh, driven by software, heavily automated and designed to handle enormous volumes of traffic with minimal latency and maximum uptime.
This revolution in networking is greatly impacting how operators analyse and optimise network traffic. The way they approach network analytics, and make sense of the enormous amount of network data, must mirror the seamless and dynamic flow of the traffic itself.
“If the goal is understanding how the network as a whole is performing, sifting through isolated data lakes is completely ineffective,” argues Naim Falandino, chief scientist at Nokia Deepfield. “Operators need a way to combine multi-dimensional data from the cloud and network in one scalable software platform so they can slice and dice in real time to uncover and resolve any network configuration issues that may lead to poor streaming quality.”
This allows operators to evolve from a reactionary mode of operations to proactively monitoring their networks so that the right action can be taken—before a customer even has the chance to complain.
A changed world
The business of analytics was, until quite recently, a very different matter. Going back only a few years, most of what travelled over a data network was web traffic of the simplest kind. End users were happy as long as they had access to a browser and a connection with minimal jitter.
There has since been an explosion not only in the volumes of traffic but also a fundamental shift in the type of traffic that networks are expected to handle. With traffic from cloud applications, video and services dominating network traffic, and the emergence of machine to machine (m2m) communications, we’re seeing massive changes in the way networks are being used, and in the value end users place on a dependable connection to their data.
Too much of what is going on in this changed paradigm is not easily visible to the providers of data services: “Operators often have no visibility about exactly what traffic is flowing across their network,” says Falandino. “Cloud-based applications and high definition video content is increasingly delivered as encrypted or encapsulated flows. As OTT traffic is becoming more opaque, it prevents network operators from optimising network resources for high volume or low latency requirements, or detecting potential security threats. Network operators need new ways to help them identify OTT traffic and understand where it is coming from, like caller ID for cloud services on the internet.”
They are demanding, he believes, tools that help them to understand the internet just as much as they need tools to understand their own network: “The only way that operators can excel in this era of cloud and IoT is with scalable tools that can track how OTT data flows into their network, through their network, and how it reaches the access edge—even if that traffic is encrypted.”
Traditional analytics products and methods do not answer this need. Since the early days of data networking, operators have used deep packet inspection (DPI) tools to analyse network traffic, but with traffic demand growing at such a rapid rate, DPI cannot keep up.
It is simply cost prohibitive to subject all OTT traffic to deep pack inspection, which leaves operators completely unequipped to monitor network performance and detect service quality issues. Operators may literally have petabytes of big data at their fingertips, but no single lens through which it can all be viewed. Current analysis methods are labour intensive, prone to human error, and do not provide crucial information—for example—what is causing traffic to back up or exactly where capacity is needed. The result can be a costly guessing game which seldom resolves any problems and can even make a challenging situation worse.
Why does any of this matter? “It matters because ‘slow’ is the new ‘down’ for subscribers,” says Falandino. “If the network fails to provide high quality streaming of OTT traffic, the customer will complain and leave for the competition. But since the ultimate goal is subscriber happiness, an operator must meet their customers’ needs in the most cost effective manner.”
There are worse things than the odd disgruntled customer that can befall the operator without proper network visibility: “The way we are managing networks today is creating security holes,” warns Falandino. “We’re at the dawn of a new era of DDoS, and we’ve seen some massive attacks happen already. These attacks are essentially being launched from unsecured cloud services and cloud-based IoT devices. This puts operators under even greater pressure to understand cloud and IoT traffic and its flow, and they need to do this in real time.”
IoT devices indeed are at the heart of the network security challenge, and all too often beyond the reach of standard analytics methodologies. The volume of these devices and their inherent lack of security make them a liability. The proper analysis of the right big data is needed to indicate how these devices are utilising the network, for example to see if they are operating normally or not.
The right tools
The answer is for operators to turn to a whole new generation of advanced analytics tools that provide the insight to automate network management and protection. With advanced analytics, operators can achieve pinpointed real-time trouble shooting across the entire network. Even before a customer raises a problem, they have a holistic view of their entire network in one place so that trouble can be anticipated, not simply allowed to happen before any action can take place.
“The operator gets an alert when something needs action,” explains Falandino. “They can be alerted to potential DDoS activity as well. If they know what normal traffic patterns look like, they can spot something out of the ordinary. Advanced analytics delivers fine grained, business-level information that the operator can act on immediately, ahead of any problems and ahead of any customer complaints.”
Network analytics based on a big data view of the network can understand immediately when half a million infected devices start acting out of character, for example sending repeated DNS requests to servers that they normally have no business contacting.
Taking days to analyse traffic, which is by no means an atypical time lag, is far too long: “If you’re deploying into data centres, you need to do that analysis in real-time, from identification of the problem to instant action,” says Falandino. “Advanced analytics also supercharges SDN. SDN lets you steer traffic and move routes around, but you need the right information or it can make the network unstable.”
Webscale companies like Google and Amazon already know this. They have no time to laboriously poll data, but instead need automatic, event-driven updates telling them when failures occur and where. Because their subscribers are intolerant of performance issues, they need to have a real-time view of their operational state of the entire network, with the ability to precisely troubleshoot and remedy those issues.
Network operators must rely on modern analytics to automatically ingest data from both the cloud and the network, and combine them into one system for deep analysis of the entire network at once. By slicing and dicing this multi-dimensional data, operators can pinpoint and cost effectively resolve service issues, in advance of a customer having the chance to complain.
“A key advantage of a software-based analytics platform such as Nokia Deepfield is that you can leverage cloud-based compute and storage platforms to virtually scale infinitely, and run multiple tasks in real time,” enthuses Falandino. “You can revamp your OTT user experience and reinvent your DDoS security strategy by effectively blocking multi-vector attacks at the network perimeter, instead of losing precious time on network forensics and damage control while customers and infrastructure resources are under attack.”
Advanced analytics based on telemetry, he says, will not only monitor points on a network, but literally everything flowing to and through the network, end-to-end: “This data will be streamed in real-time, not be polled every five minutes,” he adds.
“The granularity and richness of this data will allow it to be reported and analysed in detail to provide proactive, actionable data, allowing operators to reach the ultimate goal—integration with software-defined networks, with real-time analytics providing the intelligence to make them truly insight-driven and dynamic.”
It’s about marrying the cloud and the network together, he explains: “Just as network silos and overlays are converging into a seamless dynamic mesh, we need to join the isolated data lakes that currently exist, using big data to solve issues in real time.” 
Get more from your IP network: