Enterprise Threats: Big Data and Cyber Security


Building-security-300x243Many people are aware of the fact that Big Data has the potential to significantly enhance business and operational processes. Yet, its scalability also has substantial boons for cyber security, the consequences of which can either make or break the enterprise.
 
Due to the massive quantities and variation of data sources that organizations can process via Big Data, it is now economically feasible to capture, store, and analyze cyber security data for months at a time. Prior to the advent of Big Data technologies, most conventional security appliances could only provide point solutions that filtered data for a single threat such as malware, denial of service, or intrusion prevention – for a matter of minutes.
 
Organizations can now preserve cyber security data via two means – through the use of a dedicated repository or through sophisticated probing technology (or perhaps both) – and run analytics on long-term patterns and threats that become manifest over time. Thus, they can continue to stream for and deter threats in close to real time and augment that capability with a deeper, profound analysis of more sophisticated threats. If you’ve bet the enterprise on the network and its data, extra security doesn’t hurt.
 
Many people are aware of the fact that Big Data has the potential to significantly enhance business and operational processes. Yet, its scalability also has substantial boons for cyber security, the consequences of which can either make or break the enterprise.
 
Due to the massive quantities and variation of data sources that organizations can process via Big Data, it is now economically feasible to capture, store, and analyze cyber security data for months at a time. Prior to the advent of Big Data technologies, most conventional security appliances could only provide point solutions that filtered data for a single threat such as malware, denial of service, or intrusion prevention – for a matter of minutes.
 
Organizations can now preserve cyber security data via two means – through the use of a dedicated repository or through sophisticated probing technology (or perhaps both) – and run analytics on long-term patterns and threats that become manifest over time. Thus, they can continue to stream for and deter threats in close to real time and augment that capability with a deeper, profound analysis of more sophisticated threats. If you’ve bet the enterprise on the network and its data, extra security doesn’t hurt.
 
According to Jay Desai, senior vice-president of business development at Xtreme Data (a software company based on a proprietary ANSI SQL database engine with Big Data scalability which also consults for customers regarding security issues):
 
“In the last 15 years, if you take a typical large enterprise, it may have deflected 30 percent of its business from bricks and mortar to online. In the brick and mortar arena, it probably spent hundreds of millions of dollars on perimeter defenses and security measures like passes in and out and all kinds of things. But in the digital world, it probably hasn’t done as much of that. And in the digital world the threat is from everywhere. Anybody from anywhere in the world can enter the company’s defenses.”
 
Cyber Security Repository
Most security appliances currently operate autonomously and are dedicated to detecting a single type of threat. They analyze data as it comes in and retain it for a few fleeting minutes before acting on it. Perhaps even more tenuous is the fact that the data that each point solution analyzes is never merged with the others, when doing so could present a greater overall picture of threats.
 
Utilizing Big Data technologies to set up a dedicated repository for cyber security threats substantially increases an organization’s defenses. Not only can organizations analyze up to three months of data (“Even today, it’s not economically feasible to store more than 60 or 90 days’ worth of data before you have to throw it away” said Xtreme Data CEO Ravi Chandran) at a time – which enables them to discern historical trends – but they can also combine data types from various point solutions for a more comprehensive overview of security concerns. Best of all, they are able to combine historical data with real-time data to fully perceive indiscriminate patterns that may be of considerable consequence. Desai discussed some of the more evasive cyber security threats and this particular option’s usefulness against them.
 
“If you’ve got sleeper cells, something that someone’s planted that’s just going to sit there for months at different points in the network, now you have information to go and detect those things in the future – if you have collected the data. But if you haven’t collected the data, you’re left to your own devices.”
 
Probes
The ultimate way to supplement the repository approach is through the use of probes. Probes are a technology that is available either as hardware or software – most organizations use the former – that was typically reserved for sophisticated intelligence entities in the public sphere. They are systematically placed in the network to enable users to view large quantities of data in real time. Probes inspect data by the packet and offer views of both content and headers. Data from each packet is collected (ideally in a central repository as described above or in a conventional warehouse) and can be analyzed with other historical data via advanced analytics to allow for insight into potential security lapses.
 
Probes inspect the data flow both ways and give users a view of data sources and other pertinent information for monitoring threats. The amount of data they collect is significantly greater than that of point solutions and allows for analytics from a variety of different threats, as opposed to just one. By using Big Data technologies to maintain the history of deep packet inspections facilitated by probes, organizations can gain a comprehensive overview of potential threats with both real-time and historical analysis. Desai remarked upon the effectiveness of this approach.
 
“In the past this was limited to the highest echelons in the intelligence communities, but these days, most security appliance vendors are doing some sort of packet inspection when they sell you an appliance of sorts. So [the technology is] becoming a little bit more pervasive.”
 
Some of the many examples of vendors and security platforms/appliances that include deep packet inspection include:
Big Data Risks
The lone drawback related to security and Big Data pertains to issues of access and control that stem from simply ingesting so much data. These problems largely pertain to the particular technology an organization may use for its Big Data. Depending on where such data is stored – a fairly common method involves Hadoop, an open source platform – various departments may have access to data that they traditionally have not had when dealing with conventional relational databases.
 
Data put into Hadoop is visible to developers around the world, which certainly limits its security. These issues have been substantially mitigated, however, by a number of vendors (including Oracle, Cloudera, DataStax, and others) implementing warehouse and storage technologies that work in conjunction with Hadoop to protect an enterprise’s proprietorial data. In many of these solutions, Hadoop is simply used as a back-end data store that doesn’t necessarily require direct access. Desai commented on this trend:
 
“In most organizations, IT developers don’t get access to look at production data. They work with development data sets and test data sets and produce products that get turned over to users who then have access to production data. That’s not the case with Hadoop and a lot of enterprises. So a lot of enterprises are now stepping back from Hadoop a little and putting those things on hold and putting appropriate controls in place.”
 
The Future of Cyber Security
At present, cyber security’s role as a driver for Big Data is limited. Most organizations – whether they are utilizing Big Data or not – have  been functioning fairly effectively with real-time streaming appliances that do not preserve security data or incorporate it with their other data. But as cyber security threats get more sophisticated and as Big Data itself gains popularity with the projected expansion of sensor data that some professionals have identified for the near future, using Big Data scalability to store cyber security data may very well become ubiquitous. Security appliances are already looking to develop measures to utilize historical data with real time tools; the next logical progression is to correlate such data with other data types for a complete analysis.
 
There are also other benefits associated with using Big Data to enhance security. According to Desai, Big Data analytics can successfully predict hardware failures and are readily used throughout the automotive industry to predict malfunctioning of various electrical and engine-related components. They can do the same for hardware in an enterprise’s network. Desai summarized Big Data’s effect on cyber security issues.
 
“The biggest thing is now companies are recognizing that you can only achieve so much by putting filtering devices to protect yourself—you do have to collect data. Whereas in the past this was impossible because of the sheer cost; now it is the case that it’s becoming practical.”