Managing Big Data at the edge

Big data is often lauded as ushering in a new era of opportunity for businesses in all sectors and of all sizes. And while it’s undoubtedly true that in the right hands, the right data can be incredibly powerful, for the network administrator, all that data can cause more problems than it promises to solve. According to Cisco, the amount of data collected in the Internet of Things will reach 600 Zettabytes by 2020 – up from 145ZB in 2015.

All this data, and the arsenal of tools required to mine it from every part of the business, has the potential to have a dramatic impact on network performance if not handled with an appropriate network strategy.

So how do you ensure you facilitate big data collection, processing and analysis while maintaining the latency and reliability of your core network?

The answer, for many industry observers, lies in Edge computing. Seemingly at odds with recent moves toward a centralisation of business data in cloud and hybrid data centres, Edge treats the data from IoT devices separately to, and remotely of core business data. So why the shift in thinking?

The sheer scale of data threatens core functions

As demand on processing power and bandwidth increases, there will be an inevitable impact on the performance and availability of core business functions. Clearly, businesses cannot afford to impair the performance of ERP tools, Office 365 or other core apps for the sake of high demand but lower value traffic.

With Edge IT, that traffic and analysis can take place nearer to source, keeping its bandwidth and processing demand away from the central network and enabling more efficient, dedicated treatment of the data.

The nature of IoT data enables different treatment

The nature of much of the data collected and compiled within the big data “pot” often means it is more easily handled at the network edge. A lot of IoT information collected is “perishable” representing a specific point in time and with no long-term value.

IoT data does not need to be subject to the same storage and back-up processes of more business-critical information and can often be disposed of rapidly. In such cases, it makes no sense bringing this data into the central network environment and taking up valuable space.

More end-points = more security risk

Enterprise big data will be created by all kinds of tertiary connected devices – from vehicle telematics and manufacturing machinery to security pass usage and the temperature of the canteen fridge. And with such diversity of devices, so the number of potential security loopholes grows.

We’ve already seen malware targeting IoT devices such as security cameras and weaknesses being exploited in connected cars, so keeping this kind of data – and any related risk – at arm’s length of the core network, effectively quarantined from business critical apps and information, makes a lot of operational sense.

A hybrid approach – one that combines elements of on-premises, colocation and cloud-based infrastructure – offers an ideal solution for the challenges of IoT. Hybrid encompasses the concept of edge IT and delivers both the scalability and agility required to make the most of a big data world.

The level of security, connectivity and 24/7 support that typifies Cloud computing and colocation is a perfect fit for the increase in data traffic, storage, monitoring and analysis that come with the IoT. Non-essential data can be collated and structured at the edge of your infrastructure, before passing the intelligence that actually adds value to your organisation through to the central data centre.

Outsourcing the monitoring and management of your IoT infrastructure makes a lot of sense. With the diversity and sheer number of connected devices, maintaining everything in-house would quickly become time consuming and limit your in-house resource’s ability to focus on strategic tasks that add value to the organisation as a whole.

If you’re considering how hybrid or fully cloud-based Edge solutions can help your business overcome the challenges mentioned above, call us now on 01582 211530 or email