One of the biggest challenges the enterprise faces with the advent of big data and the internet of things is what to do about the network edge. Emerging data patterns require an edge that is not only distributed across a wide area, largely on third-party infrastructure that is beyond the direct control of IT, but one that can handle a much broader set of responsibilities than traditional security and access.

This new edge is proving to be a critical component in the need to maintain connectivity to a vast collection of digital devices and to deliver the results of processed data quickly enough to be of value in an increasingly fast-pace data ecosystem.

In the old days (maybe five years ago), the edge was a relatively simple concept for all but the most mobile-facing enterprises. Wherever IT infrastructure was housed, be it the central office, a remote site or the cloud, that was the edge. Moving data to and from the edge involved the use of clearly defined network infrastructure and data management systems, nearly all of which were preconfigured and tested regularly for optimal performance.

That all falls apart with the IoT, says HPE’s Dr. Tom Bradicich. The new edge is wherever the connected “things” are, which can be in customers’ homes, cars or even on their very bodies. In order to support the kind of end-to-end solutions that will be needed in the emerging digital economy, the edge will have to perform a variety of functions, such as data acquisition and forwarding, processing and analysis, plus enable high degrees of autonomy and self-correction. As well, the enterprise will be unable to maintain and service this edge in the traditional hands-on manner, so it will have to lend itself to extensive remote monitoring and maintenance.

A key element in the new edge is the localized data center or micro data center. With today’s digital consumer expecting always-available, real-time performance for a wide range of apps and services, the ability to push both processing and storage as deep into the network as possible becomes paramount. As Schneider Electric’s Kevin Brown notes, the sheer amount of data generated by the IoT is enough to overwhelm even the largest centralized data facility, so the only way to effectively handle it all is to maintain the bulk of data activity on the perimeter, gathering only what is necessary to the traditional data center.

This will require the edge to be populated with the latest in intelligent analytics that can absorb and condition enormous amounts of unstructured data, says Tibco Software’s Maurizio Canton. We are quickly entering a world in which non-technical users are not only creating their own data, but their own applications as well. And “users” in this case do not necessarily refer to the human variety, as much of the IoT traffic is expected to come from machine-to-machine communications. The edge, then, must not only gather and distribute data, but also understand what it means and how it can produce the most value.

The key challenge, of course, will be to match the configuration of the edge to the enterprise functions they support, and then to augment them with appropriate levels of networking, security and other tools to maintain adequate reliability and connectivity – not just to centralized resources but to other edge systems as well. In all likelihood, this will require an abstract network architecture encompassing both software-defined networking and network functions virtualization capable of automating such functions as service demarcation, performance assurance and synchronization. In this way, organizations will be able to amass a wide range of edge-based tools and platforms but manage it as an integrated environment from a central location.

In a way, the new data edge represents the culmination of a decades-long trend toward a fully immersive digital environment. No longer will we interact with data through just computers or even handheld devices but in the normal activities of our everyday lives: shopping, driving, reading …

In this world, the enterprise will need every tool at its disposal to differentiate its services over competitors’, and the best way to do that is through a robust, dynamic presence on the edge.