Data Center Network Automation has a Data Problem

Network data is essential to network automation, but too many IT organizations neglect it. Nearly 48% of companies have data center network automation solutions that require at least partially on manual data collection, according to new research from Enterprise Management Associates (EMA).

What exactly do we mean by manual data collection? Think about what a network administrator needs to know before implementing a change in a network automation tool. First, they will need to review standards for network device configurations to make sure any change they make is compliant with those standards. They may need to review an IP address management (IPAM) system to find an IP address to assign to a new device connecting to the network. They may need to review inventory and topology data to identify the device they need to automate and where it is located. They may need access to a database of security policies to ensure a change doesn’t endanger the business. They may need operational data, like device metrics and network flows, to understand the current state of the network.

Now imagine that an administrator has to collect all that data manually before making a change.

"We are somewhat manual. We have [a commercial IPAM) tool, but it's not fit for a purpose, and it's always out of date," a network engineer with a $60 billion consulting company told EMA. “So, engineers revert to using spreadsheets.”

Manual Data Collection Hinders Network Automation

Anyone who has searched for and copy-and-pasted data from a spreadsheet knows the process is slow and error-prone. This is not an ideal situation. EMA recently surveyed 359 people involved in data center network automation for our report, The Future of Data Center Network Automation. In that survey, more than 51% of technology professionals told EMA that manual data collection degrades the effectiveness of their data center network automation tools. 

EMA asked those technology professionals to describe the negative impacts of manual data collection. More than 56% said it simply takes too long. Automation is supposed to make network operations more efficient, but when users of automation get bogged down in gathering data, those efficiencies are lost.

A network design engineer with a $25 billion pharmaceutical company told EMA that his organization was able to make its network automation tools more efficient through automated data collection. His team has been integrating various data repositories with their network automation tools. “We used to spend a month to get a switch turned up, but with improved data collection, it just takes a day or two now.”

Nearly 45% said they can’t see or verify changes that they push through network automation. In other words, they can’t effectively gather data that tells them whether the change they pushed through automation actually happened and whether the network is still operating as intended.

More than 41% said manual data gathering opens up security vulnerabilities on their networks. Either they make mistakes, or they simply lack enough visibility into how automated changes affect their overall security posture.

Finally, almost 35% said they experience performance degradation due to manual data collection. This reflects a lack of visibility into how automated changes will impact network capacity and application performance.

Establish a Network Source of Truth

There is a solution to this issue. When you’re planning your data center network automation strategies, think about how you’re going to establish a source of truth about your network. This involves establishing one or more repositories that contain authoritative information about the intent and the state of your network. More importantly, these sources of truth should be integrated into hour network automation tools so that administrators have all the data they need available to them within a couple of consoles.

Some network automation vendors offer their own native sources of truth in their platforms. Others integrate with popular sources of truth, like commercial IPAM and CMDB products.

“Our automation tool holds all the data we need,” a network engineer with a $60 billion manufacturer told EMA. “It takes a snapshot of the network every time we change it. We just go in there for the data we need.”

You may never automate all the data gathering you need for network automation. Chances are there is some data that will always exist in the margins, in places like spreadsheets. But you should minimize this marginal data management as much as possible.

Related articles: