Thinking and Doing

I forget who told me that conventional wisdom is often neither, but I would do well to remember it. I have been heads down testing NAC products for the last few weeks and in between having vendor system engineers (the people how travel to site for installation and troubleshooting) in the lab helping with set-up, I have been picking their brains on what they see in deployments. When you land on a NAC vendor, be sure to add to your list of questions, if they don???t ask you, what switch models and software versions have you tested your product with. I may have fallen prey to making statements like "make sure you upgrade your switch equipment before deploying NAC" in the past because, well, NAC often leverages new features in switches and new features are in current revisions. So upgrading to the latest version makes sense, right? That's the conventional wisdom. Truth is, the latest revision is not always the best working version for whatever your doing. New bugs, regression bugs, missing features, feature changes can render required features useless.

For example, during a recent vendor install, we got the network reconfigured for 802.1X and we were using RADIUS attributes to assign ports to specific VLAN's but our enforcement wasn't working. We could see the RADIUS exchange was successful but no VLAN assignment. We checked the version to find that the switch software wasn't tested by this vendor. So we back-revved the switch to a different version and after several hours of troubleshooting and changing the switch software to different versions trying to find a match, I was left with a switch that couldn't enforce policy. The SE graciously offered to stay an extra day to work on the problem, but I sent him off and started working on the issue fresh the next morning. After reading the switch docs for the umpteenth time, I found a missing configuration parameter that was in the original configuration but was mysteriously removed during the version dance.

I am not going to name names because I think revision problems affect all switch vendors, not just a few (that's experience talking, not wisdom), but here is what I am hearing from SE's and seeing in the lab. Switch configuration features that affect everything from 802.1X, VLAN assignment, RADIUS attribute usage, and DHCP management, can vary widely from version to version even at the dot release level. Problems range from configuration options not available to the option being available but doing nothing. Worse, as you begin to move from software version to software version on a switch, some configuration options may disappear or reappear. That makes for some interesting troubleshooting and it also makes for the potential for tough choices. What if a feature in one version is affected by moving to a new version? There could be a cascade of incompatibilities ranging from minor annoyances in unrelated features to failure.

If a NAC product requires changes to any existing services, you absolutely have to test any switch features, especially recently added features whenever you rev software. Don't assume that a later version will contain all the same features as previous versions or that the details like options, output, and effect are static.