Manage Learn to apply best practices and optimize your operations.

Enterprise network testing: What engineers overlook

If only network testing were as simple as running traffic through individual components or trying out specific features. In this video, SearchNetworking experts Tom Kunath and Andy Sholomon, authors of  Enterprise Network Testing: Testing Throughout the Network Lifecycle to Maximize Availability and Performance,discuss the often overlooked strategies of network testing. Chief among their recommendations? Network engineers must conduct testing of overall systems. That means running tests on overall architectures as opposed to points in the network. In this video, Kunath and Sholomon lay out specific recommendations. 

Read the full transcript from this video below:  

Enterprise network testing: What engineers overlook

Interviewer: People think of testing as testing one piece of equipment with some code. What's testing really about?

Tom Kunath: I mean there's certainly that aspect of testing through one of our vendors, Cisco, and we've developed unit test division that when new code and new equipment come out, they thoroughly will beat up the hardware and code and run it through rehearsed automated tests. But most enterprises won't do that. They don't need to do that. You have to assume that's going to be done to a certain extent. What's more important to an enterprise is to simulate your design and do a systems test, a thorough systems test, which is completely different than a unit test. A systems test is where you'd assemble all of your equipment, your network structure, your switches, your servers, your network management stations, and some type of test equipment that can generate traffic. Assemble it all together, design what it will look like, address it, and then when you're running protocols, run a systems test. Then you make sure all the features work when they're deployed together. Too many times people will just test feature by feature, one at a time, but they don't put it together, only to find that, you maybe IPsec wasn't compatible with your QOS model, or something of that nature.

So the first thing we do is a baseline test where we enable everything like you, as the architect, see what the digital design look like. And then we'll do a feature test. We'll maybe do some what-ifs like what if I tweak these timers down in BFD [Bidirectional Forwarding Detection]? How far will I go if I have 5,000 CE [customer edge] routers connect to it, will I have instabilities? You know, [we test] those type of things to validate design, and then we'll do negative testing where you start failing over nodes or links in your core or your edge and see what happens to the traffic. “Will I get phone calls that drop? Will I get sessions that time out? What's the conversion time?" We spend a large portion of our test time on negative-testing failure scenarios.

Andy Sholomon: Right. The other thing that a lot of people don't pass is migrations. So a lot of times what they do is test the final product. But they don't test in that in-between stage when you go from the old design to a new design. And often times, that's when we have a big problem. So that's going to be a whole migration from old system to new system that is a very, very difficult and very important thing to do. And especially once you do that, also test your fall-back scenarios. So a lot of the times you'll go into production, you'll make a change, and if something fails, you never tested how to come back from that chain. And all these things are things that you have to plan for when you do testing, and are things we discuss in our book.

Tom Kunath: Right, exactly, network ready for use testing. Another different type of test where instead of going into a test lab, you go onto your pre-production network, and you go through and make sure that everything has been deployed as it was intended in design. And you'll repeat some of the tests you would do in the lab, but now you're doing it on your live network before actual traffic is on it. So you can certify that day one when you flip the switch and a lot of traffic is going over it, it's going to work like you thought, like it did in the lab.

Andy Sholomon: Right. All the circuits are connected correctly, and they're working as expected, and they're not dropping traffic. Because in the lab, usually you don't have faulty circuits, it's just a piece cable. But in the real world, you're buying a circuit from a service provider, and you want to make sure that the circuit is actually working as expected before you put production traffic on it.

Tom Kunath: So you're checking out, you're verifying that the service provider provisioned you the right quality of service and that there's not some errors on the infrastructure that you bought. That again you wouldn't see in a lab, but you would see on a live production model. We dedicate a whole case study to network ready-for-use testing. This is all based on things that we've done, so they're pretty relevant and pretty current.

Interviewer: Thank you so much.


View All Videos