Security in virtualization: IDS/IPS implementation strategy

Considering virtualization? Take into account that your IDS or IPS may not work the same way in a virtualized environment as it does in a physical one. Expert Dave Shackleford explains how to address this potential problem.

Intrusion detection and prevention, at both the host and network layer, are staples of information security infrastructure...

today. With the advent of virtualization technology, however, many security professionals have realized traditional intrusion detection tools may not integrate into or operate within virtualized networks or systems as they did in traditional enterprise infrastructures.

For example, network intrusion detection may be more difficult, since the default virtual switches from major platform vendors don't allow for the creation of SPAN or mirror ports, preventing traffic from being copied to IDS sensors. Similarly, IPS systems that are placed in-line within traditional physical network areas may not be able to integrate easily into a virtual environment, particularly for traffic within the virtual networks. A host-based IDS may still function properly on virtual machines, but will now consume resources drawn from a shared pool, making installation of a security agent less desirable.

Fortunately, there are ways to adjust an IDS/IPS implementation strategy that allow for the monitoring of virtual system traffic. That's what we'll cover in this tip.

For starters, VMware Inc.'s virtual switches allow switches or port groups to be placed into "promiscuous mode," where a virtual IDS sensor could see the traffic on the same virtual segment. Alternatively, traffic could be sent out to a physical interface that allows a physical IDS sensor to monitor it. There are numerous open source and third-party virtual switches available now that can operate as traditional switches do.

For Citrix Systems Inc., kernel-based virtual machines (KVMs), and Oracle Corp.'s VirtualBox platforms, the Open vSwitch project provides a full featured virtual switch that allows the creation of SPAN ports for traffic mirroring and monitoring. Cisco Systems Inc.'s commercial Nexus 1000v switch offers the same capabilities, and uses the well-known Cisco IOS command-line interface. Both of these switches also support flow data capture and analysis, which can be used for behavioral monitoring between systems and networks, as well.

In addition to redesigning their systems and using more full-featured virtual switches, security professionals should investigate some of the open source and commercial intrusion detection and prevention options that are available in a virtual appliance format. Many well-known vendors, such as Sourcefire Inc., HP TippingPoint, and IBM ISS have ported their existing IDS and IPS platforms to a virtual appliance format. All of these virtual appliances can be easily integrated into virtual networks, providing monitoring of traffic between virtual machines as well as traffic between the virtual and physical networks.

Specialized virtualization products are available on the market today from vendors such as Reflex Systems LLC, Catbird Networks Inc. and HyTrust Inc. that provide policy-based monitoring and analysis within virtual environments. Although they not true signature-based intrusion detection, these products can augment a more traditional IDS/IPS to allow granular traffic monitoring and access controls, as well as behavior profiling for greater security in virtualization.

There are numerous free offerings to consider, too. Both the Snort and Shadow intrusion detection systems are available for free as VMware Virtual Appliances from the VMware Virtual Appliance Marketplace, and can be connected within VMware virtual environments to monitor and detect intrusion attempts. It's worth noting this unique capability represents an advantage for VMware over competitors.

Separately, several host-based IDS and IPS products are available that have been tested and certified to work within many virtual environments. Check Point Software Technologies Ltd., McAfee Inc. and Symantec Corp. are examples of vendors who now support host-based IDS/IPS on virtual guest systems. Another example is the freely available OSSEC HIDS (now owned by Trend Micro Inc.), which has been proven to work within virtual machines, although without any guarantee of performance or stability specifically for virtual systems. For the most part, commercial HIDS and HIPS agents are tested and modified to use fewer resources on virtual systems to avoid overloading the hypervisor platform. However, host-based devices still consume many resources and require intensive management. Additional scheduling and control capabilities may also be available to ensure virtual machines are not overly taxed during scanning or monitoring operations.

The key question for many organizations should really be: "How much monitoring do we need?" For many, existing hardware-based appliances can monitor traffic to and from virtual networks, and most organizations today do little, if any, monitoring between systems in specific network segments. For those that want or need a higher level of intrusion detection and prevention, however, the good news is that there are numerous options available at both the network and host levels. However, as virtualization becomes more prevalent, virtual IDS and IPS technology will undoubtedly become more common.

About the author:
Dave Shackleford is director of risk and compliance and acting director of security assessments at Sword and Shield Enterprise Security Inc., and is a certified SANS instructor. He was formerly CSO at Configuresoft Inc. and CTO at the Center for Internet Security, and has worked as a security architect, analyst, and manager for several Fortune 500 companies. In addition to these roles, he has consulted with hundreds of organizations for regulatory compliance, as well as security and network architecture and engineering.

This was last published in March 2011

Dig Deeper on Network intrusion detection and prevention (IDS-IPS)