How IPsec provides secure communications

IPsec provides secure site-to-site VPNs, remote access and host-to-host communication. In this tip, we peek inside the tunnel and examine how IPsec works.

IPsec is widely used to provide secure site-to-site communication, remote access and host-to-host communication. Unlike SSL, IPsec does not require applications to be modified to utilize it. They can communicate with remote systems in exactly the same way as they would with systems on the local network.

IPsec includes several protocols and modes. The protocols are:

    • Authentication Header (AH)
      AH provides authentication and integrity verification. Authentication is a guarantee to each end of the communication that it is communicating with its intended partner. AH also provides integrity verification, which guarantees that the data has not been altered in transit. AH does not provide encryption.

  • Internet Key Exchange (IKE)
    IKE establishes the keys used by AH and ESP to authenticate the communication end points, verify integrity and by ESP to encrypt and decrypt data.

The two IPsec modes are tunnel mode and transport mode. Tunnel mode supports site-to-site and remote access. Transport mode supports host-to-host communication.

IKE is used with both AH and ESP, and with both tunnel and transport modes. IKE establishes three security associations between the communication endpoints. A security association consists of information specifying the choice of AH and ESP, and between tunnel and transport mode. If encryption will be used, a security association specifies which method will be used and how often encryption keys will be changed.

To initiate communication, IKE first establishes a security association between the IKE implementations on the two endpoints. The initial security association is used to establish two additional security associations -- a separate security association for data sent in each direction between the endpoints.

IKE can use Public Key Encryption or shared secret keys to authenticate the endpoints and to obtain initial encryption keys. IKE can use either SHA-1 or MD5 hash algorithms for authentication. DES, Triple DES or AES can be chosen for encryption; 3DES is often used, but AES is increasing in usage since it provides stronger security.

AH and ESP can be used individually or in combination, but AH is less common. ESP provides encryption in addition to the functions of AH. In most cases, encryption is desired, but AH alone might be chosen if encryption imposes an unacceptable computational load on the communicating systems.

While the array of protocols and modes seems confusing, a single set of protocols and modes is used for each of the supported configurations.

The most common uses of IPsec are site-to-site communication and remote access. In both cases, ESP is used in tunnel mode. In site-to-site communication, IPsec is implemented in the firewall or router connecting each site to the Internet.

When an end node initiates communication with a node at a distant site, the IPsec implementation on the initiating site's firewall uses IKE to create security associations with IPsec on the remote firewall. Data then begins to flow between the two end nodes. All IPsec processing is done by IPsec on the firewalls.

When the IPsec software on the firewall receives a packet originating from a node on its local network, it first encrypts the entire packet using the method specified in the security association governing outgoing packets. It then adds an ESP header to the beginning of the packet. Next, it uses the authentication method specified in the security association to create a hash value that is placed at the end of the packet. Finally, the firewall creates a new IP header with its own address as source and the address of the remote firewall as the destination address.

When the packet is received by the remote firewall, the receiving firewall removes the added IP header, uses the hash to authenticate the packet and then decrypts the packet before sending it on to its final destination.

More on IPsec
FastFacts on IPsec

Basic IPsec VPN topologies and configurations

Remote access operates in a similar manner. But in this case, IPsec software is installed in the remote operating system. The added IP header specifies the remote node's own address as source and the address of the remote firewall as destination.

End node-to-end node communication uses Transport Mode. In this case, the IPsec implementations are located in the operating systems of the two communicating nodes. The originating node encrypts the packet's upper layer header (usually TCP or UDP) and the packet data. Then the ESP header is added between the original IP header and encrypted upper layer header. The authentication hash algorithm is then applied to the ESP header, upper layer header and packet data. Finally, the hash value is placed at the end of the packet. Communication is end node to end node, so there is no need to create a new IP header specifying an intervening firewall.

IPsec provides great flexibility since applications can use it without modification, but this flexibility has a downside. IPsec enables connected systems to gain the same level of access to remote nodes as if they were located on the local network. Network administrators must make a careful analysis of the protocols, authentication and encryption options before choosing to provide IPsec to their users.

About the author:
David B. Jacobs has more than 20 years of networking industry experience. He has managed leading-edge software development projects and consulted to Fortune 500 companies, as well as software startups.

Dig Deeper on Network security

Unified Communications
Mobile Computing
Data Center