Exploring Encapsulating Security Payload for IPsec Technologies


When discussing the Authentication Header, we understood that stand alone AH is not appropriate to protect data from snooping or from attackers. The second Security Protocol for IPsec is ESP, which we will look into through this article.

Encapsulating Security Payload

ESP gives both authentication and encryption to the data packets. Unlike AH, which only inserts a header, ESP appends a header and footer to the payload, thus encapsulating the original data. The way it inserts these are different depending upon the mode used. It provides multiple security services to give privacy, source authentication and content integrity to the packet.
ESP is added after standard IP header so that it can route with standard devices, making it backwards-compatible with routers and other network equipment.

Level of Security

  • Encryption-Only is available for confidentiality thus providing defense for passive attackers. However, to deal with active attackers, some integrity mechanism should be applied on top of Encryption.
  • Integrity is jointly referred to as data origin authentication and connectionless integrity. Integrity-Only ESP is offered as a selection. In some cases, it is an attractive alternative to AH because of its faster process time. Confidentiality and Integrity are offered independently and in conjunction with each other.
  • Anti-Replay is available for selection if Integrity Service is selected for the chosen Security Association, SA.
  • TFC service is effective when Tunnel Mode is used. It conceals the source and destination addresses of the peers and the characteristics of individual subscriber traffic flows. It also facilitates generation and discarding of dummy traffic.

Packet Format

The protocol header that follows ESP header is to contain the value 50 in its Protocol (IPv4) or Next Header (IPv6, Extension IPv6) field. The fields that follow are given below:

Security Parameters Index (SPI):

It is a 4-byte field which tells the receiving device about the Security Associations of the packet. This is a compulsory field.

Sequence Number:

It is a 4-byte field that keeps track of each packet sent like a counter. Initially, the counters are set to 0 at the establishment of SA. If anti-replay is allowed, the counters are reset after 232 packets. This is done by exchanging new key and establishing new SA.

Extended Sequence Number (ESN):

For high-speed implementations, ESNs (64-bits) are used. In IKEv2, by default we have ESN’s. Only the lower 32-bits are transmitted in ESP headers, while the high-order 32 bits are saved for the sequence number counter. Thus decreasing overhead and increasing efficiency. Depending upon the integrity algorithm (whether separate or combined), the function of ESN’s changes.

Payload Data:

It is a variable-length field. It contains the actual data that is to be carried in the packet. In case the encryption algorithm used requires cryptographic synchronization data, then this data is carried in the payload field explicitly. It has a substructure depending upon the encryption algorithm used and the mode applied.


Padding is used if the encryption algorithm needs the text to be a multiple of some number or when we need the cipher text to end on the 4-byte boundary. It is essential that the pad length and next header fields be right aligned within a 4-byte word. The sender can append up to 255 bytes of padding if needed.

Pad Length:

It tells the number of bytes padded where 0 indicates that no byte is padded.

Next Header:

A byte long compulsory field, defining the type of data in the payload data field and the protocol used.

Integrity Check Value (ICV):

This is an optional field. It is present if integrity service is selected and used in either separate or combined algorithm using ICV. It is calculated using ESP header, Payload and ESP trailer fields.
SPI and SN’s are authenticated while Payload Data, Padding, Pad Length Field and Next Header are encrypted while transmission.


When it comes to configuring ESP, we have to specifically mention it in our code that we want to use esp as our protocol in IPsec configuration. Here is how we do it in JUNOS with this simple line of code.

set security ipsec proposal IPSEC-PRO protocol esp

And remember one thing that ESP is applied during phase II of IPsec, I hope you recall about phase I and II. If not then this article is for you ‘Basics of IPsec’.

For Cisco users out there, here is how ESP is configured with the help of crypto map keyword with a peer having address;

crypto map cmap 10 ipsec-isakmp 
set peer
set transform-set TS 
match address cmap

Stay tuned for more of ESP in which we will have a brief description on encryption algorithms that are used and some configuration lines.



No comments yet. Be the first to chime in!