Back to DMVPN Labs

Cryptography and IPsec Fundamentals for DMVPN

Build the security model behind a protected DMVPN. Translate plaintext into cryptographic controls, compare hashes and encryption, choose a safe Diffie-Hellman group, apply an IKEv2/IPsec profile, and prove the negotiated result.

CryptographyHashing and HMACAESDiffie-HellmanIKEv2 / IPsec

Cryptography and IPsec at-a-Glance

Cryptography is a toolbox, not a single algorithm. DMVPN uses different tools for different jobs: IKEv2 authenticates peers and derives keys, while IPsec ESP protects GRE packets in the data plane.

VPN categories showing site-to-site IPsec with route-based and policy-based options, and remote-access VPN with client-based and clientless options
VPN categories: Site-to-site IPsec connects networks, while remote-access VPN connects individual users through client-based or clientless methods.
Site-to-site IPsec VPN tunnels connecting two remote sites to an enterprise network across the Internet
Site-to-site VPN: IPsec protects traffic between remote networks and the enterprise edge across the public Internet.
Clientless and client-based remote-access VPN connections using SSL, TLS, or DTLS to reach enterprise resources
Remote-access VPN: Individual users connect through a browser-based clientless session or installed VPN client before reaching protected enterprise resources.
ConfidentialityAES encryption
IntegrityHMAC or AEAD
Peer identityPSK or certificate
Shared key materialDiffie-Hellman
Modern baseline: Use IKEv2 and algorithms supported by both peers. For this portable lab, start with AES-256, SHA-256, and DH group 14; use a supported elliptic-curve group or authenticated encryption when the platform and policy permit it.

In This Lesson

  1. Map the core terminology
  2. Test hashing and HMAC
  3. Compare encryption models
  4. Choose a Diffie-Hellman group
  5. Map IPsec features and protocols
  6. Configure IKEv2 and IPsec
  7. Verify, break, and recover
  8. Review common questions

1. Map the Core Cryptography Terminology

Start with the data and the security goal. A good design selects a control only after deciding whether the traffic needs confidentiality, integrity, peer authentication, or all three.

TermMeaningDMVPN example
PlaintextReadable original data before encryption.The GRE packet entering IPsec protection.
CiphertextEncrypted output that should be unreadable without the correct key.The encrypted ESP payload crossing the underlay.
EncryptionA reversible transformation from plaintext to ciphertext using an algorithm and key.ESP using AES.
DecryptionRecovery of plaintext from ciphertext using the matching key.The receiving peer processing ESP.
HashA fixed-length, one-way digest used to detect change.SHA-256 in a lab comparison.
HMACA keyed message authentication code built from a hash function.Integrity and origin assurance when both peers know the secret.
Do not confuse the controls:

A plain hash can detect a changed message only when the expected digest is trusted. HMAC adds a shared secret. Encryption provides confidentiality. Digital signatures provide origin authentication and integrity but do not normally encrypt the bulk data.

2. Test Hashing and HMAC

Create a small message, record its MD5 and SHA-256 digests, change one word, and calculate the values again. Both outputs change dramatically, but digest length stays fixed. MD5 is included only to demonstrate a legacy algorithm; collision attacks make it unsuitable for security decisions.

PowerShell

Set-Content -NoNewline message.txt 'DMVPN packet one'
Get-FileHash .\message.txt -Algorithm MD5
Get-FileHash .\message.txt -Algorithm SHA256

Set-Content -NoNewline message.txt 'DMVPN packet two'
Get-FileHash .\message.txt -Algorithm MD5
Get-FileHash .\message.txt -Algorithm SHA256

Linux or macOS

printf '%s' 'DMVPN packet one' > message.txt
md5sum message.txt
sha256sum message.txt

printf '%s' 'DMVPN packet two' > message.txt
md5sum message.txt
sha256sum message.txt

Prove why HMAC is different

Run the same message with two keys. A peer needs both the original message and the shared secret to reproduce the HMAC.

OpenSSL HMAC-SHA-256

printf '%s' 'DMVPN payload' | openssl dgst -sha256 -hmac 'LAB-KEY-ONLY'
printf '%s' 'DMVPN payload' | openssl dgst -sha256 -hmac 'WRONG-KEY'
printf '%s' 'Changed payload' | openssl dgst -sha256 -hmac 'LAB-KEY-ONLY'
Record in your lab notes: the MD5 length is 128 bits, SHA-1 is 160 bits, and SHA-256 is 256 bits. A longer digest alone is not the complete security argument, but MD5 and SHA-1 are legacy choices and should not be selected for new VPN designs.

3. Compare Symmetric and Asymmetric Techniques

ModelKeysStrength in a VPNExamples
SymmetricThe same secret key protects and recovers data.Fast enough for bulk packet encryption.AES-128, AES-192, AES-256
AsymmetricA mathematically related public/private key pair.Authentication, signatures, and secure establishment workflows.RSA, ECDSA, ECC
Key agreementEach peer contributes private material and derives the same shared secret.Creates session key material without sending the final secret across the wire.Finite-field DH, ECDH

DES

A 56-bit legacy cipher. Do not use it.

3DES

Applies DES operations repeatedly. It is slow, has a small block size, and is not recommended for new designs.

AES

The normal bulk-encryption family for modern IPsec, with 128-, 192-, and 256-bit keys.

AES-GCM

An authenticated-encryption mode that supplies confidentiality and integrity together when supported.

4. Choose a Diffie-Hellman Group

Diffie-Hellman is a key-agreement method. The peers exchange public values and independently derive shared key material; they do not transmit the final session secret. The group defines the mathematical parameters and security level.

GroupTypeLab guidance
1768-bit MODPAvoid; obsolete strength.
21024-bit MODPAvoid for new designs.
51536-bit MODPAvoid for new designs.
142048-bit MODPPortable baseline and common interoperability choice.
15 / 163072- / 4096-bit MODPStronger finite-field options with more processing cost.
19256-bit ECDHModern efficient choice when both peers support it.
20 / 21384- / 521-bit ECDHHigher-strength elliptic-curve choices when policy and platform support align.
242048-bit MODP with 256-bit subgroupDo not assume a higher group number means stronger or preferred; verify current policy and interoperability.
Selection rule: both IKE peers need at least one common encryption, integrity/PRF, and DH choice. Prefer the strongest common set that meets platform capacity, lifecycle, and organizational policy requirements.

5. Map IPsec Features and Protocols

Confidentiality

ESP encrypts the protected GRE payload with a symmetric cipher such as AES.

Integrity

HMAC or an authenticated-encryption mode detects unauthorized changes.

Authentication

IKEv2 validates peer identity with a pre-shared key or certificates.

Anti-replay

ESP sequence numbers and a receive window reject replayed packets.

ComponentRoleDMVPN relevance
IKEv2Negotiates algorithms, authenticates peers, and derives keys.Control plane that creates IKE and child SAs.
ESPEncrypts and can authenticate the protected traffic.Normal data-plane protection for GRE.
AHProvides integrity/authentication but no encryption.Rare in modern DMVPN; ESP is normally used.
Transport modeProtects the payload of the outer IP packet.GRE already supplies the tunnel, so this avoids another tunnel header.
Multicast clarification:

Routing protocols can send multicast inside the GRE overlay. Across the underlay, DMVPN maps that traffic to unicast NBMA peers, and ESP protects the resulting GRE packets.

6. Configure IKEv2 and an IPsec Profile

Apply this lab-only baseline to both peers after the GRE tunnel works without encryption. Command availability varies by IOS/IOS XE release, so use context-sensitive help and record the exact image in your evidence.

IKEv2 proposal and policy

crypto ikev2 proposal DMVPN-PROPOSAL
 encryption aes-cbc-256
 integrity sha256
 group 14
!
crypto ikev2 policy DMVPN-POLICY
 proposal DMVPN-PROPOSAL

Keyring and profile

crypto ikev2 keyring DMVPN-KEYRING
 peer DMVPN-PEER
  address 0.0.0.0 0.0.0.0
  pre-shared-key local DMVPN-LAB-ONLY
  pre-shared-key remote DMVPN-LAB-ONLY
!
crypto ikev2 profile DMVPN-IKEV2
 match identity remote address 0.0.0.0 0.0.0.0
 authentication local pre-share
 authentication remote pre-share
 keyring local DMVPN-KEYRING

IPsec transform and tunnel protection

crypto ipsec transform-set DMVPN-TS esp-aes 256 esp-sha256-hmac
 mode transport
!
crypto ipsec profile DMVPN-PROFILE
 set transform-set DMVPN-TS
 set ikev2-profile DMVPN-IKEV2
!
interface Tunnel0
 tunnel protection ipsec profile DMVPN-PROFILE
Production boundary: never reuse the example PSK. Store long random secrets securely, restrict wildcard peer matching, and prefer certificate-based authentication where the operational PKI can support it.

7. Verify, Break, and Recover the Negotiation

Generate interesting traffic through the tunnel, then verify IKEv2 and IPsec separately. The tunnel can be reachable while crypto negotiation is failing, and an IKE SA alone does not prove that ESP is carrying data.

Healthy-state evidence

show crypto ikev2 sa detail
show crypto ipsec sa
show crypto ipsec profile
show crypto session detail
show dmvpn
ping <remote-lan-ip> source <local-lan-ip>

What to prove

IKEv2 SA: established / ready
Child IPsec SA: active in both directions
Local and remote proxy/endpoint values: expected
ESP encaps and decaps counters: increasing
Decrypt or integrity failures: zero
Remote LAN ping: successful

Introduce one controlled proposal mismatch

On one peer only, replace group 14 with group 19 while the other peer offers only group 14. Clear the lab SAs during a maintenance-safe exercise, generate traffic, and record the no-common-proposal evidence. Restore group 14, clear the failed negotiation state, and prove that the SAs and counters recover.

Fault and rollback

crypto ikev2 proposal DMVPN-PROPOSAL
 no group 14
 group 19
! Capture the failed negotiation, then restore:
 no group 19
 group 14
!
clear crypto ikev2 sa
clear crypto sa
ping <remote-lan-ip> source <local-lan-ip>
SymptomLikely layerFirst evidence
No IKEv2 SAUnderlay, identity, authentication, or proposal mismatchshow crypto ikev2 sa detail
IKEv2 SA exists, no child SAIPsec transform/profile or traffic selector mismatchshow crypto session detail
Encaps rises, decaps stays zeroReturn path, remote policy, NAT, or filteringPeer counters and underlay ACLs
Decrypt/integrity errors riseKey, transform, packet corruption, or stale SA issueshow crypto ipsec sa
Evidence checklist
Platform identityRouter model, licensed image, and exact version
Healthy stateIKEv2 SA, IPsec SA, and increasing packet counters
Controlled failureTimestamped no-common-proposal evidence
Recovered stateRestored proposal, new SAs, and successful traffic

8. Standards and Platform References

Algorithm guidance changes as attacks and implementations evolve. Confirm the current requirements for the deployed platform and organization before treating a lab baseline as a production standard.

9. Frequently Asked Questions

Does a hash encrypt data?

No. A hash produces a one-way fixed-length digest. Encryption is reversible with the correct key and supplies confidentiality.

Why does DMVPN normally use IPsec transport mode?

GRE already provides the tunnel that carries the original packet. IPsec transport mode protects the GRE packet without adding another complete tunnel-mode IP header.

Is a higher Diffie-Hellman group number always stronger?

No. The number is an identifier, not a strength score. Compare the group type, parameter size, current standards guidance, platform support, and organizational policy.

Why keep MD5 and SHA-1 in the comparison?

They help learners recognize legacy configurations and understand digest lengths, but they should not be selected for new VPN designs.