hermes-skills/skills/devops/infrastructure-automation/references/l2tp-ipsec-debugging.md

L2TP/IPsec VPN Connection Debugging

Real-world troubleshooting log from setting up a L2TP/IPsec VPN tunnel between a Hetzner VPS (Ubuntu 24.04, strongSwan 6.0.4) and a MikroTik CCR gateway.

Problem: Tunnel won't establish

Symptom 1: "no shared key found" / INVAL_KE

parsed ID_PROT response 0 [ SA V V V V ]
no shared key found for '5.161.114.8' - '76.195.7.60'
received NO_PROPOSAL_CHOSEN error notify

Causes:

1. PSK in ipsec.secrets doesn't match what's on the MikroTik
2. strongSwan 6 stroke plugin doesn't match the generic : PSK "..." format

Fix: Use explicit IPs in ipsec.secrets and strip YAML inline comments from the parsed value:

printf '5.161.114.8 %s : PSK "%s"\n' "$SERVER_IP" "$PSK" > "$IPSEC_SECRETS"

Also ensure the comment was stripped from the YAML value: sed 's/ *#.*//' after extraction.

Symptom 2: "NO_PROPOSAL_CHOSEN" on ESP (Phase 2) after IKE SA established

IKE_SA wisp-vpn[5] established between 5.161.114.8[...]...76.195.7.60[...]
parsed INFORMATIONAL_V1 request [ HASH N(NO_PROP) ]
received NO_PROPOSAL_CHOSEN error notify

Cause: ESP proposal esp=aes128-sha1 doesn't include MODP group. strongSwan 6 requires exact match with the MikroTik's proposal.

Fix: Add -modp1024 suffix to both ike= and esp=:

ike=aes128-sha1-modp1024
esp=aes128-sha1-modp1024

Problem: VPN connects but kills VPS internet

Symptom: "ppp0 recursion detected" on MikroTik, VPS unreachable, Telegram/SSH dead

User sees on MikroTik: ppp0 recursion detected in logs. VPS behavior: internet completely dead — ping 8.8.8.8 fails, SSH session drops, Telegram stops responding.

Reproduction: Happens when the VPN script up handler doesn't pin the L2TP server public IP route before ipsec up. The IPsec transport mode encrypts traffic destined for 76.195.7.60 (the peer), the kernel looks up the route, finds ppp0, and sends the encrypted packet back through the tunnel → MikroTik sees it and logs recursion detected.

Recovery: The user must disable the PPP secret on their MikroTik to force the L2TP session down (/ppp secret disable shonuff). The IPsec SA may also need to be cleared on both sides.

Fix (in wisp-vpn.sh up handler, BEFORE ipsec up):

# Pin the L2TP server's route through eth0, not through the tunnel
CURRENT_GW=$(ip route get 8.8.8.8 | awk '{print $3}' | head -1)
if [ -n "$CURRENT_GW" ]; then
    ip route add "$SERVER_IP" via "$CURRENT_GW" dev eth0 metric 100 2>/dev/null || \
        ip route change "$SERVER_IP" via "$CURRENT_GW" dev eth0 metric 100 2>/dev/null
fi

Also remove defaultroute from the PPP options so pppd doesn't install itself as the default route at all. Add only specific subnet routes through ppp0 after connection.

When to keep the VPN always-on

Once the tunnel is stable, switch from connect-on-demand to always-on:

• systemd service — wisp-vpn.service with ExecStart=/bin/bash /script.sh status + ExecStart=/bin/bash /script.sh up (idempotent — if status exits 0, the second ExecStart is skipped)
• Keepalive timer — wisp-vpn-keepalive.timer running every 2 minutes, reconnecting if the tunnel drops
• Auto-start — both the service and timer are systemctl enable'd so they survive reboots

See the umbrella skill's "Always-on VPN with systemd" section for full implementation.

Symptom: L2TP connects but SSH to router fails (timeout)

Setup: ppp0 has IP 192.168.88.105, peer is 192.168.88.1. Ping fails, SSH times out.

Causes:

1. Firewall on MikroTik blocks traffic from ppp interface to router management IP
2. /ip service set ssh address= restricts SSH to specific source IPs/subnets
3. Routing table on VPS doesn't have the 192.168.88.0/24 route through ppp0

Fix (on MikroTik):

/ip service set ssh address=0.0.0.0/0

How to verify IPsec status

# Check IPsec SA state
ipsec status

# Check all SAs
ipsec statusall

# Check routing
ip route show | grep ppp

# Check VPN interface
ip addr show ppp0

# Check charon logs
journalctl -u strongswan-starter -n 30 --no-pager

# Check xl2tpd logs
journalctl -u xl2tpd -n 30 --no-pager

# Check if internet is reachable (must do after VPN connect)
ping -c 1 -W 3 8.8.8.8

Current working config (MikroTik L2TP/IPsec)

config setup
    charondebug="all"
    uniqueids=yes
conn wisp-vpn
    auto=add
    keyexchange=ikev1
    authby=secret
    type=transport
    left=%defaultroute
    leftprotoport=17/1701
    right=76.195.7.60
    rightprotoport=17/1701
    ike=aes128-sha1-modp1024
    esp=aes128-sha1-modp1024
    dpddelay=30
    dpdtimeout=120
    dpdaction=clear

Terminal output reference

When the IPsec connection succeeds:

connection 'wisp-vpn' established successfully

When the L2TP connection succeeds:

pppd[NNN]: local  IP address 192.168.88.105
pppd[NNN]: remote IP address 192.168.88.1

When authentication fails:

pppd[NNN]: MS-CHAP authentication failed: bad username or password
pppd[NNN]: CHAP authentication failed

When lock option is used (pppd 2.5.x):

pppd[NNN]: unrecognized option 'lock'