I have 3 Virtual Machine cluster (platform1, platform2 and platform3) and I have enabled ipsec tunnel communication between them using strongswan (5.6.2). The tunnel looks fine and connected, but seems there is a problem routing the traffic through the tunnel in a particular direction.
Below are the observations
- While attemptign ping from platform1 (192.168.10.1) to platform2 (192.168.10.2), packets are lost
- While attempting ping from platform2 to platform1, it works fine.
- After step2, ping from platform1 to platform2 starts working
strongswan version: Linux strongSwan U5.6.2/K4.15.0-143-generic
OS-version : Ubuntu-18.04
Any idea/suggestion to further troubleshoot the communication between the VMs?
Following are the configuration on platform2. platform1 and platform3 have similar configuration.
Host file content
192.168.10.1 platform1
10.79.196.221 platform1-external
192.168.10.2 platform2
10.79.199.178 platform2-external
192.168.10.3 platform3
10.79.199.212 platform3-external
Following are the configurations on platform2. Platform1 and platform3 also have similar configurations.
ipsec.conf
config setup
charondebug="all"
uniqueids=yes
conn platform2-to-platform1
type=tunnel
auto=start
keyexchange=ikev2
left=10.79.199.178
leftsubnet=192.168.10.2/32
leftid=@platform2
leftcert=platform2.cert.pem
leftupdown=/home/user/tunnel/sa-updown-handler.sh
right=10.79.196.221
rightsubnet=192.168.10.1/32
rightid=@platform1
mark=42
ike=aes128-sha256-ecp256!
esp=aes128-sha256!
aggressive=no
keyingtries=%forever
ikelifetime=28800s
lifetime=3600s
dpddelay=30s
dpdtimeout=120s
dpdaction=restart
conn platform2-to-platform3
type=tunnel
auto=start
keyexchange=ikev2
left=10.79.199.178
leftsubnet=192.168.10.2/32
leftid=@platform2
leftcert=platform2.cert.pem
leftupdown=/home/user/tunnel/sa-updown-handler.sh
right=10.79.199.212
rightsubnet=192.168.10.3/32
rightid=@platform3
mark=42
ike=aes128-sha256-ecp256!
esp=aes128-sha256!
aggressive=no
keyingtries=%forever
ikelifetime=28800s
lifetime=3600s
dpddelay=30s
dpdtimeout=120s
dpdaction=restart
ipsec status
user@platform2:~$ sudo ipsec statusall
Status of IKE charon daemon (strongSwan 5.6.2, Linux 4.15.0-143-generic, x86_64):
uptime: 3 days, since Jul 22 10:47:46 2021
malloc: sbrk 2695168, mmap 0, used 1145568, free 1549600
worker threads: 11 of 16 idle, 5/0/0/0 working, job queue: 0/0/0/0, scheduled: 6
loaded plugins: charon aesni aes rc2 sha2 sha1 md4 md5 mgf1 random nonce x509 revocation constraints pubkey pkcs1 pkcs7 pkcs8 pkcs12 pgp dnskey sshkey pem openssl fips-prf gmp agent xcbc hmac gcm attr kernel-netlink resolve socket-default connmark stroke updown eap-mschapv2 xauth-generic counters
Listening IP addresses:
10.79.199.178
192.168.10.2
Connections:
platform2-to-platform1: 10.79.199.178...10.79.196.221 IKEv2, dpddelay=30s
platform2-to-platform1: local: [platform2] uses public key authentication
platform2-to-platform1: cert: "C=US, O=MyOrganisation, CN=platform2"
platform2-to-platform1: remote: [platform1] uses public key authentication
platform2-to-platform1: child: 192.168.10.2/32 === 192.168.10.1/32 TUNNEL, dpdaction=restart
platform2-to-platform3: 10.79.199.178...10.79.199.212 IKEv2, dpddelay=30s
platform2-to-platform3: local: [platform2] uses public key authentication
platform2-to-platform3: cert: "C=US, O=MyOrganisation, CN=platform2"
platform2-to-platform3: remote: [platform3] uses public key authentication
platform2-to-platform3: child: 192.168.10.2/32 === 192.168.10.3/32 TUNNEL, dpdaction=restart
Security Associations (2 up, 0 connecting):
platform2-to-platform3[26]: ESTABLISHED 64 minutes ago, 10.79.199.178[platform2]...10.79.199.212[platform3]
platform2-to-platform3[26]: IKEv2 SPIs: 25a984a1cd9eb524_i 5be00d6f63ee5974_r*, public key reauthentication in 6 hours
platform2-to-platform3[26]: IKE proposal: AES_CBC_128/HMAC_SHA2_256_128/PRF_HMAC_SHA2_256/ECP_256
platform2-to-platform3{453}: INSTALLED, TUNNEL, reqid 26, ESP SPIs: cede4a64_i cc6d694d_o
platform2-to-platform3{453}: AES_CBC_128/HMAC_SHA2_256_128, 0 bytes_i, 0 bytes_o, rekeying in 23 minutes
platform2-to-platform3{453}: 192.168.10.2/32 === 192.168.10.3/32
platform2-to-platform3{454}: INSTALLED, TUNNEL, reqid 26, ESP SPIs: cc70aa87_i c7e6e2f0_o
platform2-to-platform3{454}: AES_CBC_128/HMAC_SHA2_256_128, 0 bytes_i, 0 bytes_o, rekeying in 25 minutes
platform2-to-platform3{454}: 192.168.10.2/32 === 192.168.10.3/32
platform2-to-platform3{455}: INSTALLED, TUNNEL, reqid 26, ESP SPIs: caa9b761_i cead3e1a_o
platform2-to-platform3{455}: AES_CBC_128/HMAC_SHA2_256_128, 0 bytes_i, 0 bytes_o, rekeying in 25 minutes
platform2-to-platform3{455}: 192.168.10.2/32 === 192.168.10.3/32
platform2-to-platform1[25]: ESTABLISHED 6 hours ago, 10.79.199.178[platform2]...10.79.196.221[platform1]
platform2-to-platform1[25]: IKEv2 SPIs: 25f267ea5b3d7b14_i 63a3d407e92c28a3_r*, public key reauthentication in 56 minutes
platform2-to-platform1[25]: IKE proposal: AES_CBC_128/HMAC_SHA2_256_128/PRF_HMAC_SHA2_256/ECP_256
platform2-to-platform1{456}: INSTALLED, TUNNEL, reqid 25, ESP SPIs: ce279b78_i c94beeaa_o
platform2-to-platform1{456}: AES_CBC_128/HMAC_SHA2_256_128, 0 bytes_i, 0 bytes_o, rekeying in 29 minutes
platform2-to-platform1{456}: 192.168.10.2/32 === 192.168.10.1/32
config
user@platform2:~$ ifconfig
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 10.79.199.178 netmask 255.255.252.0 broadcast 10.79.199.255
ether 00:50:56:05:39:88 txqueuelen 1000 (Ethernet)
RX packets 74096446 bytes 5149048059 (5.1 GB)
RX errors 11265 dropped 2 overruns 0 frame 0
TX packets 167716 bytes 47626811 (47.6 MB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
device interrupt 16 base 0x2000
ipsec1: flags=193<UP,RUNNING,NOARP> mtu 1480
inet 192.168.10.2 netmask 255.255.255.255
tunnel txqueuelen 1000 (IPIP Tunnel)
RX packets 123 bytes 10344 (10.3 KB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 256 bytes 21153 (21.1 KB)
TX errors 3 dropped 0 overruns 0 carrier 3 collisions 0
lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
loop txqueuelen 1000 (Local Loopback)
RX packets 241910023 bytes 55152362048 (55.1 GB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 241910023 bytes 55152362048 (55.1 GB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
Route table
user@platform2:~$ route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 10.79.199.253 0.0.0.0 UG 0 0 0 eth0
10.79.196.0 0.0.0.0 255.255.252.0 U 0 0 0 eth0
192.168.10.0 0.0.0.0 255.255.255.0 U 0 0 0 ipsec1
