I have a load balancer with a dedicated cpu that distributes traffic to two linodes with shared cpus. I'm testing a template website with 9 lazy loaded images, no database queries, that's been optimized to score 100 on pagespeed, A on gtmetrix, and loads in under 1 second.
While testing the website with the shared cpu, it performs as expected, around 1k rps with or without ssl, using nginx.
However, while testing the load balancer on the dedicated cpu, it performs strangely, around 200 rps with ssl, and around 500 rps without ssl, using nginx. It gets even stranger because the performance is cut in half while using haproxy.
I have tried using tcp and http mode in both nginx and haproxy but the performance is the same. It also doesn't appear to improve throughput by adding additional web server...
Could someone please help me understand this behavior? And hopefully, provide guidance to resolve it?
ab, wrk, traceroute, mtr, free, vmstat, htop:
https://gist.github.com/david-littlefield/9c63db85ec77aa265f6c59408af004ed
nginx http configuration:
https://gist.github.com/david-littlefield/bc62610d50b162b474c3f2fdcf3f8b06
nginx tcp configuration:
https://gist.github.com/david-littlefield/b4aa97d8c872361d26cd0ca8de85a58c
haproxy http configuration:
https://gist.github.com/david-littlefield/cbed0867d0be4db71ac4c3d0861549b2
haproxy tcp configuration:
https://gist.github.com/david-littlefield/bafde91305c46fd117f94434de75c881
