Well, how does one workaround this issue? Setting the keep alive probe down to 50 seconds and then resending it every 10 seconds seems to do the trick for me. I also tweaked the TCP settings so that if no ACK response is received for 5 consecutive times, the connection is marked as broken. Here's the quick one-liner fix:
sudo sysctl -w net.ipv4.tcp_keepalive_time=50 \
net.ipv4.tcp_keepalive_intvl=10 \
net.ipv4.tcp_keepalive_probes=5
Of course, to make these settings persistent across reboots, add them to /etc/sysctl.conf
I'm not sure if these settings are "optimal", but they do the trick. You're mileage may vary.