If I make Server 1 the default gateway, I lose 30-40% of my download speed on Server 2. It's possible, but a bad idea.
Indeed this is a bad idea and
not what Javier suggests. Furthermore it will not work because if, on site 2 (where server 2 runs) default gateway is, in a static way, server 1, you would never be able to establish VPN tunnel
What Javier said is that on each location, the easiest way is to have server running VPN service (either server or client) defined as "local" default gateway. Meaning server 2 is the default gateway on site "2" and server 1 is the default gateway on site "1".
This is an advice for
easy implementation but by no mean mandatory.
So to summarise:
- what you want to achieve does work (I did it here and you have also reproduced it on your side)
- it hopefully works without manually tweaking any configuration file
What might be difficult is to be sure that your current production environment is back to "standard" configuration given that you have already applied tweaks. Assuming this is however OK, then looking at routes should tell us whether problem lies here or not.