Request for comments: Load balancing between two connections to same host

So, I have this stupid idea. I’m going to home-host some stuff. Game servers, web, forums, etc.
Now, it gets really stupid. I’m considering getting a second connection from my ISP. I’d have a second ISP, but the only other thing available is 10mb DSL. Not worth the weight of the dollars paid for it.

Here’s my thoughts:
I’ll set up HAProxy on a handful of VPSs. For regional connections. Then I’ll forward those connections to my servers. Via load balancing.

HAP1 ---,
HAP2 ---|
HAP3 ---|
_______|
|
|--- IP1---,
|    *or*  |
`--- IP2   |
____|      |
|          |
`-Router1 -'
     |
 -servers-

This should cause connections originating from HAProxy 1 going to IP1 to return to HAProxy 1 through IP1, and connections on HAP3/IP2 to return via IP2, and HAP2/IP1 to return on IP1… etc

The bandwidth between the proxies and my connections will be 400/35 (400 from proxy to router, 35 from router to proxy), for each connection. If it works the way I think, and using a router that can handle it, I should be able to have what would appear to be 70mb upload capability back to my proxies (and thus players) This is honestly overkill for my needs, however I have other things going outbound - streams, uploads, etc.

I’m thinking of having one line for house use, and the other for game use, with the first being failover / overflow for game servers.

Would any of this work like I’m thinking? And yes, I’ve got the internal stuff handled, including my gateway router that can support multiple wan connections.

Bonus Points: If you know if Spectrum would throw a temper tantrum with me doing this on business class connections, let me know, and why?

For those interested, this is the basis of my HAProxy configs… and yes, I wrote this, and yes it works wonderfully: https://www.spigotmc.org/threads/obscure-corner-for-obscure-conf-file.345414/

This may work almost out of the box.
I recommend reading this blog article: https://www.haproxy.com/fr/blog/asymmetric-routing-multiple-default-gateways-on-linux-with-haproxy/