As a porter of software from ipv4 to ipv6 world, the real problem is parsing of addresses.
Way before 2000 the IPv6 API was pretty good: use getaddrinfo, and that’s it.
This has been described in rfc2133 from 1997.
But what happened then is that everybody started to copy code that was from before 1997, and made their own parsers.
If you hand over the parsing to getaddrinfo you are all set.
So most of the code today can not handle IPv6 link local because of not using getaddrinfo, and making their own parsers (i.e. based on software that was officially made obsolete by the POSIX standard 18 years ago I think)
Now back to haproxy: I imagine that haproxy has a more specialised approach of keeping addresses. And this is probably not compatible with the objects returned by getaddrinfo.
In my case I use socat between haproxy to bootstrap configuration of equipment. So I use socat to tcp proxy to the (Fixed EUI64) ipv6-ll of the equipment until I can configure an IPv4 that’s good for that network.
And when somebody moves the equipment, I lose access until I reconfigure it, either by telnet or by using socat again so I can reconfigure the IPv4.
ULA’s and GUA’s are in the hands of the local network maintainer. Advertising ULA or GUA is opening another can of worms.
(To be fair: socat is a wonder tool, doing everything right.)