We use haproxy for our lb platform, we have many frontends with very dynamic backend (servers will be added/removed 10000 > times per day).
We now add a server to config file and reload the process with master-worker concept which works great but adds latency in our scale of service.
We decided to use server-templating and discovering backends with DNS discovery to resolve this, while in the road to it, haproxy2.4 was released with the feature which allows us to add/remove servers with runtime api.
We bench marked discovery vs runtime api, runtime api performs better in terms of speed, but I feel DNS discovery is more streamlined and less error prone.
What does the community think is better for add/remove servers without affecting the process, discovery or runtime api? also why do you think your choice is better?