Can I proxy resources from the filesystem?

I spent a few days making beautiful error pages; informative and with options besides a get out of here message. I took care to use relative-coordinated non-rasterized SVGs (data:image/svg+xml;base64,) for imagery and kept cleaning the code as much as my limited knowledge of it allowed me. In the end, however, all styles, images plus the required HTTP dialog thing at the beginning summed up about 20KB for each file.

I’m not sure if the limit is 16KB per page or overall and shared with the live response-building thing, and the more I kept reading about it about adjusting it, it seems that HAProxy would readjust it itself to lower it. I don’t think I’m ready just yet to tweak tuning settings anyway.

Before this I was using errorloc303 and hosting the pages in a server that can’t be down because it’d take HAProxy with it if it were down. It works fine and even lets me bypass Cloudflare’s 5xx errors by changing the code, but at the expense of changing the address in the bar. The next option is to reference the images from the pages. I can continue to use the old host for the images only or enable httpd in RHEL 8.5 – HAProxy’s host – to serve the images in the loopback or some other place not claimed by HAProxy already. It seems reasonable enough, and since HAProxy can manipulate just about anything on the path, it’d be trivial to set it up. This got me thinking though, since HAProxy can manipulate just about anything on the path can it proxy it to the filesystem?

A web server with another name if you will, I haven’t worked out the details yet but it can use several things from the file system already; from what I’ve learned about HAProxy, it’s that it maps every resource on the filesystem it’s going to need if there’s need to work outside from its confines of system’s memory. I can’t map new files into its running configuration but it can modify existing mapped files. I wouldn’t be doing either, anyway, I’m just exploit this to make it host things, or perhaps use a map file to replace specific image URLs to data:image/svg+xml;base64, schemes from a map files if a redirect to – or rather content switching to – the filesystem isn’t possible.

Any advise? I’m open to anything. :slight_smile:

You can serve files with http-request return; but you are STILL LIMITED to the buffer size and those files are all read into memory when starting (so when you change a file you need to reload haproxy).

Haproxy WILL NEVER be able to server files from the disk as a webserver does.

Bumping tune.bufsize from the default 16384 to 20480 bytes or something like that is not that big of a deal, increasing bufsize is only about memory consumption (decreasing would be problematic on the other hand).

1 Like

GitHub - jiangwenyuan/nuster: A high performance HTTP proxy cache server and RESTful NoSQL cache server based on HAProxy is a fork of haproxy that supports this. I’d personally stick with Lukas’ approach though,

Thank you to both for answering–and for the links. tune.maxrewrite is the one I read about something readjusting itself but now I’m reading it slightly different, I guess it was fatigue or something. I’ll take a snapshot, try adjusting only once (or else it all turns destructive) and if it doesn’t work I think I’ll settle for Apache.

Memory there’s plenty available still, and it’s a VM so it can grow online, I don’t want to be the one running a memory-smart proxy server like an Exchange/SharePoint Server though. Hopefully I’m not, I’ll know when it comes the time to post my config.

Thank you so much again @lukastribus and @skunkwerks. :nerd_face:

→ → quick update → →

I doubled tune.bufsize to 32768 just to be safe, added some extra code that was missing so HAProxy would intercept error pages, hapchk, hapreset (shortcut scripts) and I got my beautiful 503.

I was much less thrilled about the 503 on a backend that layer7-healthchecking 200s but that I’ll figure it out later.

I thought it was one of those tiny changes that becomes exponential but memory is still at a fifth with a little more used as cache…

You’re the best! Thanks!

Regarding memory consumption, it grows until global maxconn is reached, so if you want to benchmark memory consumption you’d have to test with simultaneous connections so that memory is allocated fully.