# Robots.txt file on HAProxy

**URL:** <https://discourse.haproxy.org/t/robots-txt-file-on-haproxy/1054>\
**Category:** Help!\
**Created:** [March 6, 2017, 7:53pm UTC](https://discourse.haproxy.org/t/robots-txt-file-on-haproxy/1054 "2017-03-06T19:53:01Z")\
**Posts on this page:** 1\
**Showing post:** 2

<div class="post-metadata">

**Author:** ![lukastribus](https://avatars.discourse-cdn.com/v4/letter/l/7ea924/32.png) [@lukastribus](https://discourse.haproxy.org/u/lukastribus)\
**Post date:** [March 8, 2017, 7:34pm UTC](https://discourse.haproxy.org/t/robots-txt-file-on-haproxy/1054/2 "2017-03-08T19:34:36Z")

</div>

> [@How do I serve a single static file from HAProxy?](http://discourse.haproxy.org/t/how-do-i-serve-a-single-static-file-from-haproxy/32):
>
> We have a custom backend we use to serve a single static file: acl is\_robots\_txt path /robots.txt use\_backend robotstxt if is\_robots\_txt is\_cdn backend robotstxt mode http errorfile 503 /etc/haproxy/errors/200robots.http The file contains: HTTP/1.0 200 Found Cache-Control: no-cache Connection: close Content-Type: text/plain # Discourse CDN, all crawling is allowed (cdn contains images and text files) User-Agent: \* Allow: / This means that when people hit: https://cdn.discourse.org/ro…

---

_[View the full topic](https://discourse.haproxy.org/t/robots-txt-file-on-haproxy/1054)._