The “f” in “HAProxy” stands for “fun”

Today I figured I’d write this up since I’ve had to find this HAProxy config stuff a few times now and it’s always been a trial and error sort of thing - the last article I read on enabling HAProxy stats literally said it “100% works” in the title, published in 2022, annnnd - did not work. So here’s my own version that probably won’t work in a month or two.

In this article I’ll share some config on how to enable some extra functions in HAProxy to provide stats for the load balancer, an administrative socket for local operations against the live HAProxy instances, and how to use that socket to clear out Stick Tables, disable servers in backends, etc.

Installing HAProxy

Not going to emphasize this part since it’s pretty easy - it’s a package called haproxy. Install it if you haven’t already - alternatively you can run the container image instead and mount things if you like an extra layer of headaches.

# Install HAProxy, EL
dnf install haproxy -y

# Install HAPRoxy, Deb
apt install haproxy -y

# Set some basic config in /etc/haproxy/haproxy.cfg

# Enable HAProxy
systemctl enable --now haproxy

Configuring the Statistics Endpoint

In case you want some information on backends, frontends, servers on the backends, requests to the frontends, responses to the clients, status of the servers, etc - then you can enable the statistics endpoint for the server.

There are a few conflicting ways on how to do it that you’ll see online - ultimately, you just need to set a listener for the port you want the statistics to be served on (default is 1936), and set some stats specific configuration there:

# global config up here
# default config also maybe up here

listen stats
    # bind sets the port and interface/IP
    bind    *:1936
    # HTTP/L7 mode is required
    mode    http
    # Assume the global log settings
    log     global
    # There probably aren't that many things that are supposed to be listening for maxconn
    maxconn 10

    # Set the required timeouts
    timeout connect         4s
    timeout client          20s
    timeout server          20s

    # enable the stats module
    stats   enable

    # Obfuscate the version
    stats   hide-version

    # Refresh interval for the HTTP site
    stats   refresh         30s

    # Enable reporting of a host name on the statistics page
    stats   show-node

    # Enable reporting additional information on the statistics page
    stats   show-legends

    # Set some basic user:pass auth
    stats   auth            notadmin:securePassword
    stats   auth            othernotadmin:3xt4aSecurePassword

    # Configure where the HTTP URL should serve from
    stats   uri             /haproxy?stats

# backend/frontend config down here maybe

You can read more about all the other stats configuration parameters here, though that should be a really good starting point. You’ve got the basic needed settings, enabling of the stats module, and some good fundamentals for keeping it secure. You should be able to do a curl -u 'notadmin:securePassword' http://localhost:1936/haproxy?stats and see some output.

Enabling an Administrative Socket

You could serve the statistics endpoint via a system socket and interact with it via socat - but that’s not useful since your stats may be scraped by something like your observability tool, which is why it’s available via HTTP.

However, what is very useful is enabling an administrative system socket to interact with the HAProxy instance to disable/enable frontends, healthchecks, servers, or to clear stick tables.

To enable the administrative system socket, just drop it into your global section - alternatively, you could also place it on a frontend to make more atomic administrative sockets:

# Maybe your global section looks like this...
    log local2 debug
    chroot      /var/lib/haproxy
    pidfile     /var/run/
    maxconn     40000
    user        haproxy
    group       haproxy

    # Enable a global administrative socket
    stats       socket  /run/haproxy/admin.sock mode 660 level admin
    stats       timeout 10s

The key part of the global section are the last two lines - again, you could put this in a frontend instead for more atomic control of load balanced services and their administrative sockets.

Wherever you do put the socket, make sure to create the parent directory and properly chown/chmod it:

# Create the runtime HAProxy directory
sudo mkdir -p /run/haproxy/

# Give it the proper ownership and permissions
sudo chown root:root /run/haproxy/
sudo chmod 775 /run/haproxy/

Using the Administrative Socket

Now that there’s an administrative socket enabled, you can use it with something like socat to interact with it.

You can query the socket endpoint like so:

# Get help information
echo "help" | sudo socat stdio /run/haproxy/admin.sock

# Get load balancer information and statistics
echo "show info;show stat" | sudo socat stdio /run/haproxy/admin.sock

You can read more about the Runtime API here:

Clearing Stick Tables

In case you’re using a fail-over pattern that directs traffic to a primary server, failing over to a secondary and keeping the traffic there until manually reset, your backend may look something like this:

backend https
    stick-table type ip size 2 nopurge
    stick on dst
    server       cloudserver cloud_server_endpoint:80 check on-error mark-down observe layer7 error-limit 1
    server       localserver local_server_endpoint:80 check backup

In this example, your stick-table name would be the same as the backend so to clear it and reset the traffic back to the cloudserver, you’d run:

# Clear the https backend stick table
echo "clear table https" | sudo socat stdio /run/haproxy/admin.sock

Disable a Backend Server

If you’re needing to test traffic patterns resolving to different servers in a backend list, you can simply disable the servers:

# Disable the cloudserver from the https backend example above
echo "disable server https/cloudserver" | sudo socat stdio /run/haproxy/admin.sock

# or, enable the cloudserver from the https backend example above
echo "enable server https/cloudserver" | sudo socat stdio /run/haproxy/admin.sock

Bonus - Example Repo

In case you’re in the market for a reference repo that handles all this HAProxy stuff, automates it with Ansible, and with some keepalived goodness for HA HAProxy, then look no further: