Backend Load Balancing

Backend Load Balancing

When multiple backend servers are configured within a backend block, Radiator uses server selection algorithms to determine which server receives each request and how to handle failures. This applies to all backend types: RADIUS, LDAP, SQL (PostgreSQL, MySQL), and others.

Backend load balancing is configured using the server-selection statement within backend blocks.

Server Selection Algorithms

Radiator supports four server selection algorithms configured via the server-selection statement. The default is fallback.

fallback

Tries servers in priority order. The highest-priority (lowest number) available server receives every request. Lower-priority servers are used only when higher-priority servers are unavailable or degraded. This is the default algorithm.

backends {
    radius "BACKEND_CLUSTER" {
        server-selection fallback;  # or omit, fallback is default

        server "PRIMARY" {
            priority 0;  # Highest priority, tried first
            # ...
        }

        server "SECONDARY" {
            priority 1;  # Lower priority, backup
            # ...
        }
    }
}

round-robin

Distributes requests evenly across all available servers in a rotating fashion. Each request goes to the next server in the list. If a server fails, the request is retried on the next available server.

backends {
    radius "BACKEND_CLUSTER" {
        server-selection round-robin;

        server "SERVER1" { ... }
        server "SERVER2" { ... }
        server "SERVER3" { ... }
    }
}

least-connections

Routes each request to the available server with the fewest in-use connections. If that server fails, the next least-loaded server is tried. When servers are tied, servers with a lower priority value are preferred; within the same priority, alphabetical name order is used as tiebreaker.

This algorithm is available for SQL backends (PostgreSQL, MySQL) and LDAP backends. For RADIUS backends, least-connections is not currently supported and falls back to round-robin.

backends {
    postgres "USER_DB" {
        server-selection least-connections;

        server "replica1" {
            host "pg-replica1.example.com";
            connections 15;
            min-connections 2;
            idle-timeout 5m;
        }

        server "replica2" {
            host "pg-replica2.example.com";
            connections 15;
            min-connections 2;
            idle-timeout 5m;
        }

        query "FIND_USER" { ... }
    }
}

no-fallback

Only attempts the first available server. If that server fails, the request fails immediately without trying other servers. Use when backend failures should be immediately visible rather than masked by failover.

backends {
    radius "BACKEND_TEST" {
        server-selection no-fallback;

        server "SINGLE_SERVER" { ... }
    }
}

Server Priority

The optional priority statement (integer 0–255, default 0) controls server order for the fallback and no-fallback algorithms:

backends {
    radius "TIERED_BACKENDS" {
        server-selection fallback;

        server "PRIMARY_DC" {
            priority 0;  # Highest priority, tried first
            # ...
        }

        server "SECONDARY_DC" {
            priority 1;  # Tried if PRIMARY fails
            # ...
        }

        server "DR_SITE" {
            priority 2;  # Last resort
            # ...
        }
    }
}

Lower numbers = higher priority (priority 0 is highest). When multiple servers have the same priority value, they are tried in alphabetical order by their server names.

Server Health and Status

Each backend server tracks its availability status. Radiator automatically:

• Marks servers as unavailable after connection failures
• Skips unavailable servers during selection
• Attempts to reconnect to failed servers periodically
• Returns servers to the available pool when connections succeed

Service Level Objective

Each backend server supports a service-level-objective block for automatic circuit breaking. When a server reaches the failure rate, Radiator marks it as degraded and routes traffic to healthier servers with exponential backoff.

When server-selection is configured, Radiator automatically applies the default SLO (failure-rate 3/5, initial-backoff-period 3s, max-backoff-period 30s, recovery-probe-count 2) to every server that does not have an explicit service-level-objective block. Add an explicit block to a server to override the defaults. For single-server backends (no server-selection), an SLO block must be added explicitly if health monitoring is desired.

Status health checks (RADIUS)

The status statement controls automatic periodic health checks for RADIUS backends:

• status true — enables periodic health checks (default)
• status false — disables health checks; servers are still used for requests

Important: When status false, failed backend servers will not automatically recover. It is strongly recommended to use the default status true for production environments.

SQL Connection Pool Parameters

SQL backends (PostgreSQL, MySQL) maintain a connection pool per server.

connections — pool size per server

Default: 10

Upper limit on open connections to one database server. Size by throughput:

connections = target_tps × (round_trip_ms + query_ms) / 1000

With 10 ms RTT and 1 ms query time at a 1000 TPS target: 11 connections.

min-connections — warm connection floor

Default: 2

Number of connections Radiator keeps open at all times. Radiator uses lazy connection pooling, so startup succeeds even if the database is unreachable. When idle-timeout closes a connection that takes the pool below this floor, a replacement is opened immediately.

Set to 0 for a fully lazy pool where no connections are pre-opened.

Error: min-connections must not exceed connections.

idle-timeout — idle connection lifetime

Default: 5m

How long a connection above min-connections can sit idle before being closed.

timeout — request time budget

Set on the backend action inside an AAA handler, not on the backend or server blocks.

@execute {
    backend {
        name "USERS";
        query "FIND_USER";
        timeout 5s;
    }
}

Bounds the total time for both connection acquisition and query execution.

For the full list of server options see PostgreSQL server and MySQL server.

How SQL Requests Are Handled

In the normal case a warm idle connection is already available and the request proceeds immediately. The diagrams below show what happens in non-trivial cases.

Connection acquisition

Radiator does not queue behind a saturated pool. If a pool appears full, that server is skipped immediately. If every server's pool appears full, the request returns with a pool exhausted error (tracked by the PoolExhausted counter).

Multi-server routing

SQL Pool Lifecycle

Radiator maintains a separate connection pool per server block.

With min-connections = 2 (default)

With min-connections = 0 (fully lazy)

Pool Monitoring

Pool metrics are available through the Management API.

SQL Counters

MySQL counters use the same structure with MySQL in place of Postgres.

LDAP Counters

Pool Size Log Fields

Pool size metrics are included as fields in structured log messages:

In-use connections = pool_total - pool_idle. If this stays near the maximum under normal load, increase the pool size or add more servers.

Request Retry Behavior

When a backend server fails, Radiator's retry behavior depends on:

1. Server-level retries (RADIUS): The retries statement controls how many times to retry the same server
2. Server-level timeout: The timeout statement sets how long to wait for a response
3. Server selection algorithm: Determines if/how to try alternative servers

With round-robin or fallback, if all retries on one server fail, the next server in rotation/priority is tried.

Common Patterns

SQL Primary/Replica (Active-Passive)

Use fallback for primary database with read replicas:

backends {
    postgres "USERS" {
        server-selection fallback;

        server "primary" {
            host "pg-primary.example.com";
            database "radiator";
            username "radiator";
            password env.DB_PASSWORD;
            connections 10;
            min-connections 2;
            idle-timeout 5m;
            priority 0;
        }

        server "replica" {
            host "pg-replica.example.com";
            database "radiator";
            username "radiator";
            password env.DB_PASSWORD;
            connections 10;
            min-connections 2;
            idle-timeout 5m;
            priority 1;
        }

        query "FIND_USER" {
            statement "SELECT username, password FROM users WHERE username = $1";
            bindings { aaa.identity; }
            mapping { user.password = password; }
        }
    }
}

SQL Read Scaling (Active-Active)

Use round-robin to distribute read queries across replicas:

backends {
    postgres "USER_DB_READ" {
        server-selection round-robin;

        server "REPLICA1" { host "pg-replica1.example.com"; }
        server "REPLICA2" { host "pg-replica2.example.com"; }
        server "REPLICA3" { host "pg-replica3.example.com"; }

        query "FIND_USER" { ... }
    }
}

LDAP Round-Robin

backends {
    ldap "LDAP_CLUSTER" {
        server-selection round-robin;

        server "LDAP1" {
            url "ldap://ldap1.example.com:389/";
            timeout 3s;
            authentication {
                dn "cn=radiator,dc=example,dc=com";
                password "ldap_password";
            }
        }

        server "LDAP2" {
            url "ldap://ldap2.example.com:389/";
            timeout 3s;
            authentication {
                dn "cn=radiator,dc=example,dc=com";
                password "ldap_password";
            }
        }

        search "AUTHENTICATE" {
            base "ou=users,dc=example,dc=com";
            scope sub;
            filter "(&(uid=%{aaa.identity})(objectClass=inetOrgPerson))";
            mapping {
                user.username = uid;
                vars.dn = entry::dn;
            }
        }
    }
}

RADIUS Active-Passive Failover

backends {
    radius "ACTIVE_PASSIVE" {
        server-selection fallback;

        server "PRIMARY" {
            priority 0;
            secret "mysecret";
            timeout 3s;
            retries 2;
            connect { protocol udp; host "192.168.1.10"; port 1812; }
        }

        server "BACKUP" {
            priority 1;
            secret "mysecret";
            timeout 3s;
            retries 2;
            connect { protocol udp; host "192.168.1.11"; port 1812; }
        }
    }
}

Geo-Distributed Backends

Use fallback with priority to prefer local datacenter:

backends {
    radius "GEO_DISTRIBUTED" {
        server-selection fallback;

        server "LOCAL_DC" {
            priority 0;
            timeout 2s;
            # ...
        }

        server "REMOTE_DC" {
            priority 1;
            timeout 5s;  # Higher timeout for WAN
            # ...
        }
    }
}

Related Documentation

• Service Level Objective — Automatic circuit breaking and degradation
• High Availability and Load Balancing — Overall HA architecture patterns
• PostgreSQL server-selection — PostgreSQL algorithm reference
• MySQL server-selection — MySQL algorithm reference
• LDAP server-selection — LDAP algorithm reference
• RADIUS server-selection — RADIUS algorithm reference
• Prometheus Scraping — Metrics collection and monitoring