Load Balancing Policy Interface Polymorphism Design

In high-performance load balancers, supporting multiple load balancing strategies is a common requirement. Different business scenarios may require different strategies: Round Robin for stateless services, Least Connections for long-lived connections, Hash for session persistence.

How to support flexible strategy switching while maintaining high performance? This article provides an in-depth analysis of the policy interface polymorphism design in industrial-grade code, with a clean-room reconstruction in Rust.

The Problem: Evolution of Strategies

In early load balancers, strategies were often hardcoded:

// Pseudo-code - hardcoded strategy
if (strategy == "round_robin") {
    // Round robin logic
} else if (strategy == "least_connections") {
    // Least connections logic
}

The drawbacks of this approach are obvious:

• Adding new strategies requires modifying core code
• Code branches become numerous and hard to maintain
• Code cannot be reused between strategies

Industrial Solution: Interface Abstraction + Factory Pattern

The original code uses the classic strategy pattern:

// Policy interface
class IPolicy {
public:
    virtual ~IPolicy() = default;
    virtual IBackend* Next(IAlgorithm* algorithm, bool fastAttempt) = 0;
    virtual void RegisterSuccess() = 0;
    virtual void RegisterFail() = 0;
};

// Policy factory
class IPolicyFactory {
public:
    virtual THolder<IPolicy> ConstructPolicy(const TStepParams& params) noexcept = 0;
    virtual void FillFeatures(TPolicyFeatures& features) const noexcept = 0;
};

The core ideas of this design:

• Interface abstraction: Define policy behavior through IPolicy interface
• Runtime polymorphism: Factory pattern creates concrete strategy objects at runtime
• Feature detection: FillFeatures allows the framework to query strategy features and optimize behavior

Trade-off Analysis

Advantages

1. Open/Closed Principle: Adding new strategies doesn't require modifying framework code
2. Decoupling: Strategy implementation is isolated from framework code
3. Flexible configuration: Strategies can be switched at runtime

Costs

1. Virtual function overhead: Every strategy call has vtable lookup
2. Heap allocation: Factory pattern typically involves heap allocation
3. Indirect jumps: Code readability decreases slightly

Additional Design Wisdom: Feature Query

The FillFeatures method in the original code is a clever design:

struct TPolicyFeatures {
    bool WantsHash = false;
};

The framework can query strategy features to decide whether to pass hash values, avoiding unnecessary computation.

Rust Clean-Room Demonstration

Below is a clean-room demonstration in Rust, reconstructing the above design philosophy:

#![allow(dead_code)]
#![allow(unused_imports)]

use std::sync::Mutex;

// Policy trait - corresponds to C++ IPolicy
trait Policy: Send + Sync {
    fn select_backend(&self, backends: &[Backend], params: &StepParams) -> Option<usize>;
    fn on_success(&self);
    fn on_failure(&self);
    fn requires_hash(&self) -> bool {
        false
    }
}

// Simulated backend server
#[derive(Clone)]
struct Backend {
    id: usize,
    weight: usize,
}

// Round Robin policy
struct RoundRobinPolicy {
    counter: Mutex<usize>,
}

impl RoundRobinPolicy {
    fn new() -> Self {
        Self { counter: Mutex::new(0) }
    }
}

impl Policy for RoundRobinPolicy {
    fn select_backend(&self, backends: &[Backend], _: &StepParams) -> Option<usize> {
        if backends.is_empty() { return None; }
        let mut counter = self.counter.lock().unwrap();
        let idx = *counter % backends.len();
        *counter += 1;
        Some(idx)
    }
    fn on_success(&self) {}
    fn on_failure(&self) {}
}

// Least Connections policy
struct LeastConnectionsPolicy {
    connections: Mutex<Vec<usize>>,
}

impl LeastConnectionsPolicy {
    fn new(backend_count: usize) -> Self {
        Self { connections: Mutex::new(vec![0; backend_count]) }
    }
}

impl Policy for LeastConnectionsPolicy {
    fn select_backend(&self, backends: &[Backend], _: &StepParams) -> Option<usize> {
        if backends.is_empty() { return None; }
        let conns = self.connections.lock().unwrap();
        conns.iter().enumerate().min_by_key(|(_, &c)| c).map(|(i, _)| i)
    }
    fn on_success(&self) {}
    fn on_failure(&self) {}
    fn requires_hash(&self) -> bool { true }
}

// Policy factory
struct PolicyFactory;

impl PolicyFactory {
    fn create_policy(policy_type: &str, backend_count: usize) -> Box<dyn Policy> {
        match policy_type {
            "round_robin" => Box::new(RoundRobinPolicy::new()),
            "least_connections" => Box::new(LeastConnectionsPolicy::new(backend_count)),
            _ => Box::new(RoundRobinPolicy::new()),
        }
    }
}

fn main() {
    let backends = vec![
        Backend { id: 1, weight: 100 },
        Backend { id: 2, weight: 100 },
    ];
    
    let params = StepParams { attempts: 1, hash: None };
    
    // Use round robin policy
    let policy = PolicyFactory::create_policy("round_robin", backends.len());
    if let Some(idx) = policy.select_backend(&backends, &params) {
        println!("Selected backend: {}", backends[idx].id);
    }
}

Summary

This article provides an in-depth analysis of load balancing policy interface polymorphism design, exploring the following core trade-offs:

1. Interface abstraction vs Hardcoding: Trade flexibility for some performance cost
2. Runtime polymorphism vs Compile-time polymorphism: Rust's trait object provides an alternative
3. Feature query: Allows framework to optimize behavior without strategy implementation awareness

This design pattern is very common in large-scale systems. Understanding the trade-offs behind it is crucial for designing scalable systems.