rust-gateway

rust-gateway is an async API gateway built in Rust using Axum and Tokio. It handles routing, load balancing, circuit breaking, rate limiting, and JWT authentication in front of backend microservices. I built it as a counterpart to go-gateway to explore how the same infrastructure problem plays out in a language with ownership semantics, an async runtime, and no garbage collector. The goal was to understand where Rust's guarantees help and where they add friction.

Architecture

The gateway is organized into five packages: handlers (proxy, auth, registration), load_balancer, rate_limiter, registry, and metrics. All shared mutable state including the service registry, load balancer state, rate limiter buckets, and metrics counters is wrapped in Arc<Mutex<T>> and passed into each handler via Axum's dependency injection.

The request path runs: global rate limiter check, auth middleware on protected routes, proxy handler, load balancer instance selection, circuit breaker evaluation, then the backend HTTP call. Failures feed back into the circuit breaker, which updates per-backend state for the next selection round.

Client
  → RateLimiter::check("global")
  → AuthMiddleware::validate_jwt(token)
  → ProxyHandler::forward(path, method, headers, body)
      → LoadBalancer::next_backend()   // round-robin + CB filter
      → reqwest::Client::request(url)
      → LoadBalancer::report_success / report_failure

Circuit Breaker

Each backend instance tracks its own circuit breaker state: Closed (normal), Open (excluded from routing), and HalfOpen (testing recovery). The state machine lives inside the LoadBalancer alongside the round-robin counter, so selection and fault tolerance are handled under a single Mutex lock.

After 3 consecutive failures the instance transitions to Open and is filtered out before the round-robin calculation runs. The remaining healthy instances absorb the load. After 30 seconds the instance enters HalfOpen, allowing one request through. A success closes the circuit. A failure resets the 30-second timer and returns to Open.

pub fn report_failure(&mut self, url: &str) {
    if let Some(state) = self.backends.get_mut(url) {
        state.failure_count += 1;
        state.last_failure = Some(Instant::now());
        if state.failure_count >= 3 {
            state.circuit = CircuitState::Open;
        }
    }
}

pub fn report_success(&mut self, url: &str) {
    if let Some(state) = self.backends.get_mut(url) {
        state.failure_count = 0;
        state.circuit = CircuitState::Closed;
    }
}

Load Balancing

The selection algorithm filters all backends down to those in Closed or HalfOpen state, then picks from that healthy subset using a global counter modulo the healthy count. This differs from the least-connections approach in go-gateway. Round-robin is simpler and avoids the need for atomic request tracking, which matters when you are already holding a Mutex lock for circuit breaker state.

The proxy handler retries up to 3 times on failure, selecting a new backend on each attempt. A single request can interact with up to 3 different instances before the handler gives up and returns 502. This also distributes failure reports across multiple circuit breakers rather than concentrating them on one.

for attempt in 0..3 {
    let backend = load_balancer.lock().unwrap().next_backend()?;
    let url = format!("{}{}", backend, path);

    match client.request(method.clone(), &url)
        .headers(headers.clone())
        .body(body.clone())
        .send().await
    {
        Ok(resp) => {
            load_balancer.lock().unwrap().report_success(&backend);
            return Ok(proxy_response(resp).await);
        }
        Err(_) => {
            load_balancer.lock().unwrap().report_failure(&backend);
        }
    }
}

JWT Authentication

Auth is applied as an Axum middleware layer on protected route groups. The middleware extracts the Authorization header, strips the Bearer prefix, and validates the token against a configured secret using HS256 with standard claims including subject and expiration. Missing or expired tokens return 401 before the request reaches the proxy handler.

The secret is currently hardcoded with a planned move to config. The middleware is stateless with no session storage or token revocation. Expiration is the only way to invalidate a token.

pub async fn auth_middleware(
    headers: HeaderMap,
    request: Request,
    next: Next,
) -> Result<Response, StatusCode> {
    let token = headers
        .get(AUTHORIZATION)
        .and_then(|v| v.to_str().ok())
        .and_then(|v| v.strip_prefix("Bearer "))
        .ok_or(StatusCode::UNAUTHORIZED)?;

    decode::<Claims>(token, &DecodingKey::from_secret(SECRET), &Validation::default())
        .map_err(|_| StatusCode::UNAUTHORIZED)?;

    Ok(next.run(request).await)
}

Go vs Rust: same problem, different trade-offs

Building the same gateway in both Go and Rust was the point of the exercise. The most notable difference is not performance. It is where the complexity lands.

In Go, the concurrency tools (goroutines, sync.RWMutex, atomic) are lightweight and the standard library does most of the work. The health checker is a goroutine and a ticker. The request counter is an atomic int32. Writing it feels straightforward.

In Rust, every shared value needs Arc<Mutex<T>> or Arc<RwLock<T>> wired through the call graph. The borrow checker catches lifetime errors that Go would defer to runtime. The benefit is that data races are impossible and the compiler proves it. The cost is that go-gateway took half the time to write.

The Rust version uses Tokio's async runtime instead of goroutines, which means no blocking calls anywhere in the stack. Go does not have this constraint. In practice, both approaches handle the request loads a gateway like this would see.