Designing Async Rust Systems Through Codex’s Network Proxy

Introduction

As coding agents become increasingly common, and open-source models continue closing the gap with proprietary ones, more teams are building their own execution harnesses. As agents become more capable, the context management, tooling, and failure recovery you give them become just as important as the underlying models.

Despite this, we underappreciate the complexity of building generalized, production-grade harnesses. So I wanted to write this blog to use Codex as a case study for exploring async Rust design patterns through one commonly overlooked part of an execution harness: the network proxy.

First, I'll explain why Codex needs a network proxy at all. Then we'll build a simplified version of that proxy to explore several async Rust design patterns.

Coding Agents and Network Proxies

Every command Codex runs executes inside a sandbox, but the sandbox alone isn't enough. The filesystem is isolated, the process is constrained, but the network is still a wide-open escape hatch. If a process can make arbitrary outbound requests, it can still leak data or reach systems it shouldn't.

Instead of letting processes talk directly to the Internet, Codex routes all outbound traffic through a proxy. Every outbound connection is forced through the proxy, which decides whether to allow it, deny it, or ask the user for approval. Because of this, it becomes a single enforcement point for network policy. How does the proxy actually decide whether a request should be allowed?

Policy Evaluation

Every outbound connection goes through the same policy evaluation step before it's allowed to proceed. The proxy checks a denylist, then blocks access to local and private networks, and only then evaluates the allowlist.

Each check results in a request being accepted or denied. Explicitly denied requests are rejected immediately. Requests targeting local resources are blocked separately to prevent accidental access to internal services. Everything else that falls outside the allowlist can be escalated to the user for approval.

When a user approves a destination, that decision is cached so we don't repeatedly prompt the user for the same destination. If the policy itself changes, Codex recompiles it and swaps it into the running proxy atomically, without interrupting in-flight requests.

The real implementation is quite complex, so to understand it better, we will build a simplified example of this proxy and touch on the Rust nuances used to implement it.

Building a Simple Proxy

Before diving into implementing a simple proxy, we should understand the problem it solves. A generalized proxy must be able to make asynchronous decisions based on a policy. There can be multiple policy implementations, and we should memoize as many decisions as possible.

The first design problem is that the proxy shouldn't know how policy decisions are made. It only needs to ask for one. It does this with the PolicyDecider trait. PolicyDecider separates policy evaluation from proxy execution.

1/// Thread-safe trait to decide if a host can be accessed2trait PolicyDecider: Send + Sync + 'static {3    fn decide<'a>(4        &'a self,5        host: String,6    ) -> Pin<Box<dyn Future<Output = Decision> + Send + 'a>>;7}8
9/// Blanket implementation which takes any compatible closure10/// and returns a decision Future11impl<F, Fut> PolicyDecider for F12where13    F: Fn(String) -> Fut + Send + Sync + 'static,14    Fut: Future<Output = Decision> + Send + 'static,15{16    fn decide<'a>(17        &'a self,18        host: String,19    ) -> Pin<Box<dyn Future<Output = Decision> + Send + 'a>> {20        Box::pin((self)(host))21    }22}23
24/// Struct to hold proxy state25#[derive(Clone)]26struct Proxy {27    state: Arc<RwLock<ProxyState>>,28    decider: Option<Arc<dyn PolicyDecider>>,29    pending: Arc<Mutex<HashMap<String, Arc<PendingRequest>>>>,30}

The proxy can work with any implementation that has the trait PolicyDecider, which lets it call decide() to get a heap-allocated future that can be referenced by multiple threads. This keeps the proxy responsible only for coordinating requests. The logic that determines whether a request should be allowed lives entirely inside the PolicyDecider.

Policy decisions may require asynchronous work, such as waiting for user approval. That means decide() can't return a Decision immediately, so it must return a future.

1. Trait Objects vs. Generic Design

The proxy could have been written generically:

1trait PolicyDecider: Send + Sync + 'static {2    fn decide(3        &self,4        host: String,5    ) -> impl Future<Output = Decision> + Send;6}7
8struct Proxy<D: PolicyDecider> {9    decider: Arc<D>,10    // ...11}

This is more performant code, as it uses static dispatch. The compiler knows the concrete decider type and does not need to make a vtable lookup. However, the cost is that the decider type becomes part of the proxy type. Proxies cannot be typed simply as Proxy, but rather as Proxy<BasicDecider>, which blurs the boundary between the type and the implementation of the proxy.

The generic implementation will only be able to construct a Proxy specialized to using D as a decider. Meanwhile, the trait-object implementation erases the type, allowing any type that implements PolicyDecider to return the same proxy.

If you want to store different implementations behind the same type and avoid concrete type leakage, trait objects are a good fit. The cost is extra vtable dispatch and usually a heap allocation for boxed async futures.

The beauty of the trait-object approach is that it hides the implementation details of the decider.

1trait PolicyDecider: Send + Sync + 'static {2    fn decide<'a>(3        &'a self,4        host: String,5    ) -> Pin<Box<dyn Future<Output = Decision> + Send + 'a>>;6}7
8#[derive(Clone)]9struct Proxy {10    decider: Option<Arc<dyn PolicyDecider>>,11    // ...12}

The proxy is always typed as a Proxy regardless of the decider. This is important because it separates responsibility cleanly. The proxy can manage concurrent requests, wait for decisions, and publish decisions, while the type implementing PolicyDecider handles the decision business logic.

2. Async Trait Objects Need Type Erasure

Using trait objects introduces two places where Rust can no longer know a concrete type at compile time. First, the decider in Proxy is not a concrete type but a reference to some concrete type implementing PolicyDecider. Second, the Box<dyn Future<...>> erases the type returned by decide().

In async Rust, all futures are distinct types generated by the compiler. Because the proxy only knows it's calling a PolicyDecider, it can't name the concrete future type returned by a particular implementation. This prompts the need to define a stable return type.

With trait objects, we perform one dynamic dispatch for the PolicyDecider and another to poll the boxed future. Since these policy decisions can depend on slower user input and networking, the performance tradeoff is acceptable. The result is that the proxy exposes a single, stable interface regardless of how policy decisions are implemented.

3. Separating Fast and Slow Paths

Each field in the proxy exists to optimize a different stage of handling a request.

1#[derive(Clone)]2struct Proxy {3    state: Arc<RwLock<ProxyState>>,4    decider: Option<Arc<dyn PolicyDecider>>,5    pending: Arc<Mutex<HashMap<String, Arc<PendingRequest>>>>,6}

The proxy has three responsibilities: answer cached requests immediately, evaluate uncached requests asynchronously, and ensure concurrent requests don't duplicate work. Those responsibilities map directly to the three fields in Proxy. Every request enters the proxy through check_host(), which first attempts the fast path before falling back to asynchronous policy evaluation. The logic is shown below.

1async fn check_host(&self, host: &str) -> Decision {2    let host = host.to_ascii_lowercase();3
4    {5        let state = self.state.read().await;6
7        if state.denylist.contains(&host) {8            return Decision::Deny;9        }10
11        if state.allowlist.contains(&host) {12            return Decision::Allow;13        }14    }15
16    // pending-decision logic comes next17}

Notice that the read lock is dropped before any asynchronous policy evaluation occurs. Cached requests remain fast, while slower policy decisions don't block readers.

If a decision for a host is already in flight, the proxy does not start another decision. Instead, the caller attaches to the existing PendingRequest stored in the pending map. When the first caller finishes computing the decision, it stores the result and wakes every waiter. This ensures duplicate concurrent requests require only one policy decision.

1let (pending, is_owner) = {2    let mut pending = self.pending.lock().await;3
4    if let Some(existing) = pending.get(&host) {5        (Arc::clone(existing), false)6    } else {7        let new = Arc::new(PendingRequest::new());8        pending.insert(host.clone(), Arc::clone(&new));9        (new, true)10    }11};

The Mutex on pending ensures that there can only be one access at a time, and we can check if there is already an owner for a PendingRequest. If so, we clone the existing request and become a waiter. If there is no owner, we clone the host and become the owner.

1if !is_owner {2    return pending.wait().await;3}4
5let decision = match self.decider.as_ref() {6    Some(decider) => decider.decide(host.clone()).await,7    None => Decision::Deny,8};9
10pending.set(decision).await;11self.pending.lock().await.remove(&host);

4. Deduplicating Async Work

Once multiple callers can wait on the same decision, we need a way for one task to publish the result while everyone else waits. The PendingRequest type has wait() and set() methods. It also has a decision and a Notify field. Notify allows one task to wake any tasks waiting for a decision. Each notify_one() call adds a permit. When Notify::notified() is awaited, it returns immediately if there are permits available.

This allows tasks to go to sleep and only be awakened once a decision is added. The implementation for PendingRequest is shown below.

1struct PendingRequest {2    notify: Notify,3    decision: Mutex<Option<Decision>>,4}5
6impl PendingRequest {7    fn new() -> Self {8        Self {9            notify: Notify::new(),10            decision: Mutex::new(None),11        }12    }13
14    async fn wait(&self) -> Decision {15        loop {16            let notified = self.notify.notified();17
18            if let Some(decision) = *self.decision.lock().await {19                return decision;20            }21
22            notified.await;23        }24    }25
26    async fn set(&self, decision: Decision) {27        {28            let mut current = self.decision.lock().await;29            *current = Some(decision);30        }31
32        self.notify.notify_waiters();33    }34}

The Mutex<Option<Decision>> is the shared result. None means the owner of the PendingRequest is still computing the answer, while Some(decision) means the decision has been published and needs to be propagated to every waiter.

Decisions use Notify to park threads until they need to be awakened. The waiter creates a notification future, then checks for a result, and then goes to sleep. This means the only way to wake it up is to notify the waiting threads. When they wake up, they return a decision on the next iteration of the loop. The loop handles the race between checking for a completed decision and going to sleep.

5. Locks and Contention

One of the most important design goals is ensuring that locks are never held while performing slow work. In check_host(), locks are held for the minimum amount of time to reduce contention. The RwLock is used for the fast path to read the allowlist and denylist since they are rarely updated. It holds locks only long enough to read or update shared state. The Mutex is used to protect the PendingRequests map since it is frequently updated.

However, both locks are held for the minimum amount of time possible. This locking structure is acceptable because the majority of latency is associated with user input, not lock contention.

Conclusion

The proxy we built here is much simpler than the real implementation, but the same design ideas show up throughout the codebase. Codex uses a scaled-up version of the same principles to build a performant network proxy.