What you will build today: A 3-node Gateway cluster where each node maintains persistent TCP connections to every other node, routes events through a binary wire protocol with zero String allocations in the hot path, and enforces explicit backpressure using bounded queues — the same architectural pattern Discord uses to prevent a single slow node from taking down the entire cluster.
What you will understand after today:
Why frameworks hide the slow consumer problem and how it kills production systems
How binary framing eliminates garbage pressure in the inter-node hot path
Why Virtual Threads are the exact right tool for persistent peer connections
How a
SubscriptionTableturns an O(N²) broadcast into a targeted forward
Part 1 — The Theory
The “Spring Boot” Trap
A junior engineer hits the requirement: “When User A on Gateway-1 sends a message to a channel where User B is on Gateway-2, Gateway-1 needs to forward that event.” Their instinct: pull in Spring Cloud Bus, drop a RabbitMQ dependency, annotate with
@StreamListener, and call it solved.
Here’s what they just hid from themselves:
// What Spring Cloud Bus does under the hood — every forwarded event
String json = objectMapper.writeValueAsString(event); // Allocates String + char[] + byte[]
rabbitTemplate.convertAndSend("gateway.events", json); // Copies those bytes againAt 50,000 events/second across a 20-node cluster, that’s 50,000 ObjectMapper.writeValueAsString() invocations per second. Each creates: a String, a backing char[], and a byte[] for the UTF-8 encoding. Three heap allocations per event in the hot path. Your Eden space fills in milliseconds. Minor GC fires every 40–80ms. Users see it as jitter: a 2ms message delivery that occasionally spikes to 60ms. At Discord scale, that 60ms spike is a visible stutter in 100 million chat windows.
The deeper failure: Spring Cloud Bus gives you zero visibility into back-pressure. When RabbitMQ falls behind, your producers block silently on a Channel.basicPublish() inside a framework-managed thread pool. By the time you notice, your Gateway threads are exhausted and new WebSocket connections are being rejected.
The Failure Mode: The Slow Consumer Death Spiral
The architectural failure is the slow consumer problem. Consider a 3-node Gateway cluster:
Gateway-1 → [outbound queue] → TCP → Gateway-2 (overloaded, processing slowly)
→ TCP → Gateway-3 (healthy)If the queue feeding Gateway-2 is unbounded — the classic LinkedList or LinkedBlockingQueue(Integer.MAX_VALUE) — a traffic spike causes queue depth to grow at the rate of (production rate − consumption rate). At 10,000 events/second with Gateway-2 processing at 8,000 events/second, the queue grows by 2,000 entries/second. After 90 seconds: 180,000 event objects on heap. Each holding a byte[] payload. OOM in 10 minutes.
The JVM makes this worse. Those queued events are tenured — they survive enough minor GCs to be promoted to Old Gen. When Old Gen fills, you get a Stop-The-World Full GC. For 30 seconds, every Gateway-1 thread is frozen. Every client connected to Gateway-1 sees a 30-second message gap.
Discord’s real lesson here: their initial presence broadcast was O(N²). Every user status change went to every gateway node. The fix was lazy subscription — a SubscriptionTable mapping channelId → Set<nodeId>. You only forward to nodes that have actual subscribers. Presence updates for a channel with 3,000 members on 2 gateway nodes forward to exactly 2 nodes, not 50.
The Flux Architecture
