Day 48: Consistent Hashing

Theory of the Ring and Key Placement

Jul 04, 2026

∙ Paid

Introduction

Imagine you’re building the next Discord. You have 100 million users chatting simultaneously, and they’re all connected through WebSocket connections to your gateway servers. Everything works great with 100 servers... until you need to add one more server to handle a traffic spike.
Suddenly, 50 million users disconnect and reconnect at the same time. Your monitoring dashboard lights up red. The database is drowning in authentication requests. Your entire system grinds to a halt.
What happened? You fell into the modulo hashing trap.
In this lesson, we’re going to learn how companies like Discord, LinkedIn, and WhatsApp solve this problem using a technique called Consistent Hashing. By the end, you’ll build a real working system that can handle millions of connections without breaking when you add or remove servers.

Part 1: The Problem - Why Simple Solutions Break at Scale

The “Spring Boot” Approach

When most developers first learn about load balancing, they reach for simple tools. Maybe you use Spring Boot with a @LoadBalanced annotation, or you configure Nginx with ip_hash. For your first project with 10,000 users, this works perfectly.

Here’s what the code looks like:

// The approach that seems fine... until it isn't
int targetServer = Math.abs(sessionId.hashCode()) % numberOfServers;

This is called modulo hashing. You take the user’s session ID, hash it to get a number, then use the modulo operator (%) to pick which server handles that user.

Why It Breaks

Let’s trace through what happens when you have 100 servers and decide to scale up to 101 servers:

User "alice" with hash = 1337
Old: 1337 % 100 = 37  → Server 37
New: 1337 % 101 = 30  → Server 30 (MOVED!)

User "bob" with hash = 9876
Old: 9876 % 100 = 76  → Server 76  
New: 9876 % 101 = 82  → Server 82 (MOVED!)

Notice that both users got routed to different servers. In fact, the math is brutal: only users where hash % 100 == hash % 101 stay on the same server. For a 100-server cluster, that’s about 1% of users. The other 99% move to different servers.