When One Server Isn't Enough

You shipped your app. It lives on one server: a computer in a data centre running your code, talking to a database. A few hundred people use it and everything is fast. Life is good.

Then more people show up. The site gets slow. Requests start timing out. Something has to change.

This is the moment most frontend developers feel out of their depth, because "scaling" sounds like a topic with its own textbooks. It mostly isn't. It's a handful of moves you make in order, each one to fix a specific pain the last one couldn't. Let's walk the whole path.

Start with one server

Here's where everyone begins.

A request comes in, your server runs some code, reads or writes the database, sends a response back. One machine does all of it. Don't be embarrassed by this setup. Plenty of real businesses run on it for years. You only move off it when it actually hurts.

The pain, when it comes, looks like this: the CPU sits at 100%, memory fills up, responses crawl. The server is doing more work than it can keep up with.

Move one: get a bigger server

The first fix is the laziest, and that's a compliment. If the machine is maxed out, rent a bigger machine. More CPU cores, more memory. Move your app onto it. Done.

This is vertical scaling, also called scaling up: same number of servers, each one beefier.

So why isn't this the end of the article? Two reasons.

1. There's a ceiling. You can buy a bigger machine, then a bigger one, but eventually you hit the largest box your provider sells. You can't scale up forever.
2. It's one machine. If that single server crashes, restarts, or its data centre loses power, your whole app is down. One machine means one thing that can take everything offline.

That second point is the real motivator. Even if a giant server could handle your traffic, betting your entire business on one computer never failing is a bad bet.

Move two: add a second server

So instead of one big server, you run two smaller ones, each with a full copy of your app. If one dies, the other keeps serving. If both are busy, they share the work.

This is horizontal scaling, or scaling out: more servers, each one ordinary.

But adding a second server immediately creates a new question. The browser only knows one address. When someone visits your site, which of the two servers answers? You can't ask users to pick.

Move three: put a load balancer in front

You add one more box whose only job is to stand at the front door and hand each incoming request to one of your servers. That box is a load balancer.

Now the browser talks to the load balancer's address, and the load balancer spreads requests across your servers. The simplest rule is round robin: first request to A, next to B, next to C, then back to A. Smarter rules exist, like sending the next request to whichever server has the fewest open connections, but round robin already gets you most of the benefit.

The load balancer also quietly does a second job: health checks. It pings each server every few seconds. If server B stops answering, the balancer stops sending it traffic and routes everyone to A and C until B recovers. Your users never notice a server died. That's the payoff for all this work.

The catch nobody warns you about: state

Here's the trap, and it's the most important idea in this whole piece.

Say a user logs in. Their request happens to land on server A, and server A remembers "this person is logged in" by storing it in its own memory. Their next request gets sent by the load balancer to server B. But server B has never heard of this user. Suddenly they're logged out, for no reason they can see.

The problem: server A kept something in its own memory that the other servers don't know about. That remembered thing is called state, and the moment you have more than one server, state stored on a single server breaks.

The fix is a rule: keep your servers stateless. Anything that needs to be remembered between requests gets pushed out to a shared place all the servers can reach. The server itself becomes a forgetful worker. It does the job in front of it, remembers nothing, and any of them is as good as any other.

Two common ways to hold that shared session state: put it in a store every server reads (a database, or a fast in-memory store like Redis), or push it to the client itself as a signed token (a JWT) the browser sends on every request, so the server can verify it without looking anything up. Either works. The point is the same: the truth about who's logged in does not live in one server's head.

This rule is why horizontal scaling works at all. Interchangeable servers can only be interchangeable if none of them is secretly holding something the others need.

Sticky sessions: the tempting shortcut

There's an easier-looking way out, and it's worth knowing because people reach for it. You can tell the load balancer: "once a user lands on server A, always send that user back to A." Now server A's memory of them stays correct, because they never go anywhere else. This is a sticky session.

It works, but it quietly gives back most of what you came for.

Reach for stickiness only when going fully stateless is genuinely hard for some legacy reason, and treat it as temporary.

Where this leaves you

Notice the shape of what just happened. Every move was forced by a concrete problem:

1. One server gets overloaded

   Rent a bigger one. Cheap, simple, and it buys real time. (Vertical scaling.)

2. A bigger server still has a ceiling and can still die

   Run several ordinary servers instead, so work spreads and one failure isn't fatal. (Horizontal scaling.)

3. The browser doesn't know which server to talk to

   Put a load balancer in front to spread requests and route around dead servers.

4. Users get logged out at random across servers

   Keep servers stateless: push sessions, files, and counters to a shared store every server can read.

That's the core of scaling a web app's compute layer. You'll layer more on top later (caching, queues, splitting the database — each its own explainer), but they're all variations on the same instinct you just built: find the thing that breaks, add the smallest piece that fixes it, and keep your servers boring and replaceable.