Load balancers, API gateways, proxies, and firewalls

Module 8 left an unanswered question at the door. During the dinner rush, autoscaling means the coffee backend is suddenly twelve instances instead of two.

But Erik’s phone sends one request to one address: api.coffeeapp.com.

Which of the twelve machines answers it?

The answer is a small family of components that stand at the entrance of nearly every serious system: load balancers, API gateways, proxies, and firewalls.

This lesson introduces each one, then assembles them into the front door you will see on real architecture diagrams.

What is a load balancer?

A load balancerConcept · lights on your mapload balancerThe component that receives incoming requests and distributes them across the many identical instances behind it, checking which are healthy, skipping the sick, and including the newly added. The reason twelve machines look like one calm address from outside. receives incoming requests and distributes them across the identical instances behind it.

You met it briefly in Module 8, and now it takes its place on the diagram.

The load balancer’s job has three parts:

• Spread the traffic. Requests are distributed across the instances — often in rotation, or toward whichever instance is least busy.
• Skip the sick. The load balancer runs the health checks from Module 8. An instance that stops answering stops receiving traffic.
• Absorb the changes. When autoscaling adds a thirteenth instance, the load balancer starts using it. When instances are removed, it stops. The outside world never notices.

From the outside, the herd of machines looks like one stable address, and that illusion is the load balancer’s entire purpose.

A useful principle is: clients talk to the address. The load balancer decides which machine that means right now.

What is an API gateway?

An API gatewayConcept · lights on your mapAPI gatewayThe smart front door for APIs, reading each request and routing it to the right service (/orders → order service) while handling the shared chores at the threshold: validating tokens, enforcing rate limits, logging arrivals. Module 6’s machinery, drawn as one box. is a front door that reads each request before directing it.

In a microservices world, something must know which service owns which request:

The gateway is that something.

And because every request passes through it, the gateway is also the natural home for the chores every service would otherwise duplicate:

• Validating authentication tokens
• Enforcing rate limits
• Rejecting malformed requests early
• Logging what arrived, from whom, and how long it took

Recognize the residents? This is Module 6’s toolkit of tokens, rate limits, and request validation, given one shared house on the diagram so that the order service can concentrate on orders.

Gateway versus load balancer

The two are often confused, so here is the clean distinction.

A load balancer spreads copies of the same traffic across identical instances. It asks: which machine takes this one?

An API gateway routes different requests to different services and applies API-level rules. It asks: what is this request, and who owns it?

In practice, the roles blur. Many gateway products also balance load; many modern load balancers can route by path. Real systems often use both, with a gateway deciding which service and load balancing deciding which instance of that service.

On a whiteboard, do not panic about which label the box carries. Ask what the box is doing: spreading, routing, or both.

What is a proxy?

Both of these doors belong to a general family, and the family name is worth knowing.

A proxyConcept · lights on your mapproxyAny intermediary that forwards requests on another’s behalf. A (forward) proxy stands in front of clients (the corporate proxy that office traffic exits through), speaking to the world so the clients don’t have to. is any intermediary that forwards requests on behalf of someone else.

The classic forward proxy stands in front of clients.

A company’s office network might send all outbound web traffic through a corporate proxy. The proxy speaks to the internet so employee machines do not have to, and along the way it can filter blocked sites, log activity, and cache common responses.

From the website’s perspective, the request came from the proxy, not from the individual employee’s machine.

What is a reverse proxy?

Flip the arrangement around and you get the reverse proxyConcept · lights on your mapreverse proxyAn intermediary standing in front of servers, receiving outside requests and forwarding them inward. Load balancers and API gateways are specialized reverse proxies. “Reverse” because it faces the opposite direction from the classic client-side proxy.: an intermediary standing in front of servers.

It receives the world’s requests and forwards them inward to the machines behind it.

This is the family both of today’s stars belong to. A load balancer is a reverse proxy specialized for spreading, and an API gateway is a reverse proxy specialized for routing and API chores.

Reverse proxies commonly pick up other doorway jobs too:

• TLS termination: handling the HTTPS encryption handshake from Module 4 at the door, so internal services behind it are simpler
• Compressing responses before they travel to users
• Serving cached or static content without bothering the backend

One name worth recognizing in conversation is nginx (pronounced “engine-X”), open-source software used everywhere as a reverse proxy and load balancer.

A useful summary is: a proxy stands in front of clients. A reverse proxy stands in front of servers. The front door of a system is built from reverse proxies.

What is a firewall?

Guarding all of these doors is the firewallConcept · lights on your mapfirewallThe network guard enforcing rules about what traffic may pass at all: “web traffic to the front door, yes; connections from the internet to the database, never.” The reason the database is unreachable except through the front door.: a system that enforces rules about which network traffic may pass at all.

Firewall rules commonly consider:

• Where the traffic is coming from
• Where it is trying to go
• Which port and protocol it uses

The most important rule in nearly every real system is the one about the database:

Nothing on the internet talks to the database directly.

The database from Module 7, the product’s source of truth, accepts connections only from the backend services inside the walls. The only path from the outside world to that truth runs through the front door, past the token check, through the business logic.

Cloud platforms express these rules as configuration. On AWS, for example, they take the form of security groups attached to instances and services. That makes firewall rules part of Module 8’s infrastructure-as-code story: reviewable text, not tribal memory.

Module 14 will have much more to say about why this wall matters.

Assembling the front door

Put the whole family on one diagram:

Trace Erik’s tap through every door.

His request reaches api.coffeeapp.com. The firewall rules admit it, since it is ordinary HTTPS traffic to the public entrance. The load balancer picks a healthy machine. The gateway checks his token, confirms he is under the rate limit, and routes /orders to the order service. The order service does its work against the private database and the answer travels back out the same way.

Real systems vary the arrangement. Boxes merge, order shifts, and managed cloud services bundle several roles into one product, so the roles, not the exact boxes, are what to look for on a diagram.

The door is a component too

One caveat completes the picture.

Everything passes through the front door, which means the front door is itself a component whose failure would matter enormously. If the load balancer stopped working, twelve healthy instances would sit behind a door nobody could open.

This is why production front doors are themselves redundant. Managed load balancers run as multiple instances across availability zones, exactly the Module 8 machinery, applied to the door itself.

A component that everything depends on has a name, a single point of failure, and hunting them is one of Module 10’s main sports.

The mental model to remember

A load balancer spreads incoming traffic across identical instances, skipping unhealthy ones and absorbing autoscaling’s changes.

An API gateway reads requests, routes each to the right service, and handles the shared chores (tokens, rate limits, logging) at the threshold.

A proxy is an intermediary that forwards requests. Forward proxies stand in front of clients; a reverse proxy stands in front of servers, the family both load balancers and gateways belong to.

A firewall enforces rules about what traffic may pass at all, most importantly that the internet never reaches the database directly.

Together they form the front door: admit, spread, inspect, route, with the truth kept private behind it.

And because everything passes through it, the front door must itself be redundant, or it becomes the system’s single point of failure.

You should now be able to look at the entrance of any architecture diagram and name what each box is doing: guarding, spreading, or routing.