System Design - From Monolith to Scalable Architectures
Every system starts somewhere simple but as your product grows in users features and engineering team size your architecture must evolve.
In our last two posts we learned to gather requirements and identified the basic building blocks we need APIs databases caches and load balancers.
Now we face the first major design decision how do we organize these blocks? Do we build one giant machine that does everything or many small specialized machines that work together?
This is the question of architecture. Every system starts somewhere simple but as your product grows in users features and engineering team size your architecture must evolve. If it does not, you will hit a wall.
This guide explores the journey of an application from a simple starter system to a complex distributed environment. We will look at scaling strategies and the three major architectural patterns that define modern software development.
The Monolith (To Start With)
The monolith is where most great applications begin. A monolith or monolithic architecture is a single massive application that contains all the code for all the business functions in one codebase and is usually deployed as one single file or process.
A monolith is like a tiny apartment where the kitchen the bedroom the bathroom and the living room are all in one giant space.
Why We Love the Monolith
For new projects the monolith is fantastic.
Simplicity
It is easy to develop because all the code is right there. A developer only needs to open one project.
Easy Deployment
You only have one thing to deploy one server to manage.
Debugging
Tracing a bug is simple because all functions call each other directly within the same memory space. There is no complex network communication involved.
Why the Monolith Eventually Breaks
The simplicity that makes the monolith great early on is its greatest weakness later.
Scaling Inefficiency
Imagine your search feature is used constantly and is slowing everything down. In a monolith the search feature and the user profile feature run on the same server. You have to scale both even if the user profile feature is barely used. This is wasteful and expensive.
Slow Development
As the codebase grows massive it takes a long time to build and test. Deploying a tiny bug fix might require redeploying the entire giant application which is risky.
Technology Lock In
Since all the code is one unit you must use one language and one framework for everything. You cannot use the latest fast database language for one part and a classic stable language for another.
When the pain of the monolith becomes greater than the pain of managing complexity it is time to scale.
The Two Ways to Scale
Scaling is the process of handling more work. There are two fundamental approaches.
Vertical Scaling (Scaling Up)
This means adding more resources to your existing single server. You upgrade your server to have more RAM more CPU and more storage. You make the server bigger and stronger.
When your single delivery truck gets too slow you buy a bigger faster truck.
It has one and important benefit that, it is simple and requires almost no change to your application code.
But it has several drawbacks
Hard Limit
Eventually you reach the largest server that exists. You cannot scale past that.
Cost
The cost of server power does not scale linearly. Going from an 8 CPU server to a 16 CPU server often doubles the price. Going to a 32 CPU server might quadruple it.
Single Point of Failure
If that one massive server crashes your entire application is 100% down.
Horizontal Scaling (Scaling Out)
This means adding more instances of the same small server and distributing the load across them using a Load Balancer (a component we discussed in Blog 3).
Instead of one huge truck you buy a fleet of 10 smaller identical delivery vans.
This Scaling Method has several benefits
Limitless
You can always add another small server. You are not constrained by hardware limits.
Availability and Redundancy
If one small server crashes the other nine keep running. This is crucial for achieving high availability.
Cost Efficiency
Small servers are often cheaper per unit of power than massive servers.
But it also has some drawbacks, it is much more complex. Now you have to worry about network issues and making sure that server 1 and server 5 are not stepping on each other’s toes when writing to the database.
The Modern systems almost always prioritize Horizontal Scaling because it is the only way to achieve true unlimited scale and high availability.
The Architectural Evolution
Once you commit to horizontal scaling, your architecture itself needs to change. Engineers use three main patterns to organize code around this reality.
The Monolith (Initial State)
As defined before all code is one unit. Great for small teams and new projects.
The Modular Monolith
A modular monolith is a single deployable application but with strict internal separation. Imagine your one giant codebase is divided into clear modules User Management Orders and Billing.
Key Idea
The code inside the Orders module cannot directly access the database of the Billing module. It must talk through a small internal API.
Benefits
This approach maintains the simplicity of a single deployment but gains the maintainability of separate parts. It forces developers to write cleaner code and prepares the application for a future move to microservices by creating those strict boundaries now.
Many mid-sized companies never need to leave the modular monolith. It can offer 90% of the benefits of microservices with only 10% of the complexity.
The Microservice
A microservices architecture is a collection of small completely independent services.
Each service (e.g. the “Inventory Service” or the “Payment Service”) has its own codebase its own database and is deployed independently.
They communicate with each other over the network usually via APIs or message queues.
This is like a massive food court. The Pizza shop the Burger shop and the Chinese food place are all separate businesses with separate cash registers separate kitchens and separate suppliers.
Here are some major benefits of Microservice Architecture
Extreme Scalability
If the Pizza shop gets busy you scale only the Pizza shop servers.
Team Autonomy
Different engineering teams can own and deploy their service without coordinating with others.
Technology Diversity
The Inventory Service can use a high-performance Java server while the Reporting Service uses a Python data analysis library.
This approach also have some drawbacks
Complexity
The system is now a network of communicating parts. Debugging becomes hard (if Service A calls Service B and Service C) and the network is always a failure point.
Distributed Transactions
If a user needs to update data across three different services it is difficult to ensure all three succeed or all three fail gracefully. This is one of the biggest challenges of microservices.
The Reason to Scale
Architectural evolution is costly. You do not move to microservices just because it is cool. You move because the business needs demand it.
Deployment Friction
Deploying the monolith takes three hours and breaks production 20% of the time. Moving to microservices allows small services to be deployed in minutes with no downtime.
Technology Bottlenecks
One specific part of the monolith needs a special database (e.g. a Graph Database) or a specific technology that the main monolith language cannot support well. Moving that part into a separate service is the only solution.
Organizational Scale
Your engineering team grows past 50 people. When everyone is working on the same large codebase coordination becomes impossible. Moving to microservices allows you to organize teams around services giving them full independence. This is often the strongest driving factor.
Single Point of Failure
If the monolith crashes the whole company stops. The business risk is too high. Microservices allow for resilience if one service fails the others can continue operating.
The Refactor usually happens gradually. You identify the part of the monolith that is causing the most trouble (the slowest or most frequently updated feature) and you extract it first. This is called the strangler fig pattern where the new service slowly strangles the old functionality until the monolith is just a small husk.
The Next Step
The journey from a monolith to microservices is the story of nearly every successful company Netflix Amazon Etsy all followed this path. The key takeaway is to start simple (monolith) think modular (modular monolith) and scale out horizontally when the business requires it. Never over engineer for a scale you do not have yet.
The decision is a constant trade off between simplicity (monolith) and scalability and resilience (microservices).
Now that we know how to organize our code we need to ensure that organization is durable. Our highly distributed system has many parts that can fail.
In the next post “Designing Reliable Systems” we will focus on how to build systems that do not crash and more importantly how they recover gracefully when they do. We will look at concepts like redundancy health checks and fallbacks.