System Design - How Netflix Streams to Millions Globally (Case Study)
To serve millions of users across 190 countries streaming at different qualities all at the same time requires a truly distributed and robust system.
The design discussed below is a high-level speculation based on common industry best practices for global content streaming, known architectural requirements (like CDN and microservices), and public domain information. The actual implementation and technologies used by Netflix (specifically their Open Connect system) may differ significantly.
Netflix is more than just a massive library of movies. It is a system design marvel built to handle peak traffic equivalent to almost 40% of all internet traffic in North America during evening hours.
To serve millions of users across 190 countries streaming at different qualities all at the same time requires a truly distributed and robust system. The core design challenge for Netflix is overcoming distance and latency to deliver large video files instantly. The solution is found in three parts The Central Brain The Global Network and The Recommendation Engine.
The Microservices Layer
The Netflix core architecture is a textbook example of a Microservices Architecture (Blog 4). Everything you do before you hit the play button is handled by a central cluster of services running on cloud providers like Amazon Web Services (AWS).
The Gateway Service
Every user request whether from a TV a phone or a web browser hits the API Gateway. This single point of entry handles authentication rate limiting and routes the request to the correct internal service.
The Metadata Service
This is the database layer that knows everything about every title. It handles titles descriptions subtitles cover art and regional availability. This service experiences a high Read Write Ratio favoring reads so it is heavily cached and often relies on Eventual Consistency to ensure speed.
The Recommendation Service
This is the true intellectual property of Netflix. It is a highly specialized service that calculates exactly which titles to show you in your personalized grid. Since these calculations are complex and do not need to be instant this service often uses Asynchronous Processing (Blog 9) working in the background to pre generate your viewing options.
The central brain handles complexity and personalization. But it does not handle video delivery.
The Video Delivery Layer
A central server in the US cannot stream a 5 GB movie to a user in India without massive lag and high bandwidth costs. Therefore Netflix does not use AWS or another cloud provider for video delivery. Instead they built their own global Content Delivery Network called Open Connect.
Open Connect Appliance (OCA)
The cornerstone of this system is the Open Connect Appliance (OCA).
An OCA is a specialized, high capacity Linux server designed to do one thing serve video streams as fast as possible.
Netflix places these OCAs directly inside the internet service provider (ISP) networks of almost every major region around the world.
This is the ultimate form of Decentralization and pushing work to the Edge a concept we saw in the JioHotstar case study.
Pre Caching and Distribution
The OCA deployment relies heavily on Capacity Planning (Blog 7) and prediction.
Netflix knows what people in a certain region are likely to watch next based on global trends and local demand.
Long before the user clicks play the video files for the top 500 most popular titles in that region (e.g. India) are proactively copied and stored onto the local OCA inside the ISP’s network in that city (e.g. Mumbai).
When a user hits play in Mumbai the request routes through the Central Brain which quickly authorizes the request and redirects the user to the closest OCA right inside their local ISP network. The distance the video stream travels is often less than a mile. This near zero distance eliminates lag and network congestion.
The Recommendation Engine
The design of the recommendation service is a critical component for system success because it keeps users engaged.
The Data Pipeline
Every action you take (pause search rewind 10 seconds scroll past a title) is an Event. These events are collected by the Logging Service and sent through a massive Event Streaming pipeline (like Kafka).
The Pipeline
The stream of events is collected aggregated and processed by machine learning models.
The Outcome
The models constantly output new Rankings of titles for millions of users.
Pre Computation and Cache Hit
These rankings are stored in a fast database or cache often a Key Value Store.
When you log in the system does not calculate your feed. It simply retrieves the pre calculated list of 50 rows and 10 columns of titles that the system already decided you want to see.
This move from computation to retrieval is how Netflix achieves instant screen loading even though the calculation itself is incredibly complex and data heavy.
Conclusion
Netflix’s architecture is a fantastic example of a hybrid design. They use the agility of cloud-based microservices to handle the business logic (authentication profiles recommendations) while using a massive specialized CDN (Open Connect) to handle the heavy lifting of video delivery. The key design principles are clear.
Decouple - The Brain from The Brawn (Microservices from Video Delivery).
Distribute - Video files to the furthest edge of the network (OCAs).
Optimize Reads by pre calculating and caching all recommendation results.
You now have a deep understanding of core theory and how it applies to real world giants like Twitter, JioHotstar and Netflix.
In the final blog post of this series “The System Design Interview A Comprehensive Example” we will bring all nine concepts together. We will take a classic design problem like Designing a URL Shortener and use every concept we have learned from functional requirements to sharding to asynchronous queues to build a complete and scalable architecture.