System Design - How JioHotstar Handles 100 Million Concurrent Viewers (Case Study)
In live sports 100 million users hit the system concurrently. If you have only 1000 servers each server would need to handle 100,000 connections
The design discussed below is a high-level speculation based on common industry best practices for massive scale live streaming, known architectural requirements (like CDN and pre warming), and public domain information. The actual implementation and technologies used by JioHotstar may differ significantly.
Imagine a major cricket match between 🇮🇳India and 🇵🇰Pakistan. 100 million people are sitting at home eyes glued to their devices. At the start of the match or when a wicket falls the demand on the streaming service JioHotstar goes from massive to truly catastrophic in a single second.
100 million people all requesting the exact same video segment at the exact same moment creates an event known as a Flash Crowd. This is the ultimate test of a system’s resilience and capacity.
How does a system built on basic building blocks like the ones we discussed survive this kind of intense sudden demand without lagging or crashing? The answer is not one single trick but a masterclass in Decoupling Distribution and Extreme Capacity Planning.
The Problem of the Flash Crowd
In a typical system design the load from 100 million users is spread out over time. Some people log in now some later.
In live sports 100 million users hit the system concurrently. If you have only 1000 servers each server would need to handle 100,000 connections. No single server or data center can handle that heat. The moment the demand hits the infrastructure would collapse and the match would lag for everyone.
The solution is to move the workload away from the central data center and closer to the user.
Pushing to the Edge
The primary design strategy for handling live streaming at this scale is to use a Content Delivery Network (CDN) a tool we introduced as a key building block (Blog 3).
A CDN is a geographically distributed network of proxy servers and data centers. Think of it as an express delivery service for content.
CDN Decentralization
Instead of the video originating from one server in Mumbai and traveling 2,000$miles to a user in Chennai the CDN has local delivery points or Edge Locations in Chennai Bangalore and every major city.
The Origin Server (Source)
The JioHotstar central data center where the video stream is created only has to send the video stream out once or twice to the nearest major CDN hub. This is its only major task.
The Edge Servers (Delivery Points)
The CDN copies the stream to thousands of small servers closest to the users.
When 100 million people request the stream they are not hitting JioHotstar’s main servers. They are hitting 10,000 different local CDN servers. The load is divided by a factor of 10,000 making the 100 million requests manageable.
Caching Live Video Segments
The CDN does not just deliver files it caches them intelligently. Video streams are broken into tiny segments usually 2 to 10 seconds long (using protocols like HLS or DASH).
The CDN is constantly requesting and caching these small $5$ second segments.
The 100 million users are all requesting the exact same segment e.g. the segment that holds 300 to 305 seconds of the match.
Because the CDN server is serving the same identical piece of data to millions of users it becomes hyper efficient. The data is already in the server’s fast memory ready to be instantly delivered.
This is the most powerful use case for caching at scale.
Capacity Planning and Pre Warming
Handling this scale is not a reactive process it is a predictive one. Hotstar and their CDN partners use intensive Capacity Planning (Blog 7) months in advance.
Estimating Peak Load
Engineers do not plan for 100 million users. They plan for 120 million users plus a buffer. They forecast the peak traffic rate measured in Requests Per Second (RPS) and Bandwidth required.
If a video stream needs 10 Mbps (megabits per second) of bandwidth 100 million users require 1,000 Petabits per second of total network capacity. This is an unfathomable amount of data.
This estimate tells them exactly how many edge servers they need to turn on in every city and how much total bandwidth they must reserve from the internet providers.
Pre Warming the System
A critical step is Pre Warming. A normal CDN only caches content when the first user asks for it. For a 100 million user flash crowd that is too late.
Before the match starts the CDN infrastructure must be pre warmed. This means the system artificially sends requests to all the edge servers globally forcing them to load up the initial video segments into their fast memory.
When the 100 million users hit the system the CDN does not have to scramble to find the data. It is already sitting in the fastest possible memory ready to be served instantly. This proactive step eliminates the initial lag that would otherwise crash the system
Reliability and Decoupling
While the CDN does the heavy lifting for the video data JioHotstar’s own core systems have two major responsibilities that require high availability (Blog 5).
The Authentication Service (C P)
When you log in the authentication service must confirm your subscription and identity. This is a Consistency C P problem (Blog 8). You cannot allow a paid user to be denied access or a free user to bypass the paywall.
This service is kept separate from the streaming engine and is highly replicated across multiple availability zones.
It is designed to prioritize Consistency meaning it will take a fraction of a second longer to verify your account to ensure the result is correct.
The Live Scoring and Metadata Pipeline
The most critical non video element is the live score updates graphics and statistics. These need to be delivered to the user with the lowest possible latency.
This is handled by a separate WebSocket or Low Latency Messaging Service. This is an Asynchronous Design pattern (Blog 9 coming soon) where the server maintains a persistent open connection to the client.
When the score changes the server pushes the tiny data update instantly over this open connection. This keeps the score graphics instant without delaying the heavy video stream.
Failover and Redundancy
Because they use multiple CDN providers (e.g. Akamai Cloudflare etc.) JioHotstar achieves ultimate redundancy.
If one entire CDN provider in a region collapses the system instantly redirects all traffic to the other CDN provider.
This requires a sophisticated Load Balancer or a DNS Routing Strategy that constantly monitors the health of the providers and can failover in milliseconds. This ensures that even massive network failure does not interrupt the stream.
Conclusion
JioHotstar’s ability to handle 100 million simultaneous viewers is a demonstration of pure scaling principles.
It is a system that uses Decoupling to separate the user identity C P from the video delivery A P. It relies on Capacity Planning to know the exact load and Pre Warming to be ready for the moment the load hits. Above all it uses a massively distributed CDN Architecture to push the heaviest workload as close to the user as possible making 100 million requests look like 100 thousand requests to any single server.
This case study gives us the perfect context for our next core topic. We saw how the score updates and the video stream are separate.