A VOD box is two machines pretending to be one: a large cold catalog and a small hot working set. Price them separately. From 2.7 GB per 1080p hour through 96 TB raw becoming 52 TiB usable, to why range requests are the argument for an NVMe tier.
A video on demand server is two machines pretending to be one. There is the catalog — large, cold, mostly untouched — and there is the working set, the handful of titles everyone is watching this week. Size a single box for both and you overpay for one and starve the other. Every good VOD build starts by separating those two numbers and pricing them independently.
Start from hours of content and the bitrate you store at.
At 6 Mbps, one hour of 1080p is 6 × 3600 / 8 = 2,700 MB, or about 2.7 GB per hour. A 90-minute feature is roughly 4 GB per rendition. Multiply by your ladder: if you store four renditions rather than transcoding on the fly, a single film costs 8–10 GB all in.
Now work backwards from a real chassis. A storage server with 8 × 12 TB drives is 96 TB raw. Two of those are parity in RAIDZ2 or RAID 6, leaving 72 TB. Filesystems report in binary units, so 72 TB decimal reads as about 65 TiB. ZFS wants to stay under roughly 80% full to avoid fragmentation, so plan on 52 TiB of comfortable working capacity from a 96 TB chassis.
At 2.7 GB per hour that is about 21,000 hours of single-rendition 1080p — call it 14,000 feature films, or a few thousand with a full ABR ladder stored. If your catalog is smaller than that, one node holds everything and the architecture question disappears.
The catalog is not what your disks actually serve. Viewing is heavily concentrated: a small fraction of titles takes the large majority of plays, and that fraction moves slowly enough to cache.
Measure yours before choosing hardware. Rank titles by plays over 30 days, take the cumulative share, and find the point where it crosses 90%. That set — usually a few percent of the catalog by size — is what needs to be fast. Everything else can live on spinning disks and be read at whatever speed a sequential stream needs, which is not much.
That gives the standard build:
The difference matters because random-read latency is the failure mode. A sequential read from an HDD array is fine; twenty thousand concurrent viewers seeking around a catalog is not, and NVMe vs SATA vs HDD has the per-device IOPS numbers that decide where the line falls.
Egress is the simple part, and the part that costs money:
egress = concurrent streams × average bitrate × 1.05
At a 3 Mbps blended ABR average, a 5 Gbps guaranteed uplink carries about 1,600 concurrent streams and a 10 Gbps uplink about 3,300. Add 30% headroom over your measured peak, then buy the tier — the 5 Gbps range is where most single-node catalogs land, and larger catalogs behind a CDN can sit lower still.
Two VOD-specific details change the number:
Range requests. Players seek. Every seek is a new HTTP range request against a large file, which turns one sequential read into many partial ones. Serving from HDD, a seek-heavy audience produces far more IOPS than the stream count suggests. This is the single strongest argument for an NVMe hot tier.
Trailers and thumbnails. Small files, requested constantly, cached badly if you have not set headers deliberately. They are a rounding error in bytes and a significant fraction of your request rate.
Put a CDN in front and your origin stops serving viewers — it serves edges. Steady-state egress drops by whatever your hit ratio is, often 95% or better.
Do not size for that steady state. Size for the fill burst: a purge, a new release, or a cold region puts every edge on your origin simultaneously, and that moment is exactly when someone is watching. CDN origin server works through why the burst, not the average, sets the port.
With a CDN, the origin's job shifts from throughput to durability and correct cache headers — s-maxage, immutable segment URLs, and proper 206 range support. Get those wrong and your hit ratio collapses, which is the same as having no CDN and a bill from one.
For a catalog under 50 TB with a concentrated working set:
| Component | Choice | Why |
|---|---|---|
| Disks | 8 × 12 TB HDD, RAIDZ2 | 72 TB usable, cheapest cold capacity |
| Cache | 2 × NVMe, mirrored | The 90% working set, at IOPS |
| RAM | 128 GB or more | Page cache and ZFS ARC do the real work |
| Uplink | 5–10 Gbps guaranteed | Peak concurrent × blended bitrate × 1.3 |
| Filesystem | ZFS, recordsize=1M |
Large sequential media files |
That is close to the standard video-on-demand origin build, and it exists because the shape of the workload keeps producing the same answer. Catalogs past a few hundred terabytes stop fitting one chassis and become a storage tier plus separate origin nodes — a different conversation, and one worth having before you buy rather than after.
How much storage does a video on demand server need per hour of content? About 2.7 GB per hour at 1080p and 6 Mbps, per rendition. Multiply by the number of renditions you store rather than transcode on demand. 4K at 12 Mbps roughly doubles it.
Should I transcode on demand or store every rendition? Store them if you can afford the space; on-demand transcoding turns a storage cost into a CPU cost and adds startup latency to every play. A hybrid — store the popular ladder, transcode the long tail — is common and works well.
Does RAID level matter for VOD? Yes, for rebuild time more than for performance. A 12 TB drive resilvering in RAIDZ2 takes many hours during which a second failure is survivable; in RAIDZ1 it is not. Choosing a RAID level covers the trade-off.
Can one server handle both live and VOD? Technically yes, but the workloads fight: live wants small random writes and predictable latency, VOD wants large sequential reads and capacity. Separating them is usually cheaper than the box that does both well.
What uplink for 10,000 concurrent viewers? At a 3 Mbps blended average, about 31.5 Gbps with overhead, 41 Gbps with headroom — a 50 Gbps build, or a smaller origin behind a CDN. The CDN is almost always cheaper at that scale.
A streaming server does three jobs with three different bottlenecks. Ingest wants path quality, transcode wants cores, egress wants a port. Size them separately, work a real build end to end, and see why the blended ABR bitrate — not the top rung — sets the uplink.
Monthly transfer is the wrong starting point, because a port is a rate. Size from peak concurrency, per-unit bitrate and headroom — then check the cost per gigabit of each tier, because that curve is not monotonic and the cheapest gigabit is not where most people buy.
The edges take the load until a purge, a deploy or a cold region sends every one of them to the origin at once. Hit-ratio arithmetic, the cache headers that are the whole contract, and which machine to actually build.