Game patches, ISOs, model weights and datasets all load a server the same way: big files, many readers, dropped connections. How to size the port from arithmetic, keep the hot set in page cache or on NVMe, set nginx sendfile, kTLS and range requests, and get through a 40 TB release day.
A file download server has one job: move large files to many people at once, and let them resume when a connection drops. Game patches, Linux ISOs, model weights and datasets all load a server the same way — big files that never change once published, read start to finish by hundreds of clients in parallel.
The port has to carry concurrent downloads × per-client rate. At 8 bits to the byte, 1 Gbps is 125 MB/s.
| Port | Bytes per second | Clients at 100 Mbps | 8 GB downloads per hour |
|---|---|---|---|
| 10 Gbps | 1.25 GB/s | 100 | 562 |
| 20 Gbps | 2.5 GB/s | 200 | 1,125 |
| 50 Gbps | 6.25 GB/s | 500 | 2,812 |
| 100 Gbps | 12.5 GB/s | 1,000 | 5,625 |
At 20 Gbps, 2.5 GB/s × 3,600 s = 9,000 GB an hour, or 1,125 copies of an 8 GB patch. Add about 30% headroom above your expected peak.
You do not choose the per-client rate. One TCP connection carries at most its window divided by the round-trip time. The kernel documentation caps the auto-tuned send buffer at up to 4 MB by default, so a client 100 ms away tops out near 335 Mbps (4,194,304 bytes × 8 ÷ 0.1 s) however large your port is. Raise net.ipv4.tcp_wmem for distant users, and plan on a blended rate.
20 Gbps needs 2.5 GB/s of reads; 50 Gbps needs 6.25 GB/s.
Page cache first. Linux keeps recently read file data in free RAM. Most catalogues have a small hot set and a long tail. If the hot set fits, the disks read each file once and later downloads come from RAM. fincore from util-linux shows how much of a file is resident:
fincore /srv/files/game-1.42-patch.pak
A 128 GB server holds several 8 GB patches. A 140 GB model checkpoint (70 billion parameters × 2 bytes at 16-bit precision) does not fit, which argues for a 256 GB build.
NVMe for what does not fit. A PCIe 4.0 ×4 link carries at most about 7.9 GB/s (16 GT/s × 4 lanes × 128/130 ÷ 8), so one drive serving 50 Gbps runs close to its ceiling. Spread hot files over several.
HDD for the archive, not the release. Hard drives handle one sequential reader well and hundreds at different offsets badly, because every switch is a seek. Measure with many readers:
fio --name=readers --directory=/srv/test --rw=read --bs=1M \
--size=4G --numjobs=64 --direct=1 --group_reporting
If the aggregate falls short of the port, keep the hot set on NVMe or in RAM. Saturating a 100 Gbps port runs the same chain one tier up.
Several settings that matter are off by default:
http {
sendfile on;
sendfile_max_chunk 2m;
tcp_nopush on;
aio threads;
open_file_cache max=10000 inactive=60s;
limit_conn_zone $binary_remote_addr zone=perip:10m;
server {
listen 443 ssl;
ssl_conf_command Options KTLS;
location /files/ {
root /srv;
limit_conn perip 8;
max_ranges 1;
add_header Cache-Control "public, max-age=31536000, immutable";
}
}
}
sendfile on has the kernel copy file data straight to the socket, which the Linux man page calls more efficient than read() plus write() through user space.sendfile_max_chunk 2m is the default since nginx 1.21.4; before that one fast client could monopolise a worker.tcp_nopush on sets TCP_CORK on Linux, so the header and the start of the file share a packet and the rest goes out in full packets.aio threads moves blocking disk reads to a thread pool, so a worker waiting on a cold disk does not stall its other clients. It needs nginx built --with-threads; nginx -V tells you.directio, not set here, bypasses the page cache above a given size and disables sendfile there. Reserve it for huge, rarely repeated files that would evict the hot set.Over HTTPS, nginx must otherwise pull file data into user space to encrypt it — the very copy sendfile avoids. Since 1.21.4, nginx can hand encryption to the kernel through SSL_sendfile(), given OpenSSL 3.0 or later and the KTLS line above; the nginx write-up recommends Linux 5.2 or newer. Check the debug log for BIO_get_ktls_send(): 1 before counting on it.
A 140 GB checkpoint at 100 Mbps takes 11,200 seconds (140 × 8,000 Mbit ÷ 100), just over three hours. Connections will drop, and the client should resume. Range requests, section 14 of RFC 9110, make that work: the server sends Accept-Ranges: bytes, the client asks for Range: bytes=52428800-, and the server answers 206 Partial Content with only the missing bytes. nginx does this for static files out of the box.
Three details keep it safe:
If-Range, sending back the ETag they saw; nginx derives its ETag from modification time and size, and on a mismatch sends the whole file instead. Publish each version under a new name.max_ranges 1, nginx ignores multi-range requests and sends the full file. RFC 9110 permits this, warning that many small or overlapping ranges can signal a denial-of-service attempt. Download managers ask for one range per connection, so they keep working.limit_conn keys on client IP, and carrier-grade NAT can put hundreds of users behind one; eight per IP stops one accelerator hoarding slots without locking out a campus.On release day, divide total bytes by the time you are willing to let it take. Five thousand players fetching an 8 GB patch is 40 TB:
40 TB × 8 = 320,000 Gbit
at 10 Gbps 320,000 ÷ 10 = 32,000 s ≈ 8.9 hours
at 20 Gbps 320,000 ÷ 20 = 16,000 s ≈ 4.4 hours
at 50 Gbps 320,000 ÷ 50 = 6,400 s ≈ 1.8 hours
Demand spikes in the first hour, so these are best cases. limit_rate caps each request (two connections get twice the limit, the nginx docs note), so more people progress at once, but 40 TB still takes as long to move. What helps:
cat patch.pak > /dev/null and confirm with fincore.A CDN earns its fee when users span continents, since round-trip time caps every connection, or when one release could exceed any port you would rent. It is poor value for a known audience in one region, where per-gigabyte pricing on tens of terabytes soon outruns a flat port. If you use one, versioned names with immutable caching mean your origin sees one fetch per edge location, not one per user. Building a CDN origin server covers sizing it.
Mirrors are the middle path between one file download server and a CDN, and how Debian has long shipped its ISOs. Put two or three servers near your users — Frankfurt or Amsterdam for Europe, Ashburn for North America — sync them with rsync, and have a small redirector answer each request with a 302 to the nearest healthy mirror. Publish SHA-256 checksums from one place, and move the "latest" link only once every mirror holds the file. Choosing a server location maps cities to audiences.
Multiply concurrent downloads by the average per-client rate and add about 30% headroom: two hundred clients at 100 Mbps need 20 Gbps before headroom. For a release, divide total bytes by the time you accept. Forty terabytes in under two hours needs about 50 Gbps.
For an archive read by a few clients at a time, yes. For a popular release with hundreds of clients at different offsets, hard drives spend their time seeking. Keep hot files in page cache or on NVMe, and test the array with 64 parallel fio readers first.
Yes. It is off by default and lets the kernel move file data to the socket without copying it through nginx. Pair it with tcp_nopush and aio threads. Over HTTPS it only helps with kernel TLS active, which needs nginx 1.21.4 or later and OpenSSL 3.0.
The server advertises Accept-Ranges: bytes. After a dropped connection, the client requests the missing part with a Range header, and the server replies 206 Partial Content with only those bytes. An If-Range header carrying the file's ETag stops the client stitching two versions of a file together.
Running the S3 API on your own disks: how Garage, SeaweedFS, Ceph RGW and MinIO compare in November 2026, how much of a 96 TB server each redundancy model leaves usable, and what 50 TB costs on a storage server against Amazon S3 and Cloudflare R2.
A transcoding box is sized by its preset, not its spec sheet: x264's own presets span roughly 100× in speed. CPU encoders against NVENC and Quick Sync, why live channels and VOD queues size differently, and the ffmpeg -benchmark run that gives you channels per server on your own clips.
A bare metal server gives one customer the whole machine: every core, drive and network card, with no provider hypervisor in between. The three products sold under the name, what you gain and give up against a cloud VM, and why egress usually decides the move.