Logo

Chapter 3 Download Path: Parallel Download, Recovery & Reassembly (Key)

For recipients, “faster” mainly shows up in two ways: fetching multiple chunks in parallel to improve throughput, and being able to recover under weak networks or network switching without restarting. From an engineering perspective, the download path is symmetric to the upload path.

3.1 Download objects and manifest

  • Before downloading, the client SHOULD fetch the manifest to learn total chunk count, order, and how to locate each object.
  • The manifest MUST uniquely locate each chunk (via object-key rules or an equivalent index).

3.2 Parallel download

  • The client SHOULD download multiple chunks concurrently to get closer to the bandwidth ceiling.
  • The client SHOULD use limited concurrency and reduce it when error rates rise.
  • The benefit of concurrency is often larger with high cross-region RTT, because a single connection is more easily limited by RTT and jitter.

3.3 Resumable download

  • The client MUST be able to record the set of completed chunks (e.g., bitmap / index set).
  • After an interruption, the client SHOULD fetch only missing chunks, not re-download the entire file.
  • If the server enforces download-count limits, the client SHOULD minimize redundant downloads during recovery to avoid wasting counts.

3.4 Reassembly

  • The client MUST reassemble the output file in chunkIndex order.
  • The client SHOULD use streaming writes / progressive reassembly to reduce memory peaks (especially for very large files).
  • If multiple files are packaged, the client MAY output per-file or use a container format (depending on product design).

3.5 Integrity checks (short)

  • After download, the client SHOULD verify chunks (e.g., hash/length consistency).
  • If verification fails, it MUST fail closed and retry the corresponding chunk.
  • For boundaries around encryption/authentication failures, see the relevant sections in the Security & Privacy Whitepaper.