HTTP Internals: Protocol Versions, Caching, and Streaming
Understand the structure of an HTTP request/response, the practical differences between HTTP/1.1, HTTP/2, and HTTP/3, how compression and chunked transfer encoding work, how ETag/Last-Modified/Cache-Control drive conditional requests, and how to stream a response from PHP.
Most PHP developers work with HTTP through a thin, friendly layer -- $_GET, $_POST, header(), http_response_code() -- without ever seeing the raw protocol underneath. That's usually fine, but a handful of situations force you to understand what's actually happening on the wire: shaving bandwidth off a high-traffic API with proper caching headers, streaming a large export or an AI response chunk by chunk instead of buffering the whole thing in memory, or just explaining why HTTP/2 made 'domain sharding' obsolete overnight. This lesson goes one layer deeper than headers alone (which the 'HTTP Headers' module already covers in detail) into the protocol mechanics: message anatomy, the real differences between HTTP/1.1, HTTP/2, and HTTP/3, compression, chunked/streaming responses, and conditional caching.
Anatomy of an HTTP Message
Every HTTP request starts with a request line -- METHOD PATH VERSION, e.g. GET /articles/42 HTTP/1.1 -- followed by a set of headers (one per line, Name: value), a blank line, and then an optional body (present for methods like POST or PUT, absent for a typical GET). A response mirrors this shape: a status line (HTTP/1.1 200 OK), headers, a blank line, and a body.
PHP never hands you this raw text directly -- the web server (or, in a Swoole/ReactPHP context, the library itself) parses it and exposes the pieces through $_SERVER (method, path, protocol version, and every request header prefixed HTTP_), php://input (the raw body), and $_GET/$_POST (parsed query string and form body). On the way out, header() and http_response_code() let you set the response's status line and headers before any body output is sent. The full catalog of individual headers -- Content-Type, Cookie, CORS, security headers -- is covered in depth in the 'HTTP Headers' module; this lesson focuses on the protocol mechanics that determine how that anatomy actually travels.
• Request line: METHOD + PATH (+ query string) + PROTOCOL VERSION. • Response status line: PROTOCOL VERSION + numeric STATUS CODE + reason phrase. • Headers describe metadata about the message; a blank line separates headers from the body. • PHP exposes all of this pre-parsed via $_SERVER, php://input, $_GET/$_POST, header(), and http_response_code().
HTTP/1.1 vs HTTP/2 vs HTTP/3
HTTP/1.1 is a plain-text protocol where, in practice, a single TCP connection processes one request at a time -- keep-alive lets the same connection be reused for a second request, but that second request still has to wait for the first one's response to finish (head-of-line blocking at the application layer). Browsers historically worked around this by opening several parallel TCP connections per host (commonly 6), which is also why techniques like 'domain sharding' (spreading assets across fake subdomains to get more parallel connections) were once a real performance trick.
HTTP/2 replaces the plain-text framing with a binary layer that multiplexes many independent 'streams' over a SINGLE TCP connection, so a browser no longer needs multiple connections to get parallelism, and adds HPACK header compression to shrink the (often very repetitive) header blocks sent with every request. Its remaining weak point is that TCP itself only delivers bytes in order, so a single lost packet still stalls every multiplexed stream sharing that one TCP connection until the lost packet is retransmitted -- head-of-line blocking has moved from the application layer down to the transport layer, but hasn't disappeared.
HTTP/3 fixes that by dropping TCP altogether in favor of QUIC, a transport protocol built on UDP with its own per-stream sequencing, so a lost packet only stalls the one stream it belonged to, not every other stream on the connection. QUIC also bakes in TLS 1.3 (there's no unencrypted HTTP/3) and supports much faster connection establishment, including 0-RTT resumption for a client reconnecting to a server it's talked to recently. Crucially for PHP developers: your application code is almost entirely unaware of which version is in use -- the web server (nginx, Apache, Caddy) terminates the actual protocol and hands PHP a normalized request; $_SERVER['SERVER_PROTOCOL'] just reports which one was negotiated.
• HTTP/1.1: text-based, effectively one request in flight per connection, parallelism via multiple TCP connections. • HTTP/2: binary framing, multiplexes many streams on ONE TCP connection, HPACK header compression -- still subject to TCP-level head-of-line blocking. • HTTP/3: TCP replaced by QUIC (over UDP) -- per-stream sequencing avoids head-of-line blocking, built-in TLS 1.3, faster connection setup. • PHP application code is essentially version-agnostic -- the web server negotiates and terminates the protocol before PHP ever runs.
Compression: gzip and Brotli
A request's Accept-Encoding header lists which compression algorithms the client can decode -- typically gzip, deflate, br (br = Brotli, generally the best ratio for text). A response that used one names it in Content-Encoding so the client knows how to decompress the body before parsing it.
In production this is almost always handled at the web-server layer (nginx's gzip/brotli modules, or a CDN) rather than in PHP, because the web server can compress static assets once and cache the compressed bytes, and it sits closer to the network than PHP does. PHP does have a knob for it too -- the zlib.output_compression ini directive compresses PHP's own output automatically -- and for a specific generated download (a report, an export) you can compress on demand with gzencode().
• Accept-Encoding (request): algorithms the client can decode, e.g. gzip, br. • Content-Encoding (response): the algorithm actually applied to this body. • Usually handled by the web server/CDN in production for efficiency and caching of compressed static assets. • zlib.output_compression compresses PHP's own dynamic output; gzencode() compresses an individual generated payload on demand.
Chunked Transfer Encoding and Streaming Responses
Ordinarily a response includes a Content-Length header stating exactly how many bytes the body contains -- but that's only possible if the server knows the total size before it starts sending. When a response is generated on the fly (a large export, a live feed) and the total size isn't known up front, HTTP/1.1 offers Transfer-Encoding: chunked: the body is sent as a series of chunks, each prefixed by its own length in hexadecimal, ending with a zero-length chunk to signal completion.
Under HTTP/2 and HTTP/3, chunked transfer encoding doesn't apply in the same way, because those protocols already frame the body as a sequence of discrete DATA frames within their own binary/QUIC framing -- the streaming *behavior* PHP cares about (sending output incrementally instead of buffering it all in memory first) is achieved the same way regardless of protocol version: disable output buffering and call flush() after each piece of output you produce.
• Chunked transfer encoding: used when total body size isn't known up front; each chunk is length-prefixed, terminated by a zero-length chunk. • It's an HTTP/1.1-specific mechanism -- HTTP/2/3 stream data via their own native framing instead. • PHP achieves streaming behavior at any protocol version the same way: disable buffering, then flush() after each piece of output. • A reverse proxy (nginx) may buffer PHP's output by default even after PHP flushes -- disable that too (e.g. X-Accel-Buffering: no) or streaming won't reach the client incrementally.
Caching: ETag, Last-Modified, Cache-Control, and Conditional Requests
An ETag is an opaque identifier for a specific version of a resource -- commonly a hash of its content -- that changes whenever the resource does. Last-Modified is the simpler, coarser alternative: a timestamp of when the resource was last changed. Both exist to answer one question cheaply: 'has anything changed since the client last saw this?'
A client that already has a cached copy can ask that question directly with a conditional request: If-None-Match echoes back a previously-received ETag, and If-Modified-Since echoes back a previously-received Last-Modified value. If the server determines nothing has changed, it replies 304 Not Modified with NO body at all -- the client reuses its cached copy, and the network transfer (and, if you implement the check early enough, most of the server-side work of regenerating the page) is skipped entirely.
Cache-Control is a separate, complementary mechanism: instead of asking 'has it changed?' on every request, it tells a cache how long it's allowed to reuse a response WITHOUT asking at all (max-age=3600), whether it may be shared across users (public) or only cached per-user (private), and directives like no-store (never cache this) or must-revalidate (a stale cached copy must be re-validated with the server before reuse, rather than served as-is). Unlike static files served directly by the web server, a dynamic PHP response has to implement all of this comparison logic itself -- there's no automatic mechanism handling it for you.
• ETag: an opaque version identifier (often a content hash) for a resource. • Last-Modified: a coarser timestamp-based alternative to ETag. • If-None-Match / If-Modified-Since: the client's conditional-request headers, compared server-side against the current ETag/Last-Modified. • A match returns 304 Not Modified with NO body -- saving both bandwidth and, if checked early, server work. • Cache-Control governs how long a cache may reuse a response WITHOUT even asking (max-age, public/private, no-store, must-revalidate).
Cookies, CORS, and Security Headers -- Where the Details Live
Cookies (Cookie/Set-Cookie), CORS (Origin, Access-Control-*), and security headers (Content-Security-Policy, Strict-Transport-Security, and friends) are all, mechanically, just more headers riding along inside the message anatomy from the first section -- the 'HTTP Headers' module covers each of them individually in depth, and the 'Advanced Web Security' module covers the security-header threat model. What's worth calling out here, at the protocol-mechanics level, is that CORS in particular is not just a header: for many cross-origin requests the browser first sends an entirely separate 'preflight' request using the OPTIONS method, gets back the allowed methods/headers/origins, and only THEN sends the real request -- a genuine extra round trip baked into the protocol exchange, not merely a header a server sets.
Similarly, a session cookie's interaction with SameSite (covered from the session-security angle in the Sessions module) is really a statement about which of a browser's HTTP requests are even allowed to carry that cookie at all -- another example of a header whose effect is best understood as part of the request/response exchange as a whole, not a value in isolation.
• Cookie/Set-Cookie, CORS headers, and security headers are all part of the same message anatomy covered in section 1. • A cross-origin request often triggers a separate OPTIONS 'preflight' round trip before the browser sends the real request -- CORS has protocol-level cost, not just header-level cost. • For the full header-by-header catalog, see the 'HTTP Headers' module; for the security threat model behind these headers, see 'Advanced Web Security'.
Code Examples
The ETag is derived from data that changes whenever the article does. If the client's If-None-Match header matches the current ETag, the script exits with a bare 304 before ever calling the expensive renderArticleHtml() -- saving both the rendering work and the bandwidth of sending a body the client already has.
if ($clientEtag === $etag) { http_response_code(304); exit; }
Combined with a generator (streamRowsFromDatabase()) that yields one database row at a time instead of loading them all into an array, this pattern lets PHP export an arbitrarily large CSV using near-constant memory, with the client receiving rows incrementally instead of waiting for the whole file to be built server-side first. X-Accel-Buffering: no is nginx-specific -- without it, nginx's own response buffering can silently undo all the flush() calls by holding the whole response until PHP-FPM finishes, defeating the point of streaming.
Server-Sent Events is just a long-lived HTTP response with a special content type and a simple text framing (data: ... followed by a blank line) -- it relies on exactly the streaming/flush mechanics from the previous example, and works equally well delivered over an HTTP/1.1 keep-alive connection or a multiplexed HTTP/2 stream.
Checking Accept-Encoding before compressing avoids sending a gzip-encoded body to a client that can't decode it. The Vary: Accept-Encoding header is essential the moment any cache (CDN, browser cache) sits in front of this response -- without it, a cache could serve a compressed response to a client that never claimed to support gzip.
<?php
$article = loadArticle((int) $_GET['id']);
$etag = '"' . md5($article['updated_at'] . $article['id']) . '"';
header("ETag: {$etag}");
header('Cache-Control: private, max-age=0, must-revalidate');
$clientEtag = $_SERVER['HTTP_IF_NONE_MATCH'] ?? null;
if ($clientEtag === $etag) {
http_response_code(304); // Not Modified -- no body sent at all
exit;
}
// Only reached when the client's cached copy is actually stale --
// this is where the (possibly expensive) page rendering happens.
echo renderArticleHtml($article);<?php
header('Content-Type: text/csv');
header('Content-Disposition: attachment; filename="export.csv"');
header('X-Accel-Buffering: no'); // tell nginx not to buffer this response
// Make sure nothing is buffering output on PHP's side either.
while (ob_get_level() > 0) {
ob_end_flush();
}
$out = fopen('php://output', 'wb');
foreach (streamRowsFromDatabase() as $row) { // a generator -- never loads all rows at once
fputcsv($out, $row);
flush(); // push this chunk to the client now instead of buffering it
}
fclose($out);<?php
header('Content-Type: text/event-stream');
header('Cache-Control: no-cache');
header('X-Accel-Buffering: no');
while (ob_get_level() > 0) {
ob_end_flush();
}
for ($i = 0; $i < 30; $i++) {
$payload = json_encode(['tick' => $i, 'time' => time()]);
echo "data: {$payload}\n\n"; // SSE frame format: `data: ...` + blank line
flush();
if (connection_aborted()) {
break; // stop generating once the client has disconnected
}
sleep(1);
}<?php
$body = generateLargeReport(); // a big string built server-side
$acceptEncoding = $_SERVER['HTTP_ACCEPT_ENCODING'] ?? '';
if (str_contains($acceptEncoding, 'gzip')) {
header('Content-Encoding: gzip');
header('Vary: Accept-Encoding'); // caches must vary on this header
echo gzencode($body, 6);
} else {
echo $body;
}The Request/Response Round Trip
The web server negotiates and terminates the HTTP/1.1, HTTP/2, or HTTP/3 protocol version before PHP ever runs; PHP only ever sees a normalized request and produces a response body/headers. A conditional request can short-circuit with a 304 as early as PHP decides to check -- skipping the expensive part of regenerating the body entirely.
Client Client
| ^
| HTTP Request HTTP Response
| (request line + headers + body) (status line + headers + body)
v |
[ Web Server ] --------------------------------------------> [ Web Server ]
(nginx/Apache: TLS + HTTP/1.1-2-3 termination) (adds/streams response)
| ^
| normalized request (SAPI/FastCGI) generated response body
v |
[ PHP Application ] ------------------------------------------/
($_SERVER, php://input, $_GET/$_POST --> header(), echo/flush())Important Rules
Application code is almost entirely unaware of HTTP/1.1 vs HTTP/2 vs HTTP/3 -- the web server negotiates and terminates the protocol version before PHP ever runs.
Always check If-None-Match/If-Modified-Since as early as possible in a request, before doing any expensive work to build the response body.
For streaming, disable buffering at BOTH the PHP layer (ob_end_flush()) and the proxy layer (e.g. X-Accel-Buffering: no) -- either one alone is not enough.
Set Vary: Accept-Encoding whenever a response's body varies based on compression, so caches don't serve a compressed body to a client that can't decode it.
Where this shows up
CDNs and reverse proxies rely entirely on Cache-Control and ETag/Last-Modified to decide whether they can answer a request without ever contacting the origin PHP server -- getting these headers right is often the single biggest lever for reducing origin load on a high-traffic site. The streaming pattern in this lesson is also exactly how many modern APIs deliver incrementally-generated content (like a chat or AI response arriving token by token) over plain HTTP using Server-Sent Events, rather than requiring a separate WebSocket connection.
Regenerating and sending the full response body on every single request, even for clients whose cached copy is still current.
Compute an ETag or Last-Modified value cheaply, compare it against the client's If-None-Match/If-Modified-Since header as early as possible, and short-circuit with a bare 304 response when nothing has changed.
Skipping the conditional-request check throws away both the bandwidth savings and, if the check happens before the expensive rendering work, the server-side compute savings that HTTP's caching model was specifically designed to give you for free.
Calling flush() in a loop while PHP's default output buffering (or an upstream proxy's response buffering) is still enabled.
Explicitly drain PHP's output buffers (while (ob_get_level() > 0) { ob_end_flush(); }) before streaming, and disable buffering at the web-server/proxy layer too (e.g. X-Accel-Buffering: no on nginx).
flush() only pushes data out of PHP's own buffer -- if output buffering is still active, or a reverse proxy is buffering the response on its own end, the client still receives nothing until the entire script finishes, silently defeating the whole point of streaming.
[multiple-choice] What is the main architectural change HTTP/2 makes compared to HTTP/1.1? A. It removes headers entirely to save bandwidth B. It introduces a binary framing layer that multiplexes many streams over a single TCP connection, instead of relying on multiple parallel connections C. It replaces TCP with UDP D. It requires every request to use the POST method
B. It introduces a binary framing layer that multiplexes many streams over a single TCP connection, instead of relying on multiple parallel connections — HTTP/2's defining feature is multiplexing many independent streams over one TCP connection via a binary framing layer plus HPACK header compression -- HTTP/3 is the one that replaces TCP with QUIC/UDP.
[multiple-choice] Why does HTTP/3 avoid the head-of-line blocking that HTTP/2 can still suffer from? A. HTTP/3 doesn't support multiplexing at all, so there's nothing to block B. HTTP/3 uses QUIC over UDP with independent per-stream sequencing, so a lost packet only stalls its own stream instead of every stream on the connection C. HTTP/3 compresses all data so heavily that packet loss never occurs D. HTTP/3 opens a separate TCP connection per stream
B. HTTP/3 uses QUIC over UDP with independent per-stream sequencing, so a lost packet only stalls its own stream instead of every stream on the connection — HTTP/2 multiplexes streams over one TCP connection, so TCP's strict in-order delivery means one lost packet stalls every stream; QUIC (HTTP/3's transport) sequences each stream independently, so packet loss on one stream doesn't block the others.
[multiple-choice] What does a matching If-None-Match / ETag comparison let a PHP script do? A. Force the client to re-download the resource anyway B. Respond with 304 Not Modified and no body, letting the client reuse its cached copy C. Automatically compress the response body D. Upgrade the connection to HTTP/2
B. Respond with 304 Not Modified and no body, letting the client reuse its cached copy — When the client-supplied If-None-Match matches the resource's current ETag, the server can skip sending a body entirely and reply 304 Not Modified, saving bandwidth and, if checked early, server-side rendering work.
[multiple-choice] Why is chunked transfer encoding used in HTTP/1.1? A. To encrypt the response body B. To send a response whose total size isn't known up front, by breaking the body into length-prefixed chunks terminated by a zero-length chunk C. It's required for every HTTP/1.1 response regardless of content D. To negotiate which compression algorithm to use
B. To send a response whose total size isn't known up front, by breaking the body into length-prefixed chunks terminated by a zero-length chunk — Chunked transfer encoding lets a server start sending a response before it knows the total Content-Length, which is exactly the situation a dynamically generated or streamed response is in.
[multiple-choice] Why might flush() calls inside a PHP loop fail to actually stream data to the browser incrementally? A. flush() is a no-op in modern PHP versions B. PHP's own output buffering or an upstream reverse proxy's response buffering can still hold the data back even after flush() is called, unless both are explicitly disabled C. Streaming is impossible over HTTPS D. flush() only works with the GET method
B. PHP's own output buffering or an upstream reverse proxy's response buffering can still hold the data back even after flush() is called, unless both are explicitly disabled — flush() only pushes data out of PHP's own output buffer; if PHP's buffering isn't fully drained first, or a proxy like nginx is buffering the response on its own end, the client still won't see incremental data until buffering is disabled at every layer involved.
An endpoint GET /report.csv currently builds the entire CSV into a string in memory and echoes it in one shot, and sends no caching headers at all even though the underlying report data only changes once per day. Rewrite it to (1) stream the rows instead of buffering them all in memory, and (2) support conditional requests so a client polling this endpoint gets a 304 on days the report hasn't changed.
Stream rows from a generator with fputcsv()+flush() after disabling output buffering, as in the streaming example. For caching, derive the ETag from the report's last-generated timestamp (e.g. md5($reportGeneratedAt)), send it with header('ETag: ...'), compare it against $_SERVER['HTTP_IF_NONE_MATCH'], and respond 304 with no body when they match -- checked BEFORE starting to stream any rows.
For (1), reach for a generator plus flush() as shown in this lesson's streaming example. For (2), what single piece of data changes exactly when -- and only when -- the report itself changes, that you could hash into an ETag?