PHP Runtime and the Zend Engine
Understand what actually happens inside PHP between typing php script.php and seeing output: the Zend Engine's pipeline from lexing and parsing through AST construction, opcode generation, and execution on the Zend VM, plus the core data structures (zval, symbol tables, function/class tables) and where extensions, the garbage collector, OPcache, and JIT fit into that pipeline.
PHP the language and the Zend Engine that runs it are two different things, the same way JavaScript the language is different from V8 or SpiderMonkey. Every PHP script you've ever run -- however you ran it, CLI, PHP-FPM behind Nginx, an embedded SAPI -- went through the exact same internal pipeline inside the Zend Engine: your source text got tokenized, parsed into a tree, compiled into a low-level instruction set, and finally executed by a virtual machine that walks those instructions one at a time. Most PHP development never requires knowing any of this. But once you start caring about performance (why OPcache matters, what JIT actually speeds up), memory behavior (why copy-on-write and reference counting work the way they do), or genuinely confusing bugs (why a reference behaves differently from a normal assignment), this pipeline is the ground truth everything else is built on. This lesson walks the pipeline end to end, at the level of a working developer who wants a real mental model -- not a C programmer reading the interpreter's source code.
What the Zend Engine Actually Is
'PHP' as most developers experience it is really two layered things: the Zend Engine, a C library that implements PHP's core language semantics (variables, types, control flow, functions, classes, the whole execution model), and a SAPI (Server API) -- CLI, php-fpm/FastCGI, embed, etc. -- that adapts the engine to a particular environment (a command-line invocation, an HTTP worker process, being embedded in another program). The engine itself doesn't know or care whether its output is going to a terminal or an HTTP response; that's the SAPI's job.
Built-in functionality you use constantly -- string functions, array functions, PDO, curl, mbstring, json_encode -- mostly lives in extensions: separate C modules that register their functions and classes into the engine's global function table and class table at startup, using APIs the engine exposes for exactly that purpose (zend_register_... family of calls internally). This is why some functions exist only when a particular extension is compiled in and enabled -- curl_init() is unavailable, not merely broken, on a PHP build without ext-curl.
The engine's job, end to end, is turning PHP source code into something a CPU-adjacent execution loop can actually run, then running it -- and every stage of that process (parsing, compiling, executing) is itself just more C code inside the engine, callable and inspectable to varying degrees from userland PHP via things like the tokenizer functions, Reflection, and OPcache's own introspection functions, several of which appear in the examples below.
• Zend Engine: the C core implementing PHP's language semantics (types, control flow, OOP, execution). • SAPI (CLI, FPM, embed, ...): adapts the engine to a specific runtime environment; the engine itself is SAPI-agnostic. • Extensions (ext-mbstring, ext-pdo, ext-curl, ...): C modules that register functions/classes into the engine at startup. • A missing function usually means a missing/disabled extension, not a broken PHP build.
From Source Text to an AST: Lexing and Parsing
The first real stage is lexical analysis: a lexer (generated from a re2c grammar) scans your raw source text character by character and groups it into tokens -- T_VARIABLE for $name, T_STRING for a bareword identifier, T_LNUMBER/T_DNUMBER for numeric literals, T_FUNCTION, T_ECHO, and so on for keywords, plus punctuation and operators. This is purely lexical: the lexer has no idea yet whether $x = $y + 1; is even valid PHP, only that it recognizes a sequence of individually meaningful chunks.
The parser (generated from a bison/yacc-style grammar) then consumes that token stream and builds an Abstract Syntax Tree (AST) -- a tree structure representing the source's actual grammatical shape: an assignment node whose left side is a variable node and whose right side is a binary-addition node, itself containing a variable node and a literal node. This is the stage where genuine syntax errors surface -- an unmatched brace, a statement in the wrong place -- because the parser is checking the token sequence against PHP's actual grammar rules, not just recognizing individual tokens.
You can observe the lexer's output directly from userland PHP with the built-in token_get_all() function, which is a great way to build intuition for what 'tokenizing' concretely means before the parser ever gets involved -- the first example below does exactly this.
• Lexer: raw source text -> a flat stream of tokens (T_VARIABLE, T_STRING, T_LNUMBER, keywords, punctuation). • Parser: token stream -> Abstract Syntax Tree (AST), a tree reflecting the code's actual grammatical structure. • Genuine syntax errors ('unexpected token') are caught by the parser checking tokens against PHP's grammar. • token_get_all() lets you inspect the lexer's token stream directly from PHP code.
Compilation: AST into Zend Opcodes
Once an AST exists, the compiler walks it and emits Zend opcodes -- a low-level, engine-specific instruction set that is far simpler than the AST: things like ZEND_ASSIGN, ZEND_ADD, ZEND_ECHO, ZEND_INIT_FCALL, ZEND_DO_FCALL, ZEND_JMPZ (conditional jump), ZEND_RETURN. Each opcode is a zend_op structure carrying an opcode number plus up to two operand references (a variable slot, a constant, a temporary value, or another opcode's result). This is conceptually the same idea as JVM bytecode or CPython bytecode: a small, flat instruction set that's much cheaper to execute than re-interpreting a tree shape on every step.
The full set of opcodes generated for one function, method, or top-level file body is stored as a zend_op_array -- essentially 'the compiled form of this one unit of code'. This is precisely what OPcache caches in shared memory: not source text, not the AST, but these already-compiled zend_op_array structures, so a cache hit skips lexing, parsing, *and* compiling entirely and jumps straight to execution.
Opcode generation is where a handful of compile-time checks and optimizations also happen -- constant folding (computing 2 + 2 once, at compile time, instead of on every execution), and, as covered in the previous lesson, checks like #[\Override]'s parent-method validation. None of this requires running the code; it's all analysis and transformation of the AST/opcode representation itself.
• Compiler walks the AST and emits Zend opcodes -- a flat, low-level instruction set (ZEND_ADD, ZEND_ASSIGN, ZEND_DO_FCALL, ...). • One function/method/file body compiles to a zend_op_array -- the compiled unit OPcache actually caches. • OPcache caching skips lexing + parsing + compiling entirely on a hit -- not just parsing. • Some compile-time checks/optimizations (constant folding, #[\Override] validation) happen at this stage, before any execution.
The Executor: the Zend Virtual Machine
Execution is the job of the Zend VM: a loop that fetches one opcode at a time from a zend_op_array and dispatches it to the specific handler function written to implement that opcode (ZEND_ADD's handler adds two operands and stores the result, ZEND_DO_FCALL's handler sets up a new call frame and jumps execution into the called function's own opcodes, and so on). Without JIT enabled, this fetch-dispatch-execute loop is what 'running PHP' concretely means at the lowest level -- one opcode, one handler call, repeat, for every single opcode in the program's actual execution path.
Each function/method call gets its own execution context, a zend_execute_data frame, holding that call's local variable slots, its position in the opcode array (which opcode runs next), and a pointer back to the calling frame -- these frames chain together to form the call stack you'd see in a stack trace. A deeply recursive PHP function is, physically, a long chain of these frames, which is exactly why uncontrolled recursion can exhaust the (default, comparatively small) C stack the engine runs on and crash the process, distinct from PHP's own configurable memory_limit.
This is also the stage JIT (covered in depth in the next lesson) plugs into: instead of interpreting opcodes one by one through their C handler functions every single time, PHP's JIT compiles selected opcode sequences directly into native machine code once, so that hot code can skip the interpretation loop on subsequent runs -- a genuinely different execution strategy for the same compiled opcodes, not a different compilation stage.
• Zend VM: fetch-dispatch-execute loop over a zend_op_array's opcodes, one handler call per opcode. • Each call gets a zend_execute_data frame (locals + position + caller link) -- frames chain into the call stack. • Deep recursion consumes real C stack space, a separate concern from PHP's configurable memory_limit. • JIT changes HOW compiled opcodes get executed (interpreted vs native machine code) -- not whether/how they're compiled.
Core Data Structures: zval, Symbol Tables, Function and Class Tables
Every PHP value the engine ever touches -- an int, a string, an array, an object reference, null -- is represented internally as a zval ('Zend value'): a small structure carrying a type tag plus either the value directly (for small scalars like ints/bools/doubles) or a pointer to a separately allocated, reference-counted structure (for strings, arrays, objects). This one uniform container is what lets the engine's opcodes and functions operate generically on 'a PHP value' without needing separate code paths per type at the storage level. The next lesson goes deep on zval and reference counting specifically.
A symbol table is, physically, just a hash table mapping variable names (as strings, minus the $) to their zvals within one scope -- global scope has one, and each function call frame has its own local one. This is also why PHP's $$variable variable-variables and extract()/compact() work at all: they're directly reading and writing entries in this same hash table by name, the identical mechanism the engine itself uses for ordinary $x = ....
Function tables and class tables are separate, engine-wide hash tables mapping a function or class *name* to its compiled representation (zend_function / zend_class_entry) -- populated once, as functions/classes are declared (by your script being compiled) or as extensions register their own C-implemented functions/classes at startup. Calling a function by name is, at the lowest level, a lookup in this table followed by a jump into that function's own opcode array -- which is why get_defined_functions() and class_exists() can answer instantly: they're just reading these already-built tables, not re-scanning source code.
• zval: the universal container for any PHP value -- type tag + inline scalar value or pointer to refcounted data. • Symbol table: a per-scope hash table mapping variable names to zvals -- the same mechanism $$var and extract() use. • Function table / class table: engine-wide hash tables mapping name -> compiled zend_function / zend_class_entry. • Calling a function/instantiating a class by name is a lookup in these tables, populated once at declaration/registration time.
Where Extensions, GC, and OPcache/JIT Fit
Extensions hook into every stage described above: they register C-implemented functions and classes directly into the function/class tables (skipping the lex/parse/compile stages entirely, since there's no PHP source for them), can add their own opcode handlers or override existing ones in advanced cases, and can participate in the engine's lifecycle hooks (module init, request init/shutdown). This is how something like OPcache itself is implemented -- OPcache is a Zend extension that hooks into the compilation stage to check its shared-memory cache before compiling a file, and into it afterward to store what it just compiled.
Garbage collection sits beside the core execution loop rather than inside it: PHP's baseline memory management is reference counting (each refcounted zval structure tracks how many things point at it, and hits zero -> freed immediately), but reference counting alone can never free a cycle of objects that reference each other. PHP's GC is a separate cycle-collector that periodically scans candidate structures to find and free such cycles -- this is covered in full mechanical detail, with real code, in the next lesson.
OPcache and JIT are the two big production-performance levers built on everything above: OPcache caches compiled zend_op_array structures in shared memory across requests/workers so the lex-parse-compile stages are skipped entirely on a cache hit; JIT (built on top of OPcache's infrastructure) goes one step further and compiles selected opcode sequences into native machine code so the VM's fetch-dispatch loop can be bypassed for that code too. Both get a full, dedicated lesson next -- this section is the map showing exactly where each one sits in the pipeline you just walked through.
• Extensions register into function/class tables directly (no PHP source to lex/parse/compile for their own code). • OPcache is itself implemented as a Zend extension hooking into the compilation stage. • Refcounting handles ordinary memory reclamation immediately; a separate GC cycle-collector is needed for reference cycles. • OPcache skips lex/parse/compile on a cache hit; JIT additionally skips opcode interpretation for compiled-to-native code.
The Pipeline, Named Stage by Stage
Every one of these stages runs for every PHP file, every time, with or without OPcache -- OPcache just skips stages 1-3 on a cache hit. Constant folding (2 + 2 -> 4) is a compiler-stage optimization, not something the VM decides at runtime. JIT changes stage 4's execution strategy for hot code; it does not remove stages 1-3.
// One line of PHP, and the stages it passes through before it produces output:
//
// $total = 2 + 2;
//
// 1. Lexer: T_VARIABLE($total) T_EQUAL T_LNUMBER(2) T_PLUS T_LNUMBER(2) T_SEMICOLON
// 2. Parser: AssignNode(target: VarNode($total), value: BinaryOpNode('+', 2, 2))
// 3. Compiler: ZEND_ADD T2 = 2, 2 (often constant-folded to just 4 at compile time)
// ZEND_ASSIGN $total, T2
// 4. Zend VM: fetches ZEND_ADD, runs its handler -> fetches ZEND_ASSIGN, runs its handler
// (or: JIT-compiled native code runs directly, skipping the fetch/dispatch loop)
echo $total ?? 'not computed yet', PHP_EOL;Code Examples
token_get_all() runs the exact same lexical analysis the engine performs before parsing, and hands the token stream back to userland PHP. Each array entry is [token_id, text, line]; a plain string entry is a single-character punctuation token (like ';' or '+') that doesn't need its own named constant. Expected result: T_OPEN_TAG <?php T_VARIABLE $total PUNCTUATION = T_LNUMBER 2 PUNCTUATION + T_LNUMBER 2 PUNCTUATION ; T_WHITESPACE T_ECHO echo T_VARIABLE $total PUNCTUATION ;
$tokens = token_get_all($source);
function_exists() and class_exists() answer instantly because 'greet' and 'ExampleClass' are already sitting in the engine's global function table and class table by the time this code runs -- populated when this file was compiled, before a single line of it executed. Expected result: User-defined functions include 'greet': yes ExampleClass is registered: yes Total user-defined functions in this script: 1 $isDefined = function_exists('greet');
opcache_get_status() is a direct window into the engine's own compiled-code cache: when enabled, a rising 'hits' count across requests confirms that this file's lex/parse/compile stages are genuinely being skipped in favor of a cached zend_op_array, exactly as described in the compilation section above. Expected result: OPcache enabled: yes | cached scripts: 14 | hits: 5312 | misses: 14 The next lesson covers OPcache's configuration and behavior (invalidation, preloading, production settings) in full depth -- this example is only about confirming the engine's own compiled-code cache is active.
$status = opcache_get_status(fetch_scripts: false);
<?php
declare(strict_types=1);
$source = <<<'PHP'
<?php
$total = 2 + 2;
echo $total;
PHP;
$tokens = token_get_all($source);
foreach ($tokens as $token) {
if (is_array($token)) {
[$id, $text] = $token;
printf("%-20s %s\n", token_name($id), trim($text));
} else {
printf("%-20s %s\n", 'PUNCTUATION', $token);
}
}<?php
declare(strict_types=1);
function greet(string $name): string
{
return "Hello, {$name}!";
}
class ExampleClass {}
// These calls are lookups into the engine's already-built function/class tables --
// they do NOT re-scan source code, because the tables were populated at compile time.
$userFunctions = get_defined_functions()['user'];
$isDefined = function_exists('greet');
$isClassDefined = class_exists(ExampleClass::class);
printf("User-defined functions include 'greet': %s\n", $isDefined ? 'yes' : 'no');
printf("ExampleClass is registered: %s\n", $isClassDefined ? 'yes' : 'no');
printf("Total user-defined functions in this script: %d\n", count($userFunctions));<?php
declare(strict_types=1);
if (!function_exists('opcache_get_status')) {
echo "OPcache extension is not loaded.\n";
exit;
}
$status = opcache_get_status(fetch_scripts: false);
if ($status === false) {
echo "OPcache is loaded but disabled (opcache.enable=0).\n";
exit;
}
printf(
"OPcache enabled: %s | cached scripts: %d | hits: %d | misses: %d\n",
$status['opcache_enabled'] ? 'yes' : 'no',
$status['opcache_statistics']['num_cached_scripts'],
$status['opcache_statistics']['hits'],
$status['opcache_statistics']['misses'],
);PHP Source to Execution
Every PHP request runs this whole pipeline -- OPcache lets a request skip straight from 'Opcode' to 'Zend VM' on a cache hit, and JIT lets hot opcodes skip straight to native execution instead of the interpreter loop.
PHP Source
|
v
Lexer/Parser --tokens--> (grammar check)
|
v
AST (tree: reflects source's grammatical structure)
|
v
Opcode (zend_op_array: ZEND_ADD, ZEND_ASSIGN, ZEND_DO_FCALL, ...)
|
v
Zend VM (fetch -> dispatch -> execute, one opcode at a time)
|
v
Execution (output, side effects, return values)Important Rules
The engine always lexes -> parses -> compiles -> executes; there is no PHP execution path that skips straight from source text to running code.
OPcache caches compiled zend_op_array structures, not source text or the AST -- a hit skips lexing, parsing, AND compiling.
Function/class name lookups are hash table reads against tables built once at compile/registration time, not re-scans of source.
JIT changes how already-compiled opcodes get executed (native code vs interpreted) -- it is not a separate compilation stage.
Extensions register functions/classes directly into the engine's tables, bypassing the lex/parse/compile stages entirely for their own C code.
Why This Matters in Production
Every performance conversation about PHP -- 'should we enable OPcache', 'is JIT worth turning on for our app', 'why does this deploy need a cache reset' -- only makes sense once you know what's actually being cached (compiled opcodes, not source) and what actually executes (the Zend VM walking those opcodes, one handler call at a time, unless JIT has taken over that specific code path). Function/class table lookups being simple hash reads is also why PHP's autoloading and class-existence checks are cheap even in large codebases with thousands of classes -- the expensive part (locating and compiling the file) only ever happens once per class, the first time it's referenced.
"PHP is a purely interpreted language with no compilation step -- that's what makes it different from Java."
PHP always compiles source into Zend opcodes before executing anything; without OPcache, that compilation just happens fresh on every single request instead of being cached and reused.
PHP has always had a real compile step (AST -> opcodes) on every execution, with or without OPcache -- OPcache doesn't add compilation, it caches the result of a compilation step that was already happening, so it can be skipped on subsequent requests.
Assuming JIT compiles PHP source code directly into machine code, replacing the lexer/parser/compiler entirely.
JIT compiles already-generated Zend opcodes (the same opcodes OPcache caches) into native machine code -- it operates on the output of the existing compile stage, not on raw source text.
Lexing, parsing, and opcode generation still happen exactly as described in this lesson even with JIT enabled; JIT only changes what happens to the resulting opcodes during execution (interpreted one-by-one vs run as compiled native code).
[multiple-choice] What does PHP's parser produce from the lexer's token stream? A. Native machine code B. An Abstract Syntax Tree (AST) reflecting the source's grammatical structure C. Zend opcodes ready for the VM to execute D. A compiled shared library
B. An Abstract Syntax Tree (AST) reflecting the source's grammatical structure — The parser builds an AST from the token stream, checking it against PHP's grammar (which is where genuine syntax errors surface). The compiler stage, which comes after, is what turns the AST into Zend opcodes.
[multiple-choice] What exactly does OPcache cache in shared memory? A. The raw PHP source text of each file B. The Abstract Syntax Tree produced by the parser C. Compiled zend_op_array structures (opcodes) -- skipping lexing, parsing, and compiling entirely on a hit D. Only the results (return values) of previously executed functions
C. Compiled zend_op_array structures (opcodes) -- skipping lexing, parsing, and compiling entirely on a hit — OPcache stores the already-compiled zend_op_array for each file. A cache hit means a request goes straight from 'found in cache' to execution on the Zend VM, skipping the lex/parse/compile stages entirely.
[multiple-choice] Why can function_exists('some_function') answer instantly without scanning any source code? A. PHP caches the answer the first time you call it, for that function name only B. It's a lookup in the engine's global function table, a hash table populated once at compile/registration time C. PHP re-parses the whole codebase in the background on startup and keeps the result in a file D. function_exists() only works for built-in functions, not user-defined ones
B. It's a lookup in the engine's global function table, a hash table populated once at compile/registration time — Every declared function is registered into the engine-wide function table at compile time (or, for extensions, at startup). function_exists() is simply a hash table lookup against that already-built table, not a re-scan of any source.
Using token_get_all(), write a small script that counts how many T_VARIABLE tokens appear in a given snippet of PHP source, without using any regular expressions on the raw source text.
$tokens = token_get_all($source); $count = 0; foreach ($tokens as $t) { if (is_array($t) && $t[0] === T_VARIABLE) { $count++; } } echo $count;
token_get_all() returns each variable token as an array whose first element is the constant T_VARIABLE -- filter the token list for entries matching that.