Fast Enumeration Internals
Understand what actually happens underneath the for...in syntax -- the NSFastEnumeration protocol, the countByEnumeratingWithState:objects:count: method it requires, the NSFastEnumerationState structure that carries enumeration progress between calls, why mutating a collection mid-enumeration throws an exception, how to write a custom class that supports fast enumeration, and how fast enumeration's performance compares to NSEnumerator and index-based loops.
Every Objective-C developer writes for (id obj in collection) { ... } long before they ever ask what that syntax actually compiles down to. It looks like ordinary C for-loop sugar, but it is not iterating an index at all -- it is repeatedly calling a single method, countByEnumeratingWithState:objects:count:, that any class can implement by conforming to the NSFastEnumeration protocol. This lesson pulls back that syntax to show the real mechanism: the state structure the compiler-generated loop passes back and forth, why the loop can detect and refuse to tolerate a collection changing shape while it runs, how to make a custom collection class participate in for-in the same way NSArray and NSDictionary do, and where fast enumeration actually earns its name performance-wise compared to the older NSEnumerator-based enumeration and to plain objectAtIndex: loops.
What for...in Really Compiles To
The for (id obj in collection) syntax, introduced back in Objective-C 2.0, is not a language primitive that understands NSArray or NSDictionary specially -- it is sugar over a single, protocol-defined method call, repeated until the collection reports it has nothing left to give. The compiler translates the loop into a hidden local NSFastEnumerationState structure, an on-stack buffer of object pointers, and a loop that calls countByEnumeratingWithState:objects:count: over and over, each time asking the collection to fill that buffer with as many objects as it conveniently can and report how many it actually supplied.
Because the mechanism is protocol-based rather than built into the compiler's knowledge of any specific class, any class at all can be iterated with for-in, as long as it conforms to NSFastEnumeration and implements that one required method correctly. This is exactly what makes fast enumeration 'fast': the collection is handed the opportunity to hand back many objects per call instead of the loop asking for exactly one object per iteration, which is what both index-based loops and NSEnumerator-based enumeration effectively do underneath.
It helps to say the desugaring out loud once: for (id obj in collection) { use(obj); } becomes, conceptually, a loop that repeatedly calls countByEnumeratingWithState:objects:count:, receives back a count and a buffer pointer, walks that buffer calling use() on each entry, and stops the moment a call reports a count of zero, meaning the collection has no more objects to give.
• for...in is compiler sugar, not a built-in-to-the-language iteration primitive -- it repeatedly calls one protocol method. • Any class conforming to NSFastEnumeration and implementing countByEnumeratingWithState:objects:count: can be used in a for-in loop. • Each call to that method can hand back a whole batch of objects at once, not just one -- this batching is where the 'fast' in fast enumeration comes from. • The loop stops the moment a call reports a count of zero -- the collection's signal that enumeration is complete.
The NSFastEnumeration Protocol and countByEnumeratingWithState:objects:count:
NSFastEnumeration is a small, single-method protocol: - (NSUInteger)countByEnumeratingWithState:(NSFastEnumerationState *)state objects:(__unsafe_unretained id [])buffer count:(NSUInteger)len;. NSArray, NSDictionary (enumerating its keys), NSSet, and NSString (enumerating composed character sequences in newer runtimes) all conform to it, which is exactly why all of them can appear directly after the in keyword in a for-in loop with no special casing anywhere in your code.
The method's contract is: given a state structure to read and update, and a caller-supplied buffer of capacity len, fill in as many objects as you conveniently can (up to len), update the state structure to reflect your progress, and return how many objects you actually wrote into the buffer. A return value of zero is the universal signal that there is nothing left to enumerate, and the compiler-generated loop treats that return value as its termination condition.
Note the buffer parameter's type: __unsafe_unretained id []. The objects handed back through fast enumeration are not retained on the caller's behalf during the loop -- the collection itself is expected to keep them alive for the duration of enumeration (which is exactly why mutating the collection mid-loop is dangerous, as the next section covers), and it is the caller's responsibility not to hold onto those references past the loop without a proper strong reference of its own.
• NSFastEnumeration has one required method: countByEnumeratingWithState:objects:count:. • NSArray, NSDictionary, NSSet, and other Foundation collections all conform to it -- that conformance is the only reason for-in works on them. • The method returns how many objects it wrote into the caller-supplied buffer for this call; zero means enumeration is finished. • The buffer is __unsafe_unretained -- the collection, not the loop, is responsible for keeping the enumerated objects alive during the loop.
Inside NSFastEnumerationState: The Structure That Carries Progress
NSFastEnumerationState is a plain C struct, not an object, and it is the piece of state that lets countByEnumeratingWithState:objects:count: be called repeatedly and pick up where it left off each time, without the collection needing to expose any of its internal iteration bookkeeping publicly. Three of its fields matter most for understanding the mechanism: unsigned long *mutationsPtr, unsigned long long state, and id *itemsPtr.
itemsPtr is a pointer to the buffer of objects for this call -- notably, the collection is allowed to either fill the caller-supplied buffer parameter and point itemsPtr at that, or, for collections that already store their elements in one contiguous block of memory (a plain C array backing an NSArray, for instance), point itemsPtr directly at that internal storage instead, avoiding a copy entirely. state is a scratch field the collection can use however it likes to remember where it left off between calls -- an index, an internal cursor, whatever the implementation needs.
mutationsPtr is the field responsible for mutation detection, covered in full in the next section: it points at a value that is expected to stay constant for as long as the collection's shape is unchanged, and the compiler-generated loop compares that value across calls to notice if the collection was modified out from under it mid-enumeration.
• NSFastEnumerationState is a plain struct, not an object -- it carries progress between repeated calls without exposing internal bookkeeping. • itemsPtr: a pointer to this call's buffer of objects -- may point at the caller's buffer or, as an optimization, straight at the collection's own internal storage. • state: a free-form scratch field the collection implementation uses to remember its position between calls. • mutationsPtr: a pointer to a value used to detect whether the collection was mutated during enumeration -- the mechanism behind the mutation exception.
Mutation Detection: Why Changing a Collection Mid-Loop Throws
Every mutable Foundation collection tracks a mutation counter internally, incremented every time the collection's shape changes -- an object added, removed, or the collection otherwise structurally modified. When fast enumeration begins, the collection points the state structure's mutationsPtr at that counter (or at a value derived from it), and the compiler-generated for-in loop captures the counter's value at the start and re-checks it on every subsequent call to countByEnumeratingWithState:objects:count:.
If that value has changed between calls -- because something added or removed an object from the collection while the loop was still running -- the generated loop detects the mismatch and throws NSGenericException with the message 'was mutated while being enumerated', immediately halting the loop rather than letting it continue against a collection whose shape no longer matches what it started with. This is a defensive, fail-fast design: continuing to enumerate a collection that changed shape mid-loop could otherwise silently skip objects, revisit objects, or read past the end of storage that has since shrunk or moved.
It's worth being precise about what is and is not detected: the check is a structural mutation check, not a check on the objects themselves. Replacing an element at a fixed index in an NSMutableArray with a different object of the same identity, for instance, does not change the collection's shape and will not trip this check -- only actual insertion, removal, or reordering that shifts the collection's structure will.
• Mutable collections track an internal mutation counter, bumped on every structural change (insert, remove, reorder). • Fast enumeration captures that counter's value at the start and re-checks it on every call to countByEnumeratingWithState:objects:count:. • A mismatch throws NSGenericException with 'was mutated while being enumerated', halting the loop immediately -- a deliberate fail-fast design. • Only structural changes trip the check; replacing an object at a fixed index without changing the collection's size does not.
Writing a Custom Class That Conforms to NSFastEnumeration
Any class can support for-in syntax by declaring conformance to NSFastEnumeration and implementing countByEnumeratingWithState:objects:count: itself. The simplest correct implementation pattern for a class backed by a plain C array or an NSArray internally is to point the state structure's itemsPtr directly at that backing storage on the very first call, set state.state to a nonzero sentinel so the loop knows enumeration has started, report the full count in one call, and return 0 on every call after that to signal completion.
The example below shows this minimal pattern on a small custom collection class, RingBuffer, wrapping a fixed C array of objects. Because the entire contents are already sitting in one contiguous block of memory, the implementation can hand back everything in a single call rather than batching across several calls the way a linked-list-backed collection might need to.
For a collection that cannot expose all of its elements as one contiguous block (a linked list, a tree, anything not stored contiguously), the caller-supplied buffer parameter is what makes fast enumeration still work efficiently: each call copies as many elements as fit into that buffer (its capacity is the len parameter), and state.state is used to remember where the next call should resume from -- exactly the pattern that lets an arbitrarily large or oddly-structured collection still participate in batched fast enumeration.
• Declare conformance with <NSFastEnumeration> and implement countByEnumeratingWithState:objects:count: as the only required method. • A contiguous, array-backed collection can point itemsPtr straight at its own storage and hand back everything in one call. • A non-contiguous collection (linked list, tree) fills the caller-supplied buffer in batches across multiple calls, using state.state to track its resume position. • mutationsPtr must point at something that stays constant while unmutated -- a static or otherwise stable value is a common minimal choice for a simple, read-only custom collection.
Performance: Fast Enumeration vs. NSEnumerator vs. objectAtIndex: Loops
Before fast enumeration existed, Objective-C code iterated collections with either an explicit index loop calling objectAtIndex: repeatedly, or with an NSEnumerator obtained from objectEnumerator, calling nextObject in a while loop until it returned nil. Both approaches share the same underlying cost profile: each element requires a full, separate message send -- objectAtIndex: or nextObject -- with all the message-dispatch overhead that entails, once per element, with no batching at all.
Fast enumeration's entire performance advantage comes from batching: a single call to countByEnumeratingWithState:objects:count: can hand back many objects at once (bounded by the buffer size the loop provides, commonly on the order of tens of objects per call), which amortizes the message-send overhead across an entire batch instead of paying it per element. For large collections this adds up to a real, measurable difference, which is part of why Apple's own documentation and most style guides recommend for-in over both objectAtIndex: loops and NSEnumerator wherever the loop body does not need to mutate the collection.
NSEnumerator retains one advantage fast enumeration deliberately gives up: because it does not use the batched, buffer-based protocol, it does not need to defend against the same class of shape-changing mutation the same way, and there are NSEnumerator-based patterns that tolerate certain limited mutations that would trip fast enumeration's exception. That said, NSEnumerator's own documentation explicitly does not guarantee mutation safety either -- it is simply a different mechanism with different, and largely still unsafe, mutation behavior, not a sanctioned way to mutate mid-enumeration.
• objectAtIndex: loops and NSEnumerator's nextObject both issue one full message send per element, with no batching. • countByEnumeratingWithState:objects:count: hands back a whole batch of objects per call, amortizing message-send overhead across the batch. • For large collections this batching gives for-in a real, measurable speed advantage over both older iteration styles. • NSEnumerator behaves somewhat differently under mutation than fast enumeration does, but it is not a safe or sanctioned way to mutate mid-enumeration either.
Code Examples
This is the everyday syntax every Objective-C developer writes, and it looks like a simple loop -- but nothing here iterates an index. Underneath, the compiler generates a hidden NSFastEnumerationState, an on-stack buffer, and a loop that repeatedly calls [names countByEnumeratingWithState:objects:count:] until that call reports zero objects remaining.
for (NSString *name in names) { Removing an object from queue changes its internal mutation counter. The next call to countByEnumeratingWithState:objects:count: re-checks that counter against the value captured when the loop began, finds a mismatch, and the compiler-generated loop throws NSGenericException immediately rather than continuing over a collection whose shape no longer matches what enumeration started with.
[queue removeObject:item]; // mutates queue's shape mid-enumeration
Common mistake: Removing, inserting, or otherwise structurally modifying a mutable collection from inside a for-in loop iterating that same collection.
The first fix enumerates [queue copy], an immutable snapshot whose shape cannot change no matter what happens to the original queue during the loop, so removeObject: on queue never trips the mutation check. The second fix avoids mutating during enumeration entirely by recording which indexes need removing and applying that change only after the enumeration has fully finished -- a common pattern when the loop body needs to decide on several removals at once.
for (NSString *item in [queue copy]) { RingBuffer implements the one required NSFastEnumeration method: on the first call it fills itemsPtr with the entire backing storage in one shot, sets a stable mutationsPtr (safe here because RingBuffer is read-only and its shape genuinely cannot change), marks state.state as 'started', and reports the full count; every call after that returns 0, telling the for-in loop enumeration is complete. Because everything is handed back in a single call, this pattern is only correct for a collection small enough, or already contiguous enough, to expose all at once.
state->mutationsPtr = (unsigned long *)&_storage; // stable: read-only class
NSArray<NSString *> *names = @[@"Ada", @"Grace", @"Alan"];
for (NSString *name in names) {
NSLog(@"Hello, %@", name);
}NSMutableArray<NSString *> *queue = [@[@"a", @"b", @"c"] mutableCopy];
for (NSString *item in queue) {
NSLog(@"Processing %@", item);
if ([item isEqualToString:@"b"]) {
[queue removeObject:item]; // mutates queue's shape mid-enumeration
}
}
// Crashes with:
// *** Terminating app due to uncaught exception 'NSGenericException',
// reason: '*** Collection <__NSArrayM: 0x...> was mutated while being
// enumerated.'NSMutableArray<NSString *> *queue = [@[@"a", @"b", @"c"] mutableCopy];
// Fix 1: iterate an immutable snapshot, mutate the original safely
for (NSString *item in [queue copy]) {
NSLog(@"Processing %@", item);
if ([item isEqualToString:@"b"]) {
[queue removeObject:item];
}
}
// Fix 2: collect indexes to remove, apply the mutation after the loop
NSMutableIndexSet *toRemove = [NSMutableIndexSet indexSet];
[queue enumerateObjectsUsingBlock:^(NSString *item, NSUInteger idx, BOOL *stop) {
if ([item isEqualToString:@"c"]) {
[toRemove addIndex:idx];
}
}];
[queue removeObjectsAtIndexes:toRemove];@interface RingBuffer : NSObject <NSFastEnumeration>
- (instancetype)initWithObjects:(NSArray *)objects;
@end
@implementation RingBuffer {
NSArray *_storage;
}
- (instancetype)initWithObjects:(NSArray *)objects {
if (self = [super init]) {
_storage = [objects copy];
}
return self;
}
- (NSUInteger)countByEnumeratingWithState:(NSFastEnumerationState *)state
objects:(__unsafe_unretained id [])buffer
count:(NSUInteger)len {
// state.state == 0 means this is the very first call for this loop.
if (state->state == 0) {
state->itemsPtr = (__unsafe_unretained id *)_storage.count
? malloc(sizeof(id) * _storage.count) : NULL;
for (NSUInteger i = 0; i < _storage.count; i++) {
((__unsafe_unretained id *)state->itemsPtr)[i] = _storage[i];
}
state->mutationsPtr = (unsigned long *)&_storage; // stable: read-only class
state->state = 1; // mark "started"
return _storage.count;
}
// Every call after the first: nothing left to give.
return 0;
}
@end
RingBuffer *ring = [[RingBuff
…How for...in Pulls Objects Out of a Collection
The collection conforms to NSFastEnumeration; each call to countByEnumeratingWithState:objects:count: updates the enumeration state and fills the objects buffer; the compiler-generated for-in loop walks that buffer and repeats until a call reports zero objects.
Collection
↓
NSFastEnumeration
↓
Enumeration State
↓
Objects Buffer
↓
for-in LoopWhere This Shows Up in Production Code
Fast enumeration internals matter most in two everyday situations: debugging a mysterious 'was mutated while being enumerated' crash in a data-processing loop that grew a removeObject: or addObject: call somewhere in its body over time, and building a custom lightweight collection type (a fixed-capacity buffer, a lazily-computed sequence, a wrapper over a C array from a lower-level library) that should feel native to Objective-C code by supporting for-in like NSArray and NSDictionary do, rather than forcing callers to use a bespoke accessor method.
for (NSString *item in mutableArray) { if ([item hasPrefix:@"x"]) { [mutableArray removeObject:item]; } } for (NSString *item in [mutableArray copy]) { if ([item hasPrefix:@"x"]) { [mutableArray removeObject:item]; } }
Removing from mutableArray while a for-in loop is enumerating it changes the collection's internal mutation counter, which fast enumeration checks on every batch call -- the mismatch throws NSGenericException immediately. Enumerating an immutable copy instead leaves the snapshot being iterated untouched no matter what happens to the original.
// Custom collection: forgetting to set mutationsPtr at all state->itemsPtr = _backingArray; state->state = 1; return count;
state->itemsPtr = _backingArray; state->mutationsPtr = &_mutationStamp; // some stable or tracked value state->state = 1; return count;
mutationsPtr must always be set to point at something meaningful, even for a read-only or otherwise stable class -- leaving it uninitialized means the compiler-generated loop reads garbage stack memory when checking for mutation, which can produce a spurious crash or, worse, silently fail to detect a real mutation.
What does the compiler actually generate when it sees a for (id obj in collection) loop? A. Nothing special -- it is a built-in language primitive with no method calls involved B. A single call to objectAtIndex: for the whole collection C. Repeated calls to countByEnumeratingWithState:objects:count:, walking a buffer of objects each call returns, until a call reports zero remaining D. A call to NSEnumerator's nextObject method in a while loop
C. Repeated calls to countByEnumeratingWithState:objects:count:, walking a buffer of objects each call returns, until a call reports zero remaining — for-in is sugar over the NSFastEnumeration protocol: the compiler generates a loop that repeatedly calls countByEnumeratingWithState:objects:count:, each call filling a buffer with a batch of objects, stopping only when a call reports a count of zero.
Which field of NSFastEnumerationState is responsible for detecting that a collection was mutated during enumeration? A. itemsPtr B. mutationsPtr C. state D. There is no such field; mutation detection happens entirely inside NSArray, invisibly to the state structure
B. mutationsPtr — mutationsPtr points at a value expected to remain constant while the collection is unmutated -- typically tied to an internal mutation counter. The compiler-generated loop compares that value across calls and throws if it changes.
What exception is thrown, and with what characteristic message, when a mutable collection is structurally changed while being fast-enumerated? A. NSInvalidArgumentException, 'nil argument' B. NSRangeException, 'index out of bounds' C. NSGenericException, 'was mutated while being enumerated' D. No exception is thrown -- the loop silently skips the changed elements
C. NSGenericException, 'was mutated while being enumerated' — The compiler-generated fast-enumeration loop throws NSGenericException with the message '*** Collection ... was mutated while being enumerated' the moment it detects the mutation counter changed between calls.
A method needs to remove several items from an NSMutableArray based on a condition evaluated while iterating it. What is the safest fix that avoids the mutation exception? A. Call removeObject: directly inside the for-in loop as items are found B. Iterate an immutable copy of the array (or collect the indexes/objects to remove and apply the removal after the loop finishes) C. Switch the property from strong to weak so mutation is no longer tracked D. Wrap the removeObject: call in a @try/@catch block inside the loop and ignore the exception
B. Iterate an immutable copy of the array (or collect the indexes/objects to remove and apply the removal after the loop finishes) — Enumerating an immutable snapshot (via [array copy]) means the object actually being enumerated can never change shape, no matter what happens to the original array; alternatively, deferring the actual mutation until after the loop finishes avoids the conflict entirely.
Why is fast enumeration generally faster than an objectAtIndex: loop or NSEnumerator-based enumeration for large collections? A. It skips ARC's retain/release entirely, which the other two approaches always perform B. It runs on a background thread automatically C. countByEnumeratingWithState:objects:count: can hand back a whole batch of objects per call, amortizing message-send overhead across the batch, instead of one message send per element D. It does not actually visit every element -- it samples a subset for performance
C. countByEnumeratingWithState:objects:count: can hand back a whole batch of objects per call, amortizing message-send overhead across the batch, instead of one message send per element — objectAtIndex: loops and NSEnumerator's nextObject each require one full message send per element with no batching. Fast enumeration's protocol method can fill a buffer with many objects in a single call, spreading the fixed cost of a message send across an entire batch.
What is a correct minimal implementation strategy for countByEnumeratingWithState:objects:count: on a custom class backed by one small, already-contiguous array? A. Always return 0 immediately so the for-in loop never actually runs B. On the first call, fill in itemsPtr, a stable mutationsPtr, and the full count, mark state as started, and return the count; on every call after that, return 0 C. Ignore the state parameter entirely and just return the array's count on every call D. Throw an exception, since custom classes cannot support fast enumeration without subclassing NSArray
B. On the first call, fill in itemsPtr, a stable mutationsPtr, and the full count, mark state as started, and return the count; on every call after that, return 0 — For a small, already-contiguous backing store, the simplest correct pattern hands back everything in the first call (setting itemsPtr, a stable mutationsPtr, and the count) and signals completion by returning 0 on every subsequent call, since the loop keeps calling until it sees a zero count.
Take the RingBuffer class from the examples above and modify countByEnumeratingWithState:objects:count: to fill the caller-supplied buffer in batches of at most len items per call, using state.state to track how many items have already been handed back, instead of returning the entire contents in one call. Explain in a comment why this batched approach scales better for a very large backing collection than the single-call version.
• (NSUInteger)countByEnumeratingWithState:(NSFastEnumerationState *)state objects:(__unsafe_unretained id [])buffer count:(NSUInteger)len {\n NSUInteger startIndex = (NSUInteger)state->state;\n if (startIndex >= _storage.count) {\n return 0; // nothing left\n }\n NSUInteger batchCount = MIN(len, _storage.count - startIndex);\n for (NSUInteger i = 0; i < batchCount; i++) {\n buffer[i] = _storage[startIndex + i];\n }\n state->itemsPtr = buffer;\n state->mutationsPtr = (unsigned long *)&_storage;\n state->state = startIndex + batchCount;\n return batchCount;\n}\n// Batching avoids allocating one large block up front and copying the\n// entire collection before the loop even starts -- for a very large\n// backing collection, filling only 'len' items per call keeps memory\n// use bounded and lets enumeration begin producing results immediately.
Use state.state to store the starting index for this call. Each call should copy min(len, remainingCount) items into the buffer parameter (not into itemsPtr's own malloc'd block this time -- point itemsPtr at the buffer parameter instead), advance state.state by however many were copied, and return that count. Return 0 once state.state reaches _storage.count.