Advanced Memory Topics
Beyond new and delete lies the machinery that actually determines a program's real-world performance: how the stack and heap differ, how alignment and padding shape an object's size, and how allocators let you control memory strategy directly.
Two programs that are logically identical -- same algorithm, same asymptotic complexity -- can differ by an order of magnitude in real-world speed purely because of how they use memory. This page goes under the hood of C++'s memory model: the structural difference between stack and heap allocation, why the compiler inserts padding bytes you never asked for, what endianness means for how multi-byte values are actually stored, why cache locality and false sharing can dominate performance more than algorithmic complexity does at small scales, how fragmentation degrades a long-running heap, and the allocator machinery -- including std::pmr -- that lets you take direct control of allocation strategy instead of always deferring to the default global new/delete.
Stack vs Heap
The stack is a fixed-size region (typically a few megabytes, configurable but limited) that grows and shrinks automatically as functions are called and return -- allocating a local variable is just moving a pointer, and freeing it happens automatically for free when the function returns. The heap (free store) is a much larger pool managed by an allocator; obtaining memory from it (new) and returning it (delete) both involve bookkeeping inside the allocator to find and track free blocks, making heap operations meaningfully slower than stack allocation.
local_array's storage is part of stack_example()'s stack frame -- it exists from function entry to function exit with essentially zero allocation cost. heap_array's storage comes from the heap via new[], which must search for a suitable free block and record that it's now in use; delete[] must reverse that bookkeeping, and forgetting it leaks the memory for the rest of the program's run.
#include <iostream>
void stack_example()
{
int local_array[1000]; // stack: allocated/freed automatically, extremely fast
local_array[0] = 1;
} // local_array's memory is reclaimed here, no explicit action needed
void heap_example()
{
int *heap_array = new int[1000]; // heap: allocator does bookkeeping to find 1000 ints of space
heap_array[0] = 1;
delete[] heap_array; // must be freed explicitly, or it leaks
}
int main()
{
stack_example();
heap_example();
}Alignment, Padding, and Object Representation
Every type has an alignment requirement -- the address of any object of that type must be a multiple of its alignment (commonly 4 for int, 8 for double or pointers on a 64-bit system). To satisfy every member's alignment inside a struct, the compiler often inserts invisible padding bytes between members, which is why sizeof a struct can be larger than the sum of its members' individual sizes.
Bad has 1 + 4 + 1 = 6 bytes of actual data but reports sizeof 12, because the compiler pads after a (3 bytes, so b starts on a 4-byte boundary) and after c (3 bytes, so the whole struct's size is a multiple of its 4-byte alignment). Good has the same three members reordered largest-first, needing only 2 bytes of trailing padding -- same data, 33% smaller. This is why ordering struct members from largest to smallest alignment is a common space-saving convention.
alignof(T) reports a type's required alignment, and alignas(N) can force a stricter (larger) alignment than the compiler would pick by default -- useful for SIMD types or avoiding false sharing (covered below).
#include <iostream>
struct Bad
{
char a; // 1 byte, needs 3 bytes padding to align the next int
int b; // 4 bytes
char c; // 1 byte, needs 3 bytes padding to align the struct to 4
};
struct Good
{
int b; // 4 bytes
char a; // 1 byte
char c; // 1 byte, 2 bytes padding to reach 4-byte struct alignment
};
int main()
{
std::cout << "sizeof(Bad): " << sizeof(Bad) << "\n"; // typically 12
std::cout << "sizeof(Good): " << sizeof(Good) << "\n"; // typically 8
}Endianness
Endianness describes the byte order used to store a multi-byte value in memory. On a little-endian system (the overwhelming majority of modern hardware, including x86 and most ARM configurations), the least-significant byte is stored at the lowest address; on a big-endian system, the most-significant byte comes first. This is invisible for ordinary arithmetic but matters the instant raw bytes cross a boundary -- reading a binary file written on a different-endian machine, or interpreting network protocol bytes (which are conventionally big-endian, sometimes called 'network byte order').
The 32-bit value 0x12345678 occupies 4 bytes in memory, but which byte comes first depends entirely on the platform's endianness -- the value itself, and every arithmetic operation on it, behaves identically either way. Endianness only becomes visible when you reinterpret an object's raw bytes directly, which is exactly what memcpy into a char array does here.
#include <cstdint>
#include <cstring>
#include <iostream>
int main()
{
uint32_t value = 0x12345678;
unsigned char bytes[4];
std::memcpy(bytes, &value, sizeof value);
for (int i = 0; i < 4; ++i)
std::printf("%02x ", bytes[i]);
std::printf("\n");
// Little-endian output: 78 56 34 12
// Big-endian output: 12 34 56 78
}Cache Locality and False Sharing
Modern CPUs read memory into a cache in fixed-size lines (commonly 64 bytes), and accessing anything already in cache is roughly 10-100 times faster than a main-memory access. Cache locality means arranging data so that things accessed together are stored close together, so one cache-line fetch serves many subsequent accesses -- exactly why iterating a std::vector<int> is typically much faster than iterating an equivalent-length linked list of ints, even though both are O(n): the vector's contiguous elements share cache lines, while the list's scattered nodes each cost a fresh, likely cache-missing, memory fetch.
False sharing is the multithreaded flip side of the same mechanism: if two variables used by different threads happen to live on the same cache line, every write by one thread invalidates that entire line in the other thread's cache -- even though the threads never touch each other's variable. The result is memory-coherence traffic that can make multithreaded code dramatically slower than expected, with no logical data race at all.
In BadCounters, counter_a and counter_b likely sit within the same 64-byte cache line, so a thread hammering counter_a from one core keeps invalidating counter_b's cache line on another core that's hammering counter_b, and vice versa -- pure cache-coherence overhead. GoodCounters uses alignas(64) to force each atomic onto its own cache line, eliminating that false-sharing penalty entirely, at the cost of extra padding memory.
#include <atomic>
struct BadCounters
{
std::atomic<long> counter_a; // likely shares a cache line with counter_b
std::atomic<long> counter_b;
};
struct GoodCounters
{
alignas(64) std::atomic<long> counter_a; // padded to its own cache line
alignas(64) std::atomic<long> counter_b; // padded to its own cache line
};Memory Fragmentation
Fragmentation is what happens when a heap's free memory becomes scattered into many small, non-contiguous gaps rather than existing as large contiguous blocks -- often after a long sequence of allocations and deallocations of varying sizes. Even when the total free memory is large, a request for one big contiguous block can fail if no single gap is large enough, despite the sum of all gaps being more than sufficient.
Long-running server processes are especially vulnerable, since they allocate and free memory continuously for years without restarting. Mitigations include using a memory pool or arena allocator that groups similarly-sized or similarly-lived allocations together, preferring fixed-size allocations where possible, and periodically restarting or compacting long-lived processes when the platform allows it.
Allocators, Custom Allocators, and Memory Pools
Every standard container is actually a template over both its element type and an Allocator type (std::vector<T, Allocator = std::allocator<T>>) -- std::allocator<T> is just the default, calling ::operator new/::operator delete under the hood. Supplying a custom allocator lets a container obtain memory from somewhere other than the global heap, without changing any of the container's logic.
A memory pool (or arena) is a common custom-allocator strategy: it grabs one large block of memory up front and then hands out pieces of it on request, tracking only where the next free piece starts rather than doing general-purpose bookkeeping for arbitrary sizes. Pools are dramatically faster than the general-purpose heap for the specific pattern they're built for -- typically many same-size, short-lived allocations -- because 'allocate' becomes a pointer bump and 'deallocate' can become a no-op until the whole pool is reset at once.
allocate() does nothing but bump offset_ forward and hand back the old position -- no free-list search, no bookkeeping per allocation. reset() 'frees' every allocation made since the pool was created in a single constant-time operation, which is exactly the trade-off a pool makes: individual pieces can't be freed independently, but the whole batch is trivially fast to release together.
#include <cstddef>
#include <iostream>
class SimplePool
{
public:
explicit SimplePool(std::size_t size) : buffer_(new unsigned char[size]), size_(size) {}
~SimplePool() { delete[] buffer_; }
void *allocate(std::size_t n)
{
if (offset_ + n > size_)
return nullptr; // pool exhausted
void *ptr = buffer_ + offset_;
offset_ += n;
return ptr;
}
void reset() { offset_ = 0; } // "free" everything at once
private:
unsigned char *buffer_;
std::size_t size_;
std::size_t offset_ = 0;
};
int main()
{
SimplePool pool(1024);
void *a = pool.allocate(64);
void *b = pool.allocate(128);
std::cout << (a != nullptr) << " " << (b != nullptr) << "\n";
pool.reset(); // both a and b are conceptually freed in one O(1) step
}std::pmr: Polymorphic Allocators
Before C++17, changing a container's allocator meant changing its template type -- a std::vector<int, MyAllocator> is a different type from std::vector<int>, and the two can't be assigned or compared without extra work. std::pmr (<memory_resource>, C++17) fixes this: std::pmr::polymorphic_allocator wraps a std::pmr::memory_resource* at runtime instead of at compile time, so std::pmr::vector<int> is one single type regardless of which memory resource actually backs it.
std::pmr::monotonic_buffer_resource is a ready-made pool resource that allocates linearly out of the fixed-size stack array buffer and never releases individual pieces (much like SimplePool above). Passing &pool to v's constructor makes every push_back pull memory from that stack buffer instead of calling the global allocator, all while v remains an ordinary std::pmr::vector<int> -- no template parameter had to change.
Why is stack allocation generally much faster than heap allocation?
Stack allocation is just moving a stack pointer by a known, fixed offset when a function is entered, and moving it back when the function returns -- no search or bookkeeping is involved. Heap allocation requires the allocator to find a suitably sized free block (potentially searching a free list or similar structure), mark it as in-use, and later reverse that bookkeeping on deallocation, all of which costs real time compared to a stack pointer bump.
Why can sizeof(SomeStruct) be larger than the sum of its members' individual sizes, and how does reordering members help?
The compiler inserts padding bytes between (and sometimes after) members so that each member sits at an address satisfying its own alignment requirement, and so the struct's total size is a multiple of its own alignment. Ordering members from largest alignment/size to smallest generally minimizes the total padding needed, since large-then-small groupings tend to fall on natural boundaries more often than small-then-large ones, reducing sizeof compared to a poorly ordered layout with identical members.
What is false sharing, and why can it slow down multithreaded code that has no actual data race?
False sharing happens when two variables used by different threads happen to reside on the same CPU cache line. Even though the threads never touch each other's variable (no data race), a write by one thread invalidates the whole cache line in the other thread's cache, forcing an expensive cache-coherence refresh -- so performance suffers purely from memory layout, not from any logical conflict over the data itself.
How can a heap have plenty of total free memory and still fail to satisfy a large allocation request?
Fragmentation can scatter free memory into many small, non-contiguous gaps rather than one or a few large contiguous blocks. A request for one large contiguous block fails if no single gap is big enough, even if the sum of all the small gaps far exceeds the requested size -- the allocator can't stitch separate gaps together into one usable block.
What problem does std::pmr::polymorphic_allocator solve that a traditional template-parameter allocator (like std::vector<T, MyAllocator>) does not?
A traditional allocator is baked into the container's type via a template parameter, so std::vector<int, MyAllocator> and std::vector<int, std::allocator<int>> are different, largely incompatible types. std::pmr::polymorphic_allocator instead stores a runtime pointer to a memory_resource, so std::pmr::vector<int> is always the same type no matter which memory_resource backs a particular instance -- letting code swap allocation strategy at runtime, or pass pmr containers across interfaces, without changing types.
Write a C++ program that defines two structs with the same three members (a bool, a double, and a char) in different orders, prints sizeof each with std::cout, and shows that member order affects total size due to padding.
Both structs store identical logical data (a bool, a double, a char), but Unordered typically reports 24 bytes (1 byte for b, 7 bytes padding to align d on an 8-byte boundary, 8 bytes for d, 1 byte for c, 7 bytes trailing padding) while Ordered typically reports 16 bytes (8 bytes for d, 1 byte for b, 1 byte for c, 6 bytes trailing padding). Placing the largest-alignment member first lets the smaller members pack together afterward, needing far less padding overall -- the exact byte counts can vary by compiler and platform, but Ordered is never larger than Unordered for the same member set.
Write a C++ program implementing a simple bump-pointer memory pool class (allocate() advances an offset into a pre-allocated buffer, reset() sets the offset back to zero), and demonstrate allocating several objects from it followed by a reset.
allocate() never searches for free blocks -- it simply checks that enough room remains, hands back the current offset as the new allocation's address, and advances the offset by the requested size, making each call O(1). After allocating 32 + 64 + 16 = 112 bytes, used() correctly reports 112; calling reset() drops the offset back to 0 in constant time, conceptually freeing every previous allocation from the pool at once rather than requiring them to be released individually.
What does this program print, and why does the loop over arr (a plain array) typically run faster than an equivalent loop over a linked list of the same length, even though both are O(n)?
It prints 15. Both a contiguous array traversal and a linked-list traversal of the same length are O(n) in the number of elements, but the array's elements are stored back-to-back in memory, so a single cache-line fetch (commonly 64 bytes, enough for many ints) brings in several consecutive elements at once -- most accesses after the first are served entirely from cache. A linked list's nodes are typically scattered across the heap, so each hop to the next node is likely to miss the cache and require a fresh, much slower main-memory fetch, making the constant factor behind the linked list's O(n) far larger in practice despite the identical asymptotic complexity.
#include <memory_resource>
#include <vector>
#include <iostream>
int main()
{
unsigned char buffer[1024];
std::pmr::monotonic_buffer_resource pool{buffer, sizeof buffer};
std::pmr::vector<int> v{&pool}; // allocates from `buffer`, not the global heap
for (int i = 0; i < 10; ++i)
v.push_back(i * i);
for (int x : v)
std::cout << x << " ";
std::cout << "\n";
}#include <iostream>
int main()
{
const int n = 5;
int arr[n] = {1, 2, 3, 4, 5};
long sum = 0;
for (int i = 0; i < n; ++i)
sum += arr[i];
std::cout << sum << "\n";
}