Performance and Memory Efficiency
Learning Objectives
Identify OO performance costs: allocation churn, indirection, poor locality, virtual dispatch.
Apply data-oriented alternatives, small-object optimization, moves and measured pooling.
Profile and benchmark before optimizing.
Prerequisites: Modules 12, 13, 55.
Intuition and Definition
Performance is dominated by memory access patterns more than by instruction counts: a cache miss costs ~100 cycles, a virtual call a few. Typical OO costs:
new per object, temporary strings value types, reserve, arenas, pools (measured)
vector<unique_ptr<T>>, linked structures contiguous storage, vector<T>, variants hot and cold fields mixed split hot/cold, structure-of-arrays virtual calls in hot loops sort by type, templates, variants pass by value of large objects const&, moves, views
Small-object optimization: std::string stores short strings inside the object (no heap allocation); std::function stores small callables inline.
Why It Matters
OO designs are often "pointer-rich". For hot paths (physics, signal processing, rendering), changing the data layout can give large speed-ups without changing algorithms. For everything else, clarity wins — measure first.
Syntax and Language Notes
Benchmark with proper tools: Google Benchmark / nanobench (C++), BenchmarkDotNet (.NET), JMH (Java), pyperf/timeit (Python), mitata/tinybench (JS). Profile with perf, VTune, Instruments, dotTrace, py-spy.
Complete Example
Counting allocations (deterministic) rather than timing (machine-dependent) to compare layouts:
#include <cstdlib>
#include <iostream>
#include <memory>
#include <new>
#include <string>
#include <vector>
static long allocations = 0;
void* operator new(std::size_t n) { ++allocations; if (void* p = std::malloc(n)) return p; throw std::bad_alloc(); }
void operator delete(void* p) noexcept { std::free(p); }
void operator delete(void* p, std::size_t) noexcept { std::free(p); }
struct Particle { float x, y, vx, vy; };
struct ParticleObj { virtual ~ParticleObj() = default; float x = 0, y = 0, vx = 1, vy = 1; virtual void step() { x += vx; y += vy; } };
int main() {
const int N = 10'000;
allocations = 0;
std::vector<std::unique_ptr<ParticleObj>> objects; // pointer-rich OO layout
for (int i = 0; i < N; ++i) objects.push_back(std::make_unique<ParticleObj>());
std::cout << "vector<unique_ptr>: " << allocations << " allocations\n";
allocations = 0;
std::vector<Particle> values; // contiguous values
values.reserve(N); // one allocation up front
for (int i = 0; i < N; ++i) values.push_back({0, 0, 1, 1});
std::cout << "vector<Particle> with reserve: " << allocati
…vector<unique_ptr>: 10015 allocations
vector<Particle> with reserve: 1 allocation
vector<Particle> without reserve: 15 allocations
short string (7 chars): 0 allocations, long string (41 chars): 1
push_back(copy): 1 allocation, push_back(std::move): 0Step-by-Step Walkthrough
10,000 heap objects mean ≥10,000 allocations (plus the vector's own growth) and scattered memory; iteration chases pointers.
The same data as contiguous values with reserve is one allocation, and iteration streams through memory.
Without reserve, the vector grows geometrically: a handful of reallocations (each copying all elements).
Short strings live inside the std::string object (no allocation); long ones allocate. (libc++ keeps up to 22 characters inline on 64-bit targets; other libraries use 15.)
Moving a string transfers its buffer; copying allocates a new one.
Common Mistakes
Optimizing without measuring — the bottleneck is rarely where you guess.
Micro-benchmarks that the optimizer deletes (results unused) or that don't reflect real data sizes.
Pooling and caching everything (Module 32) — memory bloat and contention.
Sacrificing clarity in cold code for nanoseconds.
Trade-offs and Alternatives
Data-oriented design (structure-of-arrays, entity-component systems) maximizes throughput but loses some encapsulation; keep it behind a clean interface. Virtual dispatch is cheap compared with cache misses — fix layout first.
Real-World Use
Game engines (ECS), physics and audio processing, high-frequency trading, embedded control loops, database engines (columnar storage), and any server with tight latency budgets.
Practice
Q1. (Easy) Why does reserve help?
Answer: It allocates enough capacity once, avoiding repeated reallocations and element copies/moves.
Q2. (Medium) Why can vector<unique_ptr<T>> iterate slower than vector<T>?
Answer: Each element is a separate heap allocation; iterating follows pointers to scattered memory, causing cache misses, and the loop can't stream contiguous data.
Q3. (Advanced) What is structure-of-arrays and when does it help?
Answer: Storing each field in its own array (xs[], ys[]) instead of an array of structs. It helps when hot loops touch only some fields of many elements: only needed data is loaded into cache, and loops vectorize well.
Interview Question
Q1. "Your OO simulation is slow. How do you approach it?"
Answer: Measure first with a profiler on realistic data to find the hot spots. Common culprits: allocations in hot loops, pointer-chasing containers, poor locality, unnecessary copies, virtual calls per tiny object. Fixes in order of impact: better algorithms; contiguous data and reserve; hot/cold field splitting or structure-of-arrays; batching by type to make dispatch predictable or removing it (templates/variants); moves instead of copies; pooling only where allocation shows up in profiles. Re-measure after each change and keep the clean interface intact.
Quick Quiz
Q1. What is small-string optimization?
Answer: Storing short strings inside the string object without heap allocation.
Q2. First step in performance work?
Answer: Measure (profile) with realistic workloads.
Q3. Usually more expensive: a virtual call or a cache miss?
Answer: A cache miss.
Summary
OO performance problems are mostly allocation and locality problems.
Contiguous values, reserve, moves and data-oriented layouts help; measure before and after.
Related: Modules 32, 38, 55, 58.