C++ performance — measure, then optimize

The number-one mistake in optimization is optimizing without measurement. Donald Knuth: "Premature optimization is the root of all evil." This does not mean "never optimize" — it means "first measure where slow is." In 95% of cases the bottleneck is not where you thought.

Topic map

• Profiling — sampling vs instrumentation, tools (perf, VTune, callgrind, gprof).
• Benchmarking — micro-benchmarks via Google Benchmark, what they measure and what they don't.
• Optimization techniques — algorithmic, cache-friendly, branch prediction, SIMD, inlining.
• Anti-patterns — premature optimization, optimization without measurement, optimizing cold code.

Profiling

Before speeding anything up, find where it is slow. Two classes of tool:

Sampling profilers periodically (1000–10000 Hz) snapshot the call stack. Low overhead (1–5%), production-friendly.

• perf record/report (Linux) — the standard. Uses CPU PMU.
• VTune (Intel) — GUI, hotspots, micro-architecture analysis, memory access patterns.
• Instruments (macOS) — bundled with Xcode, time profiler + allocations.

Instrumentation profilers insert measurements at every function entry/exit. High overhead (2-10×), exact call counts.

• callgrind (Valgrind) — slow but produces an exact call graph and cache simulation.
• gprof — obsolete, kept for legacy.

⚠️ Profile a release build with -fno-omit-frame-pointer -g. A debug build will show fake bottlenecks.

Benchmarking

Google Benchmark is the micro-benchmarking standard. It correctly averages repeats and fights dead-code elimination via benchmark::DoNotOptimize.

static void BM_VectorPush(benchmark::State& state) {
  for (auto _ : state) {
    std::vector<int> v;
    for (int i = 0; i < state.range(0); ++i) v.push_back(i);
    benchmark::DoNotOptimize(v);
  }
}
BENCHMARK(BM_VectorPush)->Range(8, 8<<10);

⚠️ A micro-benchmark shows the speed of an isolated operation. It does not answer "how much faster will my program run" — for that you need a real-workload profile.

Optimization techniques (in order of payoff)

1. Algorithmic. O(n²) → O(n log n) — typically a 10–1000× speedup. Every other item below gives at most 2–10×.
2. Cache-friendly data. std::vector over std::list. SoA (Structure of Arrays) over AoS when you read by field.
3. Reduce allocations. A heap allocation costs 100–1000 cycles. Reserve, reuse, pool, stack-buffer.
4. Branch prediction. Sorting the input can make a branch predictable and speed up a loop 2–3×.
5. Inlining. Small functions in a hot loop. The compiler does most of it; [[gnu::always_inline]] for edge cases.
6. SIMD. AVX/AVX2/AVX-512 — 4–16× speedup for numeric code. The compiler's autovectorization + -O3 -march=native covers 80% of cases.
7. Parallelism. Threads, std::execution::par, GPU. Pays off after the sequential version is already optimized — otherwise you parallelize slow code.

Anti-patterns

• "Replace std::string with char*, it will be faster." Without a profile it is not faster, it is harder to read, and strlen bugs start appearing.
• Using the inline keyword for speed. Modern inline is about ODR, not optimization. The compiler decides on its own.
• Replacing i++ with ++i in a for (int i…). The compiler emits identical code. Only matters for iterators with expensive temporary destructors.
• Micro-optimizing cold code. If a function is 0.1% of the profile, doubling its speed improves the program by 0.05%.
• Optimizing in a debug build. Numbers there are meaningless — no inlining, no vectorization, plus iterator debugging.

Common traps

Interview relevance

Performance is a common senior topic, especially in trading, game dev, embedded, and databases. The interviewer checks two skills:

1. Method: "The program is slow — what do you do?" The right answer starts with "I profile and find the hot function," not "I'll rewrite it with std::array."
2. Tool knowledge: do you know sampling vs instrumentation, what cachegrind measures, when to use perf vs VTune.

Typical wrong answer: "I'd replace std::string with char* and std::vector with a raw array." That is cargo-cult. Without a profile, optimization is blind, and usually does more harm than good.

Popular question directions:

1. The program is slow. How will you profile it?
2. How does a sampling profiler differ from an instrumentation profiler?
3. What is a cache miss and how do you detect it?
4. When does inline affect performance, and when does it not?
5. SIMD: what is it, and when do you write it by hand instead of relying on the compiler?