Programming glossary 64 terms mentioned in this lesson
Select any term for a clear definition. The return button brings you back to the exact term link you used.
Lesson: Arrays, Locality & Performance (C++)
Ensuring that related state lives together in memory pays off—especially in programs where timeliness is essential. When data that is used in the same part of an execution trace is laid out contiguously, the CPU hits cache more often and avoids expensive page faults. That is why the humble array—a sequence of same-typed elements stored in contiguous memory—is the fundamental building block for performance-oriented programmers.
Programming Insight (AI) — Locality Advisor
- Paste your loop; ask for a locality check (row-major vs column-major access, loop order, fusion/splitting suggestions).
- Have AI propose a data layout (AoS vs SoA) to improve cache behaviour for your workload.
1) What a C++ Array Is (and why it’s fast)
A C++ array stores N objects of the same type in one contiguous block. This allows hardware prefetchers to bring in the next elements efficiently (cache-line friendly) and makes iteration very cheap.
int a[5] = {1, 2, 3, 4, 5}; // 5 contiguous ints
- Contiguity helps: sequential access benefits from spatial locality.
- Arrays have a fixed size known at compile time.
- Any complete type works: built-ins, structs/classes, even user-defined types (UDTs).
Programming Insight (AI) — Pick the Right Container
- Describe size/resize needs; get a recommendation:
T[N],std::array<T,N>,std::vector<T>, orstd::unique_ptr<T[]>. - Ask for a “cache-aware” pass (reserve, padding/alignment tips, SoA refactor).
2) Scope, Storage & Size
Fixed-size arrays require the element count at compile time:
int a[5]; // ok: size is a constant expression
// int n = read(); // runtime value
// int b[n]; // ❌ Variable Length Arrays are not standard C++
Safer fixed-size alternative: std::array<T, N> (knows its size, works with STL, still contiguous):
#include <array>
std::array<int, 5> a = {1,2,3,4,5};
for (int x : a) { /* ... */ }
Runtime-sized, still contiguous: use std::vector<T> (dynamic size) or std::unique_ptr<T[]> (manual size, no resize):
#include <vector>
std::vector<int> v(5); // 5 ints, contiguous, resizable
auto buf = std::make_unique<int[]>(5); // contiguous, fixed count, RAII
3) Arrays of User-Defined Types (and variable-sized internals)
Arrays hold elements of the same type and size. If elements vary logically in size, store handles/pointers/references to
independently allocated data (or use types like std::string that manage their own dynamic storage).
struct Blob { std::size_t n; unsigned char* data; }; // handle to variable-sized payload
Blob blobs[3]; // contiguous Blobs; each Blob's 'data' may point elsewhere (heap)
Remember: an array of std::string is contiguous in its string objects, but not necessarily their character buffers.
4) Access Patterns & Locality (make the hardware happy)
Row-major order (C/C++ default for multi-dimensional arrays):
int m[ROWS][COLS];
// Best: iterate rows outside, columns inside
for (int r = 0; r < ROWS; ++r)
for (int c = 0; c < COLS; ++c)
use(m[r][c]); // contiguous progression
Bad locality example (column-first on row-major data) increases cache misses.
Programming Insight (AI) — Loop Order & Blocking
- Ask AI to rewrite your nested loops to match memory layout and propose blocking (tiling) for large matrices.
5) Arrays vs Vectors — When to Choose Which
- C-style array
T[N]: fixed N, no size metadata, decays to pointer on pass; very lightweight. std::array<T,N>: fixed N, knows its size, STL-friendly; same contiguous layout asT[N].std::vector<T>: dynamic size; contiguous, but may reallocate when growing (invalidates pointers/iterators).
// Avoid reallocation penalty by reserving
std::vector<float> samples;
samples.reserve(1'000'000); // contiguous block up front
for (...) samples.push_back(...);
Vectors do add a small overhead (size/capacity tracking, growth strategy), but are still contiguous and usually very fast. Prefer them when size isn’t known until runtime.
Programming Insight (AI) — Growth Strategy
- Provide rough element counts; AI suggests
reserve()sizes and when to switch to fixed storage.
6) Passing Arrays Safely (don’t lose the size!)
C-style arrays decay to pointers; you lose the length:
void f(int* data, std::size_t n); // pass pointer + size
Prefer std::span (C++20) or templates that capture size:
#include <span>
void g(std::span<const int> s) {
for (int x : s) { /* ... */ }
}
template<std::size_t N>
void h(const int (&arr)[N]) { /* N is known here */ }
7) Common Pitfalls (and fixes)
- Out-of-bounds: no bounds checks on
T[N]; preferstd::array/std::vectorand checked access in debug (.at()). - Pointer decay: passing
ato a function loses size; passstd::spanor size parameter. - VLA usage: Variable Length Arrays are non-standard in C++; use
std::vectororstd::unique_ptr<T[]>. - Reallocation invalidation: keep in mind that
std::vector::push_backmay invalidate pointers/iterators; callreserve()first.
Programming Insight (AI) — Bounds & Invalidation Audit
- Ask AI to scan for potential out-of-bounds and iterator invalidation; get refactors to
span/at()/reserve().
8) Worked Examples
A) Summing an array with good locality
#include <cstddef>
int sum(const int* p, std::size_t n) {
int s = 0;
for (std::size_t i = 0; i < n; ++i) s += p[i]; // linear access
return s;
}
B) 2D traversal (row-major friendly)
constexpr int R = 512, C = 512;
int m[R][C] = {};
for (int r = 0; r < R; ++r)
for (int c = 0; c < C; ++c)
m[r][c] = r + c; // sequential in memory per row
C) Fixed vs dynamic
#include <array>
#include <vector>
std::array<float, 1024> a; // compile-time size, no allocation at runtime
std::vector<float> b(1024); // runtime-sized, resizable if needed
9) Mini Exercise — Choose the Right Structure
Scenario: You will collect ~1,000,000 samples, then process them once (no further growth after load).
- Option 1:
std::vector<float> samples; samples.reserve(1'000'000);thenpush_backduring load. - Option 2: Two-phase: read the exact count first, then construct
std::vector<float> samples(count);and fill by index. - Option 3: If the count is a true compile-time constant, use
std::array<float, N>.
Programming Insight (AI) — Pick & Justify
- Ask AI to justify a choice based on determinism, memory footprint, and cache behaviour for your specific workload.
Summary Checklist
- Use arrays (
T[N]orstd::array) for fixed-size, performance-critical data; enjoy contiguous storage. - Use
std::vectorfor runtime-sized contiguous sequences; callreserve()if you know the count. - Iterate in memory order (row-major) and keep hot data together for cache efficiency.
- Avoid non-standard VLAs; pass arrays with size info (prefer
std::spanor templates). - When elements vary in size, store handles in the array (pointers/indices) or use owning types like
std::string.
Advanced perspective: a sequence is useful only while its boundary remains attached
Contiguous storage does not excuse an access whose bound cannot be proved
An array gives us a sequence of adjacent elements of one type. That layout can support efficient traversal and simple interoperability, but neither property answers the first correctness question: does this particular subscript designate an element of this particular sequence? For int values[3], the valid indices are 0, 1 and 2. The expression values[3] steps beyond the elements. A program that appears to survive the access has demonstrated only that one failure was not immediately visible.
The bound is not a suggestion for sensible input. It is part of the proof required before each indexed access. If count is the number of elements, a valid subscript satisfies 0 <= index && index < count. For an unsigned index the lower comparison is implicit in the type, but the upper comparison remains essential. A pointer may be formed one position past the final element for range reasoning; dereferencing that one-past value is still invalid.
| Claim | What supports it | What does not support it |
|---|---|---|
| The index is valid. | The index is compared with the current extent of the same sequence. | The loop ran without crashing yesterday. |
| The pointer may be dereferenced. | It designates a live element within the permitted range. | It is non-null or numerically close to another valid address. |
| The saved bound is current. | No intervening operation changed the logical sequence it describes. | The variable is named size or count. |
| The traversal is efficient. | Measurement on the relevant data, build and machine supports the claim. | The elements happen to be contiguous. |
Array-to-pointer conversion removes information from the interface
The type of a built-in array object includes its element count. While the expression still refers to that array, facilities such as std::size can obtain the bound. In many expression contexts, however, the array is converted to a pointer to its first element. A function parameter written with array-looking syntax is adjusted to a pointer parameter; writing a number inside those brackets does not make the function receive or enforce that number.
Once the interface contains only const int* data, the callee cannot recover how many integers the caller intended to make available. A separate count can restore the contract by convention, but pointer and count can disagree. Which sequence produced the pointer? Is the count measured in elements or bytes? Was it captured before the owner resized? Each question exists because the type no longer carries the complete relationship.
| Interface shape | Information retained | Remaining responsibility |
|---|---|---|
const int* data | An access path to a possible first element. | The extent and lifetime must be established elsewhere. |
const int* data, std::size_t count | Address and claimed element count. | The caller must supply a matching pair. |
Reference to const int (&)[N] | The built-in array and its compile-time bound. | The interface accepts that array form and fixed extent. |
std::span<const int> | A non-owning contiguous range and its extent as one view. | The owner must keep the elements alive and uninvalidated. |
Keep ownership and extent visible at the call
#include <array>
#include <iostream>
#include <span>
int sum(std::span<const int> values) {
int total{0};
for (const int value : values) {
total += value;
}
return total;
}
int main() {
const std::array values{4, 7, 9};
std::cout << sum(values) << '\n';
}
The std::array in main owns three integers as part of its value. The span passed to sum is a view of those elements. It does not copy them and it does not own them. Its const element type prevents this function from modifying the integers through the view; it does not extend their lifetime.
The range-based loop obtains the elements described by the span without making the function invent a second count or maintain a manual index. The trace is 0 plus 4, then 7, then 9, so the result is 20. That visible arithmetic is useful, but the interface is doing the more important work. Data and extent arrive together, while ownership remains visibly with the caller for the duration of this call.
| Object or view | Owns elements? | Can change length? | What its type communicates |
|---|---|---|---|
Built-in T[N] | Yes, as part of the array object. | No. | A fixed contiguous sequence; the bound is present before decay. |
std::array<T, N> | Yes. | No. | A fixed-size value with container operations. |
std::vector<T> | Yes. | Yes. | A runtime-sized contiguous sequence with managed storage. |
std::span<T> | No. | The view's extent is fixed after construction. | Borrowed contiguous elements, with either static or dynamic extent. |
A span makes the bound available; it does not turn every operation into a checked operation. If code uses an explicit index, the program must still establish its validity or choose an operation whose contract performs the required check. Better information makes a better interface possible. It does not abolish preconditions.
Select the representation from change, ownership and identity
"Arrays are fast" is not a container-selection rule. First decide whether the element count is fixed by the problem or known only at runtime. Then decide who owns the elements, whether the sequence must resize, and whether existing references must survive changes. A std::array is appropriate when the count is genuinely fixed and value semantics fit. A std::vector is appropriate when the program owns a contiguous sequence whose size changes at runtime. A span is appropriate when an operation borrows an already existing contiguous range.
Using a dynamic allocation merely because the count is large answers none of those questions. Nor does a built-in array become the correct choice merely because it has no separate size or capacity fields. Representation overhead matters only after the required behaviour is correct and the workload shows that the difference is material.
Growth changes more than the number reported by size()
A vector owns a sequence and normally retains capacity for some number of elements. When an insertion requires more storage than the current capacity, the vector obtains a new allocation, moves or copies its elements as required, and releases the old allocation. Pointers, references and iterators into the former allocation are then invalid. The vector object is still alive. Its old element addresses are not.
reserve can establish enough capacity for a known amount of growth. If it reallocates, references into the old storage are invalidated at that point. Successful later insertions that remain within capacity avoid reallocation, but this is a bounded statement about those operations, not a permanent licence to store addresses. The past call to reserve must be compared with the actual capacity and subsequent growth.
| Event | Sequence state | Borrower question |
|---|---|---|
| Construct a vector with three elements. | Three live elements in owned contiguous storage. | Any view must stay within those three elements. |
| Save a pointer to element zero. | The vector remains the owner. | The pointer is valid only while that element and allocation remain valid. |
| Insert without reallocation. | The sequence grows in existing storage. | Check the specified invalidation rules, including the old end position. |
| Insert with reallocation. | Elements occupy a new allocation. | Old pointers, references and iterators into the elements are invalid. |
Contiguity permits a locality argument; it does not finish one
For int grid[2][3], the rightmost index varies across adjacent elements. Row-first traversal therefore follows the stored order, while column-first traversal uses a larger stride. On a sufficiently large and repeatedly processed row-major data set, that difference can affect cache behaviour. This is a mechanism worth testing, not a ritual for rearranging every nested loop.
Observed performance also depends on element size, working-set size, compiler optimisation, access frequency, hardware, surrounding work and whether a different data organisation reduces the useful computation. An array of objects may be contiguous in those objects while the data reached through members is allocated elsewhere. An array of std::string places the string objects together; it does not promise that all character buffers follow one another. Draw what is actually contiguous before predicting what the cache will see.
The same discipline applies to array-of-structures and structure-of-arrays designs. The correct choice depends on which fields are accessed together across which elements. A benchmark should represent that access pattern and preserve the program's required behaviour. Faster output from a test that quietly computes less is not a locality success.
Programming Insight (AI): require a bound source and an invalidation proof
When AI produces a loop, ask where the extent came from, which sequence it describes, and which operation could make it stale. Require tests for an empty range, one element, the final legal index and the first rejected index. If a pointer, iterator or span is retained, mark its owner and every mutation between creation and final use.
For a performance suggestion, require the claimed mechanism and a measurement plan. "Contiguous is cache friendly" is a starting hypothesis. The answer must still identify traversal order, data volume, fields used, build configuration and a correctness check for the transformed code. Otherwise a confident optimisation can conceal either a boundary error or a benchmark that measures the wrong task.
Transfer task: preserve the slice, not merely its first address
Design a function that sums elements first through last of a caller-owned vector, with first included and last excluded. State what must be true when the slice is empty, when it covers the full vector, and when either endpoint is outside the vector. Decide whether the function should receive the vector and two indices, a span already restricted by the caller, or a different representation.
Now assume the caller saves that slice, appends elements to the vector and uses the slice again. Draw the owner, data allocation, extent and saved view before and after the append. Your answer must identify the condition under which reallocation occurs and what that does to the view. Do not repair the design by saying "call reserve" until you have stated the maximum growth that makes that promise sufficient.
Reveal answer
The summing operation does not need to own a vector and it does not need to interpret two unrelated integers. It needs one already validated range. I would therefore separate checking the requested endpoints from processing the resulting span:
#include <cstddef>
#include <cstdint>
#include <numeric>
#include <optional>
#include <span>
#include <vector>
std::optional<std::span<const int>> checked_slice(
const std::vector<int>& values,
std::size_t first,
std::size_t last) noexcept
{
if (first > last || last > values.size()) {
return std::nullopt;
}
std::span<const int> all{values};
return all.subspan(first, last - first);
}
std::optional<std::span<const int>> checked_slice(
std::vector<int>&&,
std::size_t,
std::size_t) = delete;
std::optional<std::span<const int>> checked_slice(
const std::vector<int>&&,
std::size_t,
std::size_t) = delete;
std::int64_t sum(std::span<const int> values)
{
return std::accumulate(
values.begin(), values.end(), std::int64_t{0});
}
The contract is now visible. An empty slice is valid when first == last, including when both equal values.size(), and its sum is zero. The full slice is [0, values.size()). Any request with first > last or last > values.size() is rejected before subspan is called. The deleted rvalue overloads prevent a temporary vector from producing a span whose owner disappears at the end of the call. The wider accumulator avoids ordinary int overflow for realistic vectors of integers; a domain that can exceed std::int64_t still needs an explicit checked-arithmetic policy.
The saved view remains non-owning. Its lifetime picture is:
| Moment | Owner | Allocation | Saved view |
|---|---|---|---|
| Before append | The vector owns allocation A. | A contains the vector's live elements. | A pointer into A plus the fixed slice extent. |
| Append within capacity | The vector still owns A. | The new element is constructed in A. | A view whose data pointer is before the old end remains usable and does not grow. An empty view positioned at the old end should be created again because that past-the-end position is invalidated. |
| Append beyond capacity | The vector owns a new allocation B. | Elements move or copy to B and A is released. | The saved pointer still names A and is dangling. Using the view has undefined behaviour. |
For one push_back, reallocation is required when the old size() equals the old capacity(). More generally, a sequence of appends reallocates when the required new size exceeds the current capacity. Calling reserve is sufficient only if the maximum growth is known. If the vector currently contains n elements and at most g will be appended, call reserve(n + g) before creating the span, check that the addition itself is representable, and do not exceed that bound. Calling reserve after saving the view is too late because reserve may itself reallocate.
If the growth bound is not known, do not retain the span across mutation. Save the two indices and create a fresh checked span after the append when the slice is meant to follow the vector. Copy the elements into a new vector when the slice is meant to be an independent snapshot. Those are different lifetime promises, and the representation should say which one was intended.
A safe sequence interface keeps four facts connected: the storage owner, the first element, the number of permitted elements and the operations that can invalidate the relationship. Lose any one of them and the array may remain contiguous while the program's reasoning develops a hole.