Rust Memory Layout: Optimizing Struct Size and Alignment for Cache Locality

During a high-throughput production incident at a payment gateway, our transaction processing service hit a wall at 85,000 transactions per second (TPS). CPU utilization was pegged at 100%, but profiling with perf revealed that the CPU cores were spending 42% of their cycles stalled on memory fetches (L1-dcache-load-misses). The root cause was not database latency or network serialization, but a poorly aligned internal state struct that straddled L1/L2 cache line boundaries. By reordering the struct fields and enforcing strict cache alignment, we reduced the struct size by 33%, eliminated cache line split loads, and saw throughput immediately scale to 125,000 TPS with a 30% reduction in CPU consumption.

The High Price of Cache Misses in Modern Hardware

Modern CPUs are incredibly fast, but memory is slow. To bridge this gap, hardware designers use hierarchical caches: L1 (fastest, smallest), L2, and L3 (slowest, largest). While an L1 cache hit takes approximately 1 to 2 nanoseconds (4 to 5 CPU cycles), retrieving data from main memory (DRAM) requires 60 to 100 nanoseconds—a latency penalty of up to 200 CPU cycles. During this time, the CPU core is idle, unable to progress through its instruction pipeline.

Caches do not load memory byte-by-byte; they operate on fixed-size blocks called cache lines, which are almost universally 64 bytes on modern x86_64 and ARM64 processors. When your program accesses a single 8-byte integer, the CPU fetches the entire 64-byte cache line containing that integer. If your data structures are bloated with alignment padding or scattered randomly across the heap, you pollute your caches with useless data. This reduces the cache density—the ratio of useful payload to total cached bytes. A struct that is bloated from 16 bytes to 24 bytes due to padding means that a 64-byte cache line can only hold two instances instead of four, effectively cutting your cache capacity by half and doubling your L1 cache miss rate in hot loops.

Understanding Rust’s Alignment Rules and the Default Compiler Strategy

Every type in Rust has a size (the number of bytes it occupies) and an alignment requirement (the memory addresses at which it can be safely stored). You can query these programmatically using std::mem::size_of::<T>() and std::mem::align_of::<T>().

For primitive types, the alignment matches the size:

• u8 / i8: Size = 1 byte, Alignment = 1 byte
• u16 / i16: Size = 2 bytes, Alignment = 2 bytes
• u32 / f32: Size = 4 bytes, Alignment = 4 bytes
• u64 / f64 / usize: Size = 8 bytes, Alignment = 8 bytes

To guarantee that fields are stored at addresses that are multiples of their alignment, the compiler must insert padding bytes (empty space that carries no information). By default, Rust uses the repr(Rust) representation. The compiler is free to reorder fields of a repr(Rust) struct to minimize size and padding. However, when we need stable layouts—such as for FFI (Foreign Function Interface), network protocol decoding, or zero-copy disk serialization—we must opt out of this behavior using #[repr(C)]. Under repr(C), the fields are laid out in the exact order they are declared, and poor ordering will result in severe memory bloat.

Zero-Copy Serialization and repr(C) Optimization

In low-latency backends, serialization and deserialization (e.g., JSON parsing or Protobuf decoding) are major CPU bottlenecks. To achieve microsecond-level parsing, systems like database storage engines, ledger backends, and networking proxies utilize zero-copy deserialization. Libraries like bytemuck or zerocopy allow casting a raw byte buffer (such as from a TCP socket or memory-mapped file) directly into a Rust struct reference.

For this cast to be safe, the struct must be marked #[repr(C)] to prevent compiler reordering, and it must implement traits ensuring it contains no invalid bit patterns. In this scenario, manual layout optimization becomes mandatory. Let’s look at a production-quality network packet parser implementing zero-copy casts.

The Silent Performance Killer: Cache Line Split Loads

When a data structure is not properly aligned, or when it crosses a cache line boundary, CPU performance drops sharply. This occurs because of split loads (and split stores). If a 8-byte integer starts at byte 60 of a cache line, the first 4 bytes reside in cache line $N$, and the last 4 bytes reside in cache line $N+1$.

To read this integer, the CPU must fetch both cache line $N$ and cache line $N+1$ from the cache hierarchy, merge the bytes in its internal registers, and only then proceed with the instruction. This doubles the L1 cache bandwidth consumption and introduces pipeline bubbles.

We can benchmark the latency difference of memory access between cache-aligned memory access and unaligned boundary-crossing access.

On modern Intel Xeon processors, executing sum_unaligned_offset with offset 60 (forcing split loads on every single iteration) is 1.8x to 2.4x slower than running it with offset 0 (where the integers are perfectly aligned with the cache line starts), even when all data is entirely resident in L1 cache.

Multi-Threaded Disasters: False Sharing in Concurrent Data Structures

In multi-threaded backends, memory alignment goes beyond optimizing single-thread access. A massive performance hazard is false sharing. False sharing occurs when two threads running on different CPU cores modify independent variables that happen to reside in the same 64-byte cache line.

Modern CPUs maintain cache coherence via protocols like MESI. If Core 1 modifies variable head on cache line $X$, the cache line is marked as Modified on Core 1 and Invalid on Core 2’s cache. When Core 2 tries to read or write to variable tail, which is also located on cache line $X$, it detects the invalidation. Core 2 is forced to halt, wait for Core 1 to write the cache line back to L3 or DRAM, and then fetch the updated cache line. This cache-line “ping-pong” completely destroys the scalability of lock-free queue structures, ring buffers, and counters.

To prevent false sharing, we must isolate hot, concurrently accessed variables onto their own independent cache lines using #[repr(align(64))] (or 128 bytes depending on target architecture).

When to Pack: The Tradeoffs and Undefined Behavior of repr(packed)

If you are building an in-memory database or a key-value store containing billions of small records in RAM, you might prioritize memory usage over cache line layout. Under repr(C), Rust inserts padding to satisfy alignment requirements. To force the size to its absolute minimum, you can use #[repr(packed)] (or #[repr(packed(N))]). This overrides alignment rules, squeezing out all padding and aligning fields to 1 byte.

However, packing has severe consequences:

1. CPU Speed degradation: On x86_64, unaligned access is supported in hardware but suffers a performance penalty. On some ARM64 and MIPS cores, unaligned memory accesses are not supported in hardware and will crash the program with a bus error or trigger a slow kernel trap.
2. Undefined Behavior (UB): Creating a standard reference (&T) to an unaligned field in Rust is undefined behavior. References must always be aligned. If you pass an unaligned reference to a function, the compiler assumes it is aligned, leading to miscompiled code and random segfaults.

Here is how you must handle packed structs safely:

Maximizing Space in Enums: Niche Filling and Box Payload Optimization

In Rust, enums are tagged unions. The size of an enum is equal to the size of its largest variant plus the size of the discriminant tag, aligned to the largest alignment requirement among its variants. An unoptimized enum variant can bloat the memory footprint of an entire array of elements.

Rust provides a powerful optimization called niche filling. If a type has invalid bit patterns (niches), the compiler uses those invalid patterns to represent enum tags. For example, a reference &T or Box<T> cannot be null (address 0). The compiler uses the 0 bit pattern to represent None in Option<Box<T>>, making Option<Box<T>> the exact same size as Box<T>.

When building production enums, you must ensure that a single large variant does not bloat the entire enum. If one variant is 100 bytes and all others are 8 bytes, every instance of the enum will occupy 104+ bytes. The solution is to box the large variant to keep the enum size small.

CI/CD Pipelines: Enforcing Layout Quality Automatically

Memory optimizations are fragile. It is incredibly easy for a developer to add a field to a critical database index struct or a networking packet, misorder it, and silently degrade performance. To guarantee layout quality, you must automate tests and checks.

There are three key strategies for enforcing struct layout in CI/CD:

1. Nightly Compiler Layout Printing: Run cargo rustc -- -Z print-type-sizes on your codebase. This lists the exact offsets, size, padding, and alignments of every type in your application. You can parse this output in CI to detect unexpected padding.
2. Cargo Bloat: Integrate cargo-bloat into your pipeline to monitor how changes affect binary sizes and structural footprints.
3. Compile-Time Size Assertions: Use inline static assertions on performance-critical data structures. If anyone modifies a struct and increases its size or changes its alignment, the compiler will refuse to build the application.

By enforcing compile-time assertions and carefully designing struct layouts with cache locality in mind, you can eliminate memory access bottlenecks. In high-performance backend systems, cache efficiency is not a micro-optimization—it is the line between a system that scales linearly and one that stalls on RAM limitations. Keep your structs small, pack them descending by alignment size, isolate concurrent fields, and write static compile checks to lock in those performance gains forever.