Zero-Copy Serialization: Protobuf, FlatBuffers, and Cap'n Proto Compared

In distributed systems, data spends a significant portion of its life in transit. Before a data structure (like an object in memory) can be sent over a network card, it must be converted into a stream of bytes. This process is Serialization. On the receiving end, the reverse happens: Deserialization.

For years, Protocol Buffers (Protobuf) has been the gold standard for high-performance RPC systems (like gRPC). But as network cards scale to 100GbE+ and CPU cycles become the primary bottleneck, the overhead of Protobuf’s parsing phase is becoming a luxury some high-performance systems cannot afford.

Enter Zero-Copy Serialization formats like FlatBuffers and Cap’n Proto.

In this article, we’ll dive deep into the internals of serialization, dissect why Protobuf has high CPU overhead at scale, and compare how FlatBuffers and Cap’n Proto achieve zero-copy data access.

1. Why Protobuf has a Parsing Tax

To understand why Protobuf has CPU overhead, we have to look at its wire format and how it is read.

The Wire Format (Varints and Tags)

Protobuf uses a compact, binary, tag-value format. To save bytes over the wire, it uses Varints (variable-length integers). In a varint, the Most Significant Bit (MSB) of each byte is a “continuation bit”. If set to 1, it means the next byte is also part of the integer.

Integer 150 representation in Varint:
Binary: 10010110 00000001
        ^        ^
        │        └─ MSB is 0 (last byte of integer)
        └─ MSB is 1 (continue to next byte)

Because of this variable encoding, the compiler cannot know the byte-offset of any field in a message without parsing the preceding bytes.

The Deserialization Pipeline

When a server receives a Protobuf message:

1. Memory Allocation: The parser allocates memory on the heap for the target object graph (e.g., in Go or C++).
2. Bit-by-Bit Parsing: A state machine loops through the byte buffer, decoding varint tags and fields, doing bitwise shifts.
3. Data Copying: Data from the network packet buffer is copied into these newly allocated heap variables.

For a deeply nested message, this results in hundreds of small memory allocations, CPU cache misses, and significant garbage collection overhead. The deserialization phase can easily consume 40% of the entire processing time of a high-throughput microservice.

2. FlatBuffers: Vtables and Offsets

FlatBuffers (developed by Google) takes a completely different path. It is designed so that the wire format of the data is exactly the same as its in-memory representation.

FlatBuffers Wire Memory Layout:
[Offset to Table Start] ---> [Vtable Offset] [Field 1 Offset] [Field 2 Offset]
                             └─ Pointer to fields inside data section

The Magic of Vtables

Instead of packaging fields sequentially, FlatBuffers packages a vtable (virtual table) inside the byte buffer. The vtable contains the exact byte-offset of each field within that specific message.

When you access a field in FlatBuffers:

1. No Parsing: The library does not parse the buffer or copy any bytes.
2. Pointer Arithmetic: To read message.age(), the generated code reads the vtable offset, adds it to the buffer’s start address, and casts the target memory address directly to the primitive type (e.g., *const uint8_t).
3. Zero Allocation: No objects are allocated on the heap. The library reads data directly out of the raw packet memory buffer.

Write-Once Construction

To achieve this, FlatBuffers requires you to construct messages in a strict bottom-up order. You must write child elements before parent elements, appending data sequentially to a single flat buffer. This makes the writer code slightly more complex but guarantees lightning-fast reads.

3. Cap’n Proto: Pointer-Based Structs

Cap’n Proto was created by Kenton Varda, the primary author of Protobuf version 2. Having realized the limitations of Protobuf’s design, Kenton set out to create the ultimate zero-copy protocol.

Its motto: “Infinitely faster than Protocol Buffers because there is no encoding/decoding step.”

Memory-Aligned Arenas

Cap’n Proto structures the wire format to match the native memory alignment of modern CPUs (usually 8-byte alignment).

A Cap’n Proto struct contains two main segments:

1. Data Section: For fixed-size primitive values (e.g., int32, float64).
2. Pointer Section: For variable-size fields (strings, lists, sub-structs).

Cap'n Proto Struct Layout (8-byte aligned):
┌───────────────────────────┬───────────────────────────┐
│     Data Section (16B)     │   Pointers Section (16B)  │
│ [int32: age] [int32: ID]  │ [*string: name] [*list]   │
└───────────────────────────┴───────────────────────────┘

Because fields are aligned to 8-byte boundaries, a 64-bit CPU can load the values into registers using standard CPU load instructions directly from the network buffer, without shifting bits or dealing with byte boundaries.

The Pointer Representation

Pointers in Cap’n Proto are not absolute virtual addresses (which would be invalid on a receiving system). They are relative offsets (e.g., “the string starts 24 bytes after this pointer”).

When you receive a Cap’n Proto buffer:

1. You pass the raw byte array to your code.
2. Accessing a field involves resolving relative pointer offsets.
3. Because it matches CPU native struct layouts, you can often reinterpret_cast the buffer pointer straight to a C++ struct definition.

Performance Showdown: Protobuf vs. FlatBuffers vs. Cap’n Proto

To see the massive impact of zero-copy, consider these benchmarks measuring the time to read a complex payload 1,000,000 times (in C++):

Note: Deserialization time for FlatBuffers and Cap’n Proto is essentially just the time to set up a pointer reference—literally sub-nanosecond operations.

Technical Tradeoffs: Which One to Choose?

If zero-copy formats are so fast, why isn’t everyone using them instead of Protobuf?

1. Wire Size (Protobuf Wins)

Protobuf is highly optimized for size. By variable-encoding integers and omitting missing fields entirely, a Protobuf message is typically 30-50% smaller over the wire than FlatBuffers or Cap’n Proto. If your primary bottleneck is network bandwidth (e.g., mobile connections), Protobuf is still superior.

2. API Ergonomics (Protobuf Wins)

Writing a Protobuf message is as simple as working with native language structs:

msg := &pb.User{Name: "Alice", Age: 30}

Constructing a FlatBuffers message requires working with a sequential builder, which is verbose and error-prone:

builder := flatbuffers.NewBuilder(1024)
name := builder.CreateString("Alice")
flatbuffers.UserStart(builder)
flatbuffers.UserAddName(builder, name)
flatbuffers.UserAddAge(builder, 30)
user := flatbuffers.UserEnd(builder)

3. File I/O and IPC (Zero-Copy Wins)

If you are writing data to disk (e.g., high-performance log engines) or passing data between processes on the same host (Inter-Process Communication, Shared Memory), zero-copy formats are the undisputed champions. You can use mmap to map a file on disk directly to a memory pointer and read it instantly, without loading it into RAM through an intermediate parser.

Conclusion

For general web APIs and services where data size is small and code simplicity is paramount, Protobuf (and gRPC) remains a robust choice.

However, if you are building storage engines, system databases, high-frequency trading platforms, or real-time gaming backends where CPU cost is high and allocations are the enemy, migrating to Cap’n Proto or FlatBuffers is a highly effective way to unlock next-level throughput. By matching wire formats directly to CPU layouts, we eliminate the parsing tax entirely.