Data Exchange Formats: From JSON to Protobuf

Moving data between systems is the foundation of any integration. And the moment you have more than one service, a question pops up that a lot of people skip on autopilot: what language will these services use to talk to each other? Let's go through it in order — from the familiar text formats to the binary ones you reach for when text can no longer carry the load.

Why the format is an architectural decision

Choosing a data format is often treated as a triviality: "it's JSON, obviously, what's there to think about." That's a mistake. The format directly affects performance, scalability, reliability, and — something people frequently forget — how easy things are to debug. The same data stream in text form is readable by eye and debugs beautifully, but takes up more space and parses slower. In binary it's the opposite: compact and fast, but without special tooling you can't even peek inside that stream.

So "which format should I pick" isn't a matter of taste — it's a matter of requirements. Do you care more about development speed or throughput? Strict validation or flexibility? The answers are what determine what you build your integration on. And, as a bonus, it's a great way to gauge someone's architectural maturity in an interview: ask how Protobuf differs from JSON and when it's justified. The answer will tell you more about the person than the list of frameworks on their résumé.

The text trio: JSON, XML, YAML

Let's start with the ones humans can read.

JSON — the workhorse of the web. The main format for REST and GraphQL: lightweight, fast to process, equally understandable to humans and machines. Simple syntax plus excellent support in practically every programming language — that's why it became the default. If you're not sure where to start an integration, start with JSON and you won't go wrong nine times out of ten.

XML — the heavyweight veteran that never went anywhere. It lives in SOAP services and still turns up in enterprise REST setups where strict structure and validation are required. XML's main trump card is schemas (XSD): you can describe up front exactly how the data must look and reject anything that doesn't conform. For complex interactions in banks and enterprises, that's not bureaucracy — it's insurance.

YAML — a format you rarely see inside the API requests themselves, but it's everywhere around them. API descriptions (OpenAPI/Swagger), configs, everything DevOps-related — that's YAML. People love it because it's as human-friendly as it gets: reading and hand-editing it is more pleasant than anything else on this list. You won't push data over the wire with it, but for describing a system it's just right.

Which format goes with which API

A format is rarely chosen in a vacuum — more often it's dictated by the type of API you're working with.

• REST API — mostly JSON, occasionally XML. REST is valued for flexibility and simplicity, and JSON fits that philosophy perfectly.
• SOAP API — XML only. Strict validation and a high degree of structural control make it a frequent choice where the cost of a mistake is high: banking, insurance, enterprise integrations.
• GraphQL API — JSON. The trick here is in the queries themselves: the client grabs exactly the fields it needs, with no extra ballast.
• WebSocket API — usually JSON, but when performance and message size matter, binary formats like MessagePack come into play.

Message brokers: the heartbeat of distributed systems

Message brokers are a story of their own. In the world of microservices they work like a circulatory system, and here the data format hits performance and reliability directly.

Apache Kafka — the leader for big data and streaming: events, logs, service-to-service exchange. Kafka is home to a whole zoo of formats: JSON for simplicity, Avro for compactness and schema-evolution support, Protobuf when you need binary speed, MessagePack as a compromise between size and structure.

RabbitMQ — the all-rounder for tasks where delivery reliability is critical: queues, background jobs, microservices. Most often JSON travels inside; XML shows up in legacy setups; and Protobuf gets plugged in when you need to shrink messages and speed up the exchange.

Apache ActiveMQ and its modern reincarnation — a frequent guest in enterprise systems with a zoo of protocols and high reliability requirements. JSON and XML are the usual suspects here, and in tandem with document stores you'll sometimes see BSON surface.

NATS — a lightweight, fast broker popular in microservices and IoT. JSON by default, but when you need to shave off latency and message size, people reach for Protobuf and MessagePack.

The pattern is simple: the closer you get to heavy load, the more often text gives way to binary formats.

When text becomes expensive

JSON and XML are wonderful as long as there isn't much data and the priorities are development speed and easy debugging. But text formats carry an unavoidable tax: they're verbose. Every field name, every quote and bracket is bytes you push across the network and parse on each end. At a stream of millions of messages, that tax turns into real money and real latency.

This is where binary formats step onto the stage. Their idea is simple: don't drag the structure along in every message — describe it once with a schema and transmit only the data itself, packed as tightly as possible. Let's go through the three main ones.

Protobuf — the de facto standard for gRPC

Protocol Buffers by Google is a binary serialization format where you describe the data structure with a schema, and code is generated from it for several languages at once. It's more compact and faster than JSON and XML, plus it gives you strict typing — that is, a guarantee that the data matches the defined structure and fewer chances to hit an error out of nowhere.

The schema looks like this:

syntax = "proto3";

message Message {
  string sender = 1;
  string text = 2;
}

Where Protobuf especially shines:

1. High-load systems and microservices — fast serialization and deserialization reduce latency and traffic volume.
2. gRPC — Protobuf is its native format. If you work with gRPC, you inevitably work with Protobuf.
3. Complex data structures — nested objects, arrays, and different types are described and validated through the schema.

The main upsides: compactness and performance, strict typing as protection against malformed data, and code generation support for a pile of languages. For anyone designing high-load architectures, knowing Protobuf isn't an option — it's a baseline tool.

Avro — the king of schema evolution

Avro was born inside the Apache Hadoop ecosystem, but it quickly moved beyond big data. It's also a compact binary format with schemas, but it has a superpower that's exactly why people love it.

An Avro message schema:

{
  "type": "record",
  "name": "Message",
  "fields": [
    { "name": "sender", "type": "string" },
    { "name": "text", "type": "string" }
  ]
}

When Avro fires on all cylinders:

1. Big data and streaming — Kafka, Flink, and similar platforms where the event stream is enormous and the format has to be lightweight and fast to read.
2. Complex structures — nested records and arrays, not just plain "key-value."
3. Schema evolution — here it is, the main thing. You can add new fields and change the data structure without breaking old records or rewriting everything you've already accumulated. For large systems, where the schema inevitably changes over time, this saves you from a world of pain.

The bottom line on Avro: compactness versus text formats, fast serialization under load, and schema flexibility without stopping the system. If you work with data streams, Avro is definitely worth learning.

MessagePack — "JSON, just compact"

If JSON feels bulky and Protobuf, with its schemas and codegen, feels like overkill, there's a middle ground. MessagePack aims to be as simple as JSON but packs the data into dense binary messages. As a bonus it supports more types — for example, binary strings and extension types — which makes it more versatile.

The idea in plain terms: you take your usual JSON…

{
  "sender": "Vladimir",
  "text": "This is a MessagePack example."
}

…and after packing it into MessagePack, the same data takes up less space and flies faster. Where it's good:

1. High-load systems and microservices — where performance and minimal latency matter.
2. WebSocket and real time — real-time exchange with less strain on the network.
3. Resource-constrained systems — IoT and mobile apps, where every byte counts.

MessagePack's main advantage is that it slots easily into places that already use JSON: the integration is minimal, and the data exchange speeds up almost for free.

How to choose

There's no single correct format — there's the one that fits the task. The logic goes roughly like this:

• Need simplicity, development speed, and easy debugging — go with JSON (and for configs and descriptions — YAML).
• Need strict structure and validation in an enterprise setup — XML with schemas.
• Load ramped up, compactness and low latency matter — switch to binary: Protobuf for gRPC and microservices, Avro for streaming and schema evolution, MessagePack where you just want to speed up your existing JSON.

Text formats are convenient for humans but lose out on efficiency. Binary ones win on performance but require schemas and a bit more discipline up front. It's a classic trade-off, and making it deliberately is part of systems design. The better you understand these formats, the easier it is to assemble solutions that work the way they should — not the way everyone else's do.

---

Originally published on my Telegram channel @it_underside.

Yours, DPUPP