How to Document Event-Driven API Message Payloads

Documenting event-driven payloads is necessary to promote healthy consumption.

Being familiar with OpenAPI-generated reference documentation is one thing. Knowing how to define—and document—event-driven APIs is something else. The REST mindset doesn't apply to event-driven design. And then you have to define the messages that subscribers will consume. What are the tools and processes that can help you?

---

This article is brought to you with the help of our supporter, Scalar.

Scalar is a suite of powerful & customizable developer tools to help you at all stages of API development. Create beautiful API Documentation in a notion-like editing experience.

---

Using OpenAPI means you have a range of options for automatically building documentation. From open-source tools to ready-to-use SaaS products, you can surely find something that fits your needs. But, with OpenAPI all you can do is define and document RESTful APIs. You work your way by defining the API HTTP endpoints, their parameters, and request and response payloads.

The nature of the APIs you can define using OpenAPI is mostly synchronous. In other words, these are APIs running over HTTP that let consumers make requests and wait for the server to provide a response. Consumers are active in the sense they need to initiate a connection for an API request to happen.

On the other hand, event-driven APIs are asynchronous by nature. Every time an interesting event takes place a notification is triggered on a specific topic. Any consumer interested in a topic can subscribe to it and receive notifications—or messages—every time there's a new event. Here consumers are passive meaning they're not the ones initiating the communication. Instead, consumers are waiting for a new event to be dropped on a specific topic they're listening to.

So, how do you document this kind of API and what are the things that matter the most? Luckily there's a specification similar to OpenAPI but directed at defining event-driven APIs. I'm talking about AsyncAPI. It's a specification that lets you define an API that's asynchronous in nature. By using AsyncAPI you can define, among other things, the different topics where events will be dropped, and the shape of the messages that represent each topic.

And this is where things get interesting. The shape of messages or, in other words, its payload, can adhere to specific standards. Without messages, there's no way to communicate events. And, following standards helps to guarantee the correct publishing, transport, and consumption of messages. If messages don't follow any standards, it's hard for developers to understand the shape of the messages. In addition, it's easy for consumers to stop working because, suddenly, messages are being shared in a slightly different format.

Among different message standards, there's one particularly interesting to me. Apache Avro isn't just a theoretical standard. It's also a serialization format. You can say it's a competitor to JSON but specialized in working with event-driven APIs. In the same way you can use JSON Schema to define the shape of JSON data you have Avro Schema to help you specify what Avro message payloads look like.

{
  "type": "record",
  "name": "Person",
  "namespace": "com.example.avro",
  "doc": "An example Avro Schema of a person.",
  "fields": [
    {
      "name": "Name",
      "type": "string",
      "doc": "The person's full name."
    },
    {
      "name": "Age",
      "type": "int",
      "doc": "The person's age."
    }
  ]
}

As you can see from the example, using an Avro Schema document is not that different than using JSON Schema. So why can't you use JSON Schema to define Avro payloads? The main reason is the Avro payloads are binary while JSON isn't. The existing tooling uses Avro Schema to define, document, and validate Avro messages. Even though these are different worlds you can convert Avro Schema documents to JSON Schema and back using available open-source tools.

Something else you can do with existing tools is generate documentation for your Avro Schema definitions. You can generate documentation independently or as part of your whole event-driven API definition. Generating independent documentation is useful when you're reusing schemas and want to give developers an easy way to understand them individually. I went looking for a Node.js tool that would let me generate documentation for the Avro Schema I shared above. Unfortunately, the best I found was avrodoc, which hasn't been active since 2013. After a few tries with some of its forks and some adaptations of my own I was able to generate a nice-looking Avro Schema documentation.

Another option is to generate the documentation for the Avro payload inside your API reference. One possible way to do that is to use AsyncAPI to generate the whole reference, including the schema documentation. AsyncAPI supports the inclusion of Avro Schema definition documents. According to AsyncAPI's documentation you can Avro 1.9.0 by specifying it as a message payload. To do that you use the schemaFormat attribute with the value application/vnd.apache.avro+json;version=1.9.0.

asyncapi: 3.0.0
info:
  title: Person Service
  version: 1.0.0
  description: Register new people into the system. 
channels:
  personRegistered:
    address: person/registered
    messages:
      PersonRegistered:
        $ref: '#/components/messages/PersonRegistered'
operations:
  sendPersonRegistered:
    action: send
    channel:
      $ref: '#/channels/personRegistered'
    messages:
      - $ref: '#/channels/personRegistered/messages/PersonRegistered'
components:
  messages:
    PersonRegistered:
      payload:
        schemaFormat: 'application/vnd.apache.avro+json;version=1.9.0'
        schema:
          {
            "type": "record",
            "name": "Person",
            "namespace": "com.example.avro",
            "doc": "An example Avro Schema of a person.",
            "fields": [
              {
                "name": "Name",
                "type": "string",
                "doc": "The person's full name."
              },
              {
                "name": "Age",
                "type": "int",
                "doc": "The person's age."
              }
            ]
          }

It's interesting to see that while the API definition is written using YAML, the embedded Avro Schema uses JSON. You could easily change its format by modifying the schemaFormat attribute, though. Let's see how rendering this reference using the AsyncAPI Studio looks like in a browser.

In summary, you can safely use Avro Schemas to define your event-driven API message payloads. With the solutions I presented you can have independent and embedded schema documentation. The advantages of defining and documenting your message payload are bigger than the difficulties in generating good-looking documentation.

---