Using APIs.json to Create a Catalog of APIs
Using APIs.json to Create a Catalog of APIs
What is missing for APIs.json to take off as the standard for publishing API catalogs?
One of the greatest things about building technology publicly is how anyone can follow what you're doing. That's exactly what's been happening with the APIs.json specification and the api-catalog IETF draft. Are these two projects overlapping? Or, can they complement each other?
This article is brought to you with the help of our supporter: Speakeasy.
Speakeasy provides you with the tools to craft truly developer-friendly integration experiences for your APIs: idiomatic, strongly typed, lightweight & customizable SDKs in 8+ languages, Terraform providers & always-in-sync docs. Increase API user adoption with friction-free integrations.
APIs.json is a project started by Kin Lane and Steven Willmott in May 2014. Its goal is to create a file format that can represent a collection of APIs. Each APIs.json document can hold information about multiple APIs. And it's not just technical information, though. Each API definition on an APIs.json collection can hold any properties. This leaves room for attempting to use the format to describe things such as the team responsible for the API or a way to reach support.
APIs.json has evolved since 2014 and other people have contributed to the project—myself included. In August 2023, APIs.io was born as a natural evolution of what APIs.json alone could offer. If, with APIs.json, you can describe a collection of APIs, APIs.io indexes any known collections and a search engine that anyone can use.
All this work has been delivered without any existing standards behind. The authors of APIs.json thought of first experimenting with the format before attempting to submit RFCs. The specification even alludes to that in sections 3.5, 3.7, and 3.8. According to the specification, the goal is to, "if there is enough traction," to submit the "application/apis+json" and "application/x-yaml" media types, the "well-known" location "/apis.json" or "/apis.yaml," and the link relation "api."
It's precisely these last two items that the api-catalog IETF draft describes. Instead of waiting for "enough traction," the author of the draft just published what he thought would be interesting. Kevin Smith, currently working for Vodafone, published the first version of the draft in September 2023 and evolved it until March 2024. It's unclear to me if the author involved Kin Lane in the process. However, the draft references among other documents, APIs.json specification version 0.16 from September 2020.
The main goal of the api-catalog IETF draft is to "facilitate the automated discovery of a Publisher's public API endpoints, along with metadata that describes the purpose and usage of each API." To achieve that it proposes that API producers publish an API catalog document on a "well-known" URI. It doesn't, in any way mandate what the format of the API catalog should be, or even the name of the API catalog document.
The IETF draft does, however, recommend that the API catalog uses a linkset of endpoints to describe how the APIs can be accessed. The draft also recommends the inclusion of what the author calls "useful metadata," namely API usage policy information, and links to OpenAPI documents defining each one of the mentioned APIs. The draft goes further and mentions a few document formats, other than linkset, that can be used with the api-catalog URI. Among those formats is—yes, you guessed it right—APIs.json.
So, what I understand from reading both the APIs.json specification and the api-catalog IETF draft is that they can complement each other. While the focus of APIs.json is on the format for describing a collection of APIs, the api-catalog draft focuses on defining where consumers can find it. Could Kin Lane and Kevin Smith collaborate to enrich both approaches and produce and better-fitted specification? I believe they could. It's unclear to me why they're moving forward separately.
In any case, I see a lot of potential in services and tooling that can exist to help produce, maintain, and consume APIs.json documents and keep them available in a "well-known" location. To me, APIs.io is more of a proof-of-concept showing what can be possible rather than the only way to index information about APIs. I imagine a world where API documentation tooling can also generate APIs.json documents and deploy them along with the reference and other assets. It's not hard to picture a scenario where services such as an API gateway can help producers maintain their API catalogs.
From a consumer perspective, I think things will take longer to gain traction. What consumers want is an easy way to discover the APIs they need and also to discover interesting information about APIs they already know. Offering a service that can automatically index API catalogs and offer a search interface to the gathered information is not hard. Just look at APIs.io. What seems to be hard is obtaining a large number of API catalogs to index. And that's the only way to scale such a service. You can't go on adding APIs.json documents by hand forever. It simply doesn't scale.
What's missing is an incentive to make API producers want to publish their API catalogs in a "well-known" location. The incentive doesn't have to be of a financial nature. Just look at how robots.txt has been growing in usage since its inception. Its continued adoption is a good example of a simple, yet effective incentive. Website maintainers want to use robots.txt to advertise to crawlers—including popular search engines—about their content and even exclude some content from being indexed. Using robots.txt has become an agreed approach to engage in a contract that people building Web crawlers tend to respect.
What we need, in my opinion, is a similar effect for APIs.json. We need APIs.json to carry a value that's as strong as how robots.txt is perceived. We need it to be an entry point into all the API-related information published by a producer. We need it to be the primary way to find information about the APIs behind a domain. I think we can attempt to achieve traction by making APIs.json the default approach for interacting with APIs on an already popular consumer tool.
Imagine if a popular IDE such as VSCode would automatically load all API information just by knowing the domain you wanted to interact with. It could then present all the known APIs for the domain along with any relevant information. Opening one of the APIs would access its machine-readable definition and could automatically generate both human-readable documentation and code.
Altogether I believe we're just witnessing the beginning of what fully automated API discovery can look like. The future of API discovery is in the hands of API producers and tooling vendors. Together they can change the ability consumers have to discover, learn about, and interact with APIs.