They say that third-party APIs are a black box. In other words, it's not always easy to know what is happening behind the scenes in the backend code. And as more and more development teams come to rely on APIs, the need to validate the stability of these interfaces has never been more pressing. So much so that OWASP now includes Unsafe Consumption of APIs in their top ten list for APIs.
So, prospective API consumers need some way to verify the reliability and safety of third-party APIs, hopefully *before* committing to a shaky integration. Simultaneously, there is an onus on API providers to ensure their APIs are highly performant, follow industry best practices, and are devoid of any security issues. One way to expose issues with an API is to scan its OpenAPI definition file to flag potential problems.
Enter API Insights. API Insights is one of the best tools for quickly analyzing OpenAPI files. You can upload an OpenAPI file in YAML or JSON or point it to a link where one is hosted, and the tool instantly responds with an overall health check and detailed findings. And, what's best — it's free to use.
Below, we'll explore using API Insights to analyze OpenAPI files from popular public REST APIs like Stripe, GitHub, and WhatsApp. We'll zone in on the three key areas (security, performance, and overall design) and consider how schema scanning can help API development at large.
Evaluating Stripe's API Design
First, let's examine the design of the OpenAPI 3.0 specification for the public Stripe API. API Insights rates its design as a D, with a score of 65.
Stripe's API design was nicked since it failed a couple of tests around naming conventions. The API doesn't use consistent pluralization semantics for endpoints and doesn't always use nouns, both of which are considered best practices for designing REST APIs.
Furthermore, the test caught a potential flaw regarding rate limiting — API Insights recognizes the file doesn't specify a `x-ratelimit-*` header in the response payload. Proper rate limiting is crucial to have in place to avoid denial of service attacks and to track developer consumer usage. Now, to be fair, this flag doesn't necessarily mean an API isn't using rate limiting at all in production, but since it's unmentioned in the OpenAPI file, it's worth looking into.
That low score may seem a little harsh for such a well-loved developer API. So, it's good to note that the Stripe API design is doing a lot of things right! For instance, the API uses HTTP methods correctly and does not just throw everything into a single GET method. It's handling versioning well and specifying the proper content types. It also provides human-readable details around common errors, which is a nice perk and more helpful than just seeing 200 OK or 504 Gateway Timeout Error when something goes wrong.
Gauging GitHub's API Security
Cybersecurity is tough to get right. And when it comes to APIs, there is a heap of security concerns to keep in mind, from access control issues to unrestricted resource consumption, shadow APIs, business logic flaws, and more.
Thankfully, GitHub's API does pretty well concerning security. For this test, I pointed API Insights to a URL for the raw api.github.com.json file hosted in this GitHub repository. Its 925 endpoints check most of the boxes, and overall, the API's security is graded a B with a score of 86.
The API hits the marks in a number of ways. For example, it doesn't allow iFrame embedding, decreasing the risk of clickjacking attacks. Both the content-security-policy and x-content-type-options headers exist, restricting what type of resources and content the API works with. Furthermore, the API utilizes encryption so we can be sure that secure communication is enforced.
However, the GitHub API specification was dinged because the securitySchemes object is empty for one or more methods. This object is important as it denotes what security types, like apiKey or oauth2, are necessary, as well as the type of flows and scopes required for authorization.
Testing WhatsApp's API Performance
Ok, now that we've seen how we can spot API design and security flaws within OpenAPI specifications, let's look into another specification to see how API Insights calculates performance insights. For this test, I uploaded the WhatsApp Business API's v3.0.0 YAML OpenAPI file.
To calculate performance metrics, API Insights actually goes beyond the specification to interrogate the API itself to get insights. As it turns out, performance is an area where many APIs seem to be struggling, and WhatsApp is no exception. API Insights scanned all 55 endpoints and calculated an F grade with a score of 36.
According to the results, the WhatsApp API appears to have a quick load time, but its HTTP server does not use HTTP2, meaning it's missing out on some significant performance enhancements. The API also has no content-encoding information, indicating that payloads are not being compressed over the wire. There is also a lack of caching support, and it appears no CDN is being used to route and deliver requests efficiently.
As evidence, a separate test for a Twilio API component found very similar performance results. It appears there are many performance enhancements that API providers could benefit from, but according to their OpenAPI specifications, they are going underutilized.
General Impressions
As you can see, scanning OpenAPI schemas is a constructive way to quickly diagnose the quality of APIs, bringing a number of potential benefits to both API providers and users. For API providers, following some of the design suggestions from API Insights could help you build more performant and consistent interfaces that follow best practices. Because as we know, consistency is a positive force for improving API developer experience, which is becoming more of a competitive advantage.
On the user side, this analysis empowers developers with human-readable information about what the general developer experience is like using an API, informing their prospective buying decisions. What's nice is that API Insights generates a temporary unique URL for every definition it analyzes, meaning you can bookmark the results or share them with your colleagues.
One drawback is that API Insights is not as thorough as other open-source OpenAPI linters that enable linting via custom rulesets. Furthermore, it would be helpful to see more in-depth responses highlighting the precise areas in the specification it is flagging. (For example, it would be helpful to know which exact endpoint uses inconsistent pluralization out of the 494 Stripe API endpoints scanned). That said, for a free tool, API Insights is a simple and effective way to quickly compare an API against industry benchmarks.
Final Thoughts
In recent years, OpenAPI Specification, formerly known as Swagger, has become a standard format for defining and sharing RESTful API definitions. OpenAPI files are usually pretty detailed and can help generate documentation, code libraries, sample code, and even sandboxes.
Of course, OpenAPI file analysis can only go so far. The problem is that not all software providers are design-first or use the API specification as a source of truth. Therefore, there is sometimes a mismatch between documentation and production behaviors.
As evidenced above, some API developers simply leave certain fields empty, leaving other developers guessing about whether features are supported or not. These are areas where the industry is actively working to improve, but they reflect the reality of modern API development. It also underscores why interrogating the endpoints in a live environment is essential for performance and reliability testing.
Therefore, OpenAPI specification analysis should be considered as part of a larger API management strategy. It's a great way to quickly inform business and technical stakeholders and validate that APIs meet industry benchmarks. And if an API scores well on API Insights, it'll most likely be a stable and pleasant integration in practice, too.
