One way that your developer team can ensure your APIs are robust and reliable is by implementing automated testing.
There are several different types of automated testing with the main ones being unit testing, integration testing, and end-to-end testing.
In this article, we’ll see why automated testing is essential for building production-ready APIs. We’ll also get an understanding of the different types of automated testing and explore the pros and cons of each. For additional strategies, check out 5 Ways to Test APIs for Breaking Changes.
What is automated testing?
Firstly, let’s ask: what is a software test? It’s a confirmation that some part of the system will produce a specific output for a given input.
The three types of testing we’ll be looking at provide three different ways of breaking down the system into parts.
Secondly, what makes a test automated? Automated tests are written in code and can be executed with a command. This is in contrast to manual tests which require a human to follow a set of instructions to perform a test.
The advantage of automated tests is that they can be run over and over again. For example, before deploying an application, or even while the developer is writing the code (a practice known as test-driven development).
Unit testing
The most common and well-known type of automated testing is unit testing. Here, we break down the system into atomic parts (typically functions, but sometimes classes).
For example, say we’re building a Node.js REST API. It has an endpoint POST /pets which creates a new pet object.
In the business logic layer, we may decide to properly capitalize a pet’s name before we add it to the database. For this, we provide a helper function properCase.
To ensure the robustness of our API, we may want to unit test this function.
// helpers.js
const properCase = (str) => {
return str.replace(/(^|\\s)\\S/g, function(t) {
return t.toUpperCase()
})
}
module.exports = { properCase }Creating unit tests
In a JavaScript project, we can use a unit testing framework like Mocha. This allows us to write our tests as a series of functions that determine whether the test passed or failed.
Consider the test below which is not too difficult to understand even without knowledge of Mocha.
First, we generate our test output by passing our test input to the subject function i.e. const output = properCase('tibbles').
We then assert that the output is what we expect i.e. expect(output).toBe('Tibbles'). This will return a boolean that determines if the test passed or failed.
// helpers.test.js
const { properCase } = require('./helpers')
describe('Helper functions', () => {
it('should put the pets name into proper case', () => {
const output = properCase('tibbles')
expect(output).toBe('Tibbles')
})
})We can now use Mocha from the command line to run this test and we’ll see if properCase works as expected or not.
mocha ./helpers.test.js
Advantages of unit testing
The advantage of unit testing is that it can pinpoint bugs. For example, if the properCase test fails we’ll know we need to fix that function. As we’ll see, other types of testing aren’t as precise.
Also, because you only need to test a single, isolated function, unit tests run very quickly.
Disadvantages of unit testing
The main disadvantage of unit testing is that it’s time-consuming to cover every function in an app with a test. Developers often say that writing unit tests takes twice as long as writing the app code itself!
Integration testing
While unit testing breaks down an application into “units” (typically functions) integration testing breaks down the application into larger pieces of functionality.
Just what these pieces are will depend on the nature of the application you’re building. In API projects, it’s common to perform integration tests on complete REST endpoints.
For more insights on building API descriptions, check out How to Build a Perfect API Description.
Unit tests tell us if individual functions work as expected, but integration tests tell us if these functions work together.
Returning to our example of a Node.js REST API, we may decide to write an integration test for another endpoint GET /pets**.**
Like all test types, integration tests assert an output for a given input. What are the inputs and outputs of this REST endpoint? We can see from the OpenAPI spec that the input is an HTTP GET request, while the output is an HTTP 200 status with an array of pet objects.
openapi: 3.0.0
info:
version: 1.0.0
title: Petstore
description: A sample API to illustrate OpenAPI concepts
paths:
/pets:
get:
description: Returns all pets
responses:
'200':
description: ok
Creating an integration test
We can again use the Mocha framework for writing integration tests. Let’s create a test for GET /pets and ensure it returns a 200 status in the response.
// pets.test.js
describe('GET /pets', () => {
it('should respond with 200 status', () => {
// test body
})
})
Since we’re working with HTTP requests, we’ll use the Supertest library which provides a neat abstraction of HTTP.
Let’s first require Supertest as well as the Node app we’ve created for our REST API.
We’ll then use Supertest to make an HTTP request to GET /pets with the code request(app).get('/pets'). We can then chain an assertion .expect(200) which asserts that the response is 200.
// pets.test.js
const request = require('supertest')
const app = require('./app')
describe('GET /pets', () => {
it('should respond with 200 status', async () => {
request(app)
.get('/pets')
.expect(200)
})
})
In more complex REST endpoint tests we can check the response body, provide POST data, auth headers, and more.
Advantages of integration testing
Integration testing aligns more closely with the required functionality of the app. Stakeholders care more that the features work rather than individual functions.
Also, integration testing covers more code than unit testing does. For example, one integration test might engage multiple functions and is therefore implicitly testing them all.
Disadvantages of integration testing
Because integration tests will typically engage the database, filesystem, and other external services, they tend to run slower than unit tests.
Not only that, but if there is a failing test, an integration test will not pinpoint it. You’ll have to use a stack trace or examine each function of the feature to identify any bugs.
End-to-end testing
Unlike unit and integration testing which test pieces of your app in isolation, end-to-end testing will test the entirety of your app.
For a website, this means testing via the user interface. For example, you could use browser automation software like Cypress to automate user actions like visiting pages, clicking buttons, and filling out forms.
This type of testing is popular because it is most aligned with the user’s interest and answers the question - does the app do what it’s supposed to?
For a REST API, you can create end-to-end tests via the REST interface. One way to do this is with Postman, which is a platform for building and testing APIs.
Alternatively, Treblle’s API testing app allows you to test APIs locally with a super-optimized native Mac app, speeding up API integrations and generating data models with AI, all without requiring login.
Creating end-to-end tests
In the following image, from Postman’s website, we see how end-to-end tests can be written within the Postman app interface under the “Tests” tab.
This tab allows you to provide a JavaScript-based test script that will run whenever you press the “Send” button.
Image credit: postman.com
How would we apply this to our Node API example?
Let’s say we deployed the API to a staging site e.g. staging.petsapi.com. This means our GET /pets endpoint would have the URL https://staging.petsapi.com/pets.
We could load that endpoint in the Postman interface and provide a test script that asserts that the endpoint provides a 200 status, for example.
// In the Postman "Tests" tab for https://staging.petsapi.com/pets
pm.test('should respond with 200 status', () => {
pm.response.to.have.status(200)
})
This is basically the same test we wrote for our integration test. But the differences are:
- We’re using Postman’s test runner and HTTP library instead of Mocha and Supertest
- We’re running the test on our deployed staging app, not in a local test environment
While the code is similar, an end-to-end test engages our full app and will therefore cover any production environment bugs.
Advantages of end-to-end testing
Since an end-to-end test engages the full app, we can find the bugs that real users will encounter.
Not only that, but even just one end-to-end test can implicitly test a lot of our code base.
Disadvantages of end-to-end testing
Since end-to-end tests run over HTTP and engage your full application stack, they are the slowest running of all tests. To run a full suite of end-to-end tests may take 10 mins or longer.
As a result of its broadness and the fact that it runs “black box”, end-to-end testing is also the most inaccurate and difficult to debug. The only error you will get is a production error message which your developers will have to decipher to pinpoint the issue.
Which type of automated testing should you use?
As we’ve seen, all three types of tests have their pros and cons. So how do we decide which to use?
The general wisdom for automated testing is that you should use a combination of all three types. This ensures that you get the advantages of each and negate the disadvantages of each.
In the following image, we see the “testing pyramid”. This diagram captures best practice thinking around automated testing.
At the top of the pyramid, we have end-to-end tests. These run slowly but are broad and cover a lot of the application. For this reason, you should write just a few of these to cover the key use cases of the app.
At the bottom, we have unit tests. You should write quite a few of these as they’re cheap to run and provide a lot of accuracy.
However, don’t worry if you don’t cover every single function in your unit tests, as any cracks can be covered by a moderate amount of integration tests which are in the middle of the pyramid.
Following this formula will ensure a good balance between test robustness and developer resources.
Treblle
If you’re looking to build a robust API for your organization, you should consider using Treblle.
Treblle is an API observability platform that can be installed in an Express app with just a few lines of code. It will monitor and trace any issues with your API so that they can be caught and fixed before affecting your uses.
In fact, the Treblle JS SDK also supports Node, Express, KoaJS, Strapi, and Cloudflare Workers! For a full list of supported platforms and integrations, visit the Treblle Integrations page.
In addition to API observation, Treblle offers your organization a variety of other useful features including:
- Auto-generated API documentation
- API quality scores
- 1 click testing
And more. Treblle is free for up 250,000 API requests and can be installed in minutes, so give it a try!
Wrap up
In this article, we’ve seen three important automated API test types - unit, integration, and end-to-end. We’ve learned their key difference including their strengths and weaknesses.
Keep in mind that this is by no means an exhaustive introduction to API testing. There are other manual test types that you can consider like behavior testing and acceptance testing.
For a deeper dive into the complexities and obstacles teams may face during API testing, be sure to read API Testing Challenges to better understand how to overcome these hurdles.
