If you've built a REST API and deployed it to production, you might think the job is done, but... does it have any testing? How do you know it's working properly right now? How do you know it'll keep working next time you push a change?
What is API testing?
Testing takes a lot of different forms in software, and APIs add another layer of interface that needs testing, but basically the idea of API testing is to make sure your API is working as expected, responding correctly for success cases, running the appropriate business logic, reporting errors correctly, rejecting malicious traffic, and keeping up with performance expectations.
There are a lot of different types of testing that aim to solve different pieces of the puzzle, and these will be done at different stages of the API lifecycle. To better understand the core principles, you can dive into the fundamentals of API testing, which lays the groundwork for testing practices.
Here’s a quick overview of 5 core types of API testing:
1. Unit Testing: Tests individual components or functions of your API in isolation to ensure they work as expected. This is often performed early in the development cycle.
2. Integration Testing: Ensures that various parts of your API work together correctly by testing their interactions. This involves testing the combined functionality of different components.
3. Functional Testing: Verifies that the API behaves as expected from the end-user’s perspective, typically by testing it with real dependencies and data.
4. Performance Testing: Measures the API’s ability to handle different loads and stress factors, helping you identify bottlenecks and optimize performance.
5. Security Testing: Focuses on identifying vulnerabilities in your API, ensuring that it can resist attacks and protect sensitive data.
Now that we’ve covered the basics, let’s dig into each type of testing in more detail. We’ll break down what each one is, why it’s important, and how it fits into the bigger picture of keeping your API running smoothly and securely.
Unit Testing
Unit testing may or may not be "API testing", but it's critical for making sure the application code for your API is working properly, so let's not get stuck on semantics.
Unit testing involves writing targeted tests for individual components, classes, and functions in the API to see if they react to good and bad inputs as expected. Maybe you have an add() function, so check that when you call add(1, 2) it returns 3. You can also check that passing in invalid values like add(1, "OH HAI") returns a NaN, or throws an exception.
Isolation is the name of the game here. You want to make sure that one component works, without it running off and calling loads of other components, and especially not other APIs. If your unit tests are sending emails and making API calls to other systems you're gonna have a really bad time.
Unit testing is generally done on the "guts" of the application code, but it can handle the API itself if the web/API framework supports it.
RSpec.describe WidgetsController, type: :controller do
describe "GET #show" do
before do
get :show, id: widget.id
end
let(:widget) { Widget.create(title: 'Star Fangled Nut') }
it "returns http success" do
expect(response).to have_http_status(:success)
end
# ... other assertions ...
end
end
"Controller Test" in Ruby on Rails
This is a "Controller Test" in Ruby on Rails using RSpec, which allows you to set up complex situations in the database, and pass all sorts of combinations to examine every single edge.
It's not actually running a HTTP server nor is it going over HTTP to make the call, but the framework is emulating that within the get :show, id: widget.id. HTTP body and headers can also be sent. Laravel has similar, as do most modern web application frameworks.
Unit tests are run frequently, by the developer as they work, and automatically run by CI/CD on pull requests.
Error Handling & Logging in Unit Testing
Proper error handling is key in unit testing. When testing individual functions or components, ensure that invalid inputs trigger appropriate exceptions and that your API generates useful error messages. You should also test that these error messages are logged correctly, making it easier to trace issues later on.
This helps ensure that your API fails gracefully when things go wrong, and logs provide enough information for debugging without overwhelming developers with unnecessary details.
Integration Testing
Integration testing verifies that different parts of your API work together correctly. Instead of messing with the codebase by mocking various other classes and methods, you leave the code to do its thing. Class A might call class B which calls function X and that sends an email!
Instead of sending a real email to a real user you might swap out the mail server for something like Mailpit, and instead of making HTTP requests to a real dependency, you might want to use record-replay tools like VCR to respond realistically but without messing with the code.
class CreateWidgetsTest < ActionDispatch::IntegrationTest
describe "POST /widgets" do
let(:token) { AuthenticationToken.create(token: 'this-is-a-good-token')}
it "will not let unauthorized users create widgets" do
params = { name: 'Star Fangled Nut' }
post '/widgets', params: params, as: :json
expect(response).to have_http_status(:unathorized)
post '/widgets', params: params, as: :json, header: { Authorization: 'invalid-token'}
expect(response).to have_http_status(:unathorized)
post '/widgets', params: params, as: :json, header: { Authorization: 'this-is-a-good-token'}
expect(response).to have_http_status(:created)
end
end
end
This is similar to the unit test in the previous example, but involves more layers of code and dependencies. Integration tests are running all the request/response HTTP middleware registered on that route, possibly hitting the network, generally getting more of the stack involved. To better understand the common challenges faced during API testing and how to overcome them, you can explore common API testing challenges.
As this is slower it's common to write drastically more unit tests to cover infinite subtle variations trying to trigger every error condition or possible output, then write a smaller number of integration tests to check that errors are handled properly and a few positive/negative outcomes work as expected.
In integration testing, error handling becomes even more critical, as multiple systems interact. Ensure that any errors occurring during these interactions are properly captured and logged.
For instance, if a dependency fails or a third-party service returns an error, your system should not only handle this gracefully by responding with meaningful error messages but also log the failure in a detailed way. This allows you to trace the error across different layers of the system and make debugging much easier.
In the API lifecycle, integration tests are typically performed after unit tests, once individual components are working correctly. They might not be run quite as frequently by developers, but they are a crucial step before deploying to staging or production.
This is also a great place to handle Contract Testing, which is the art of making sure your API responses match the expected shape. People for decades duplicated this effort by repeating their contract out in the tests, but now you can use OpenAPI-based tooling to confirm that a response matches the API description.
Functional Testing
Functional testing ensures that your API behaves as expected from an end-user perspective. It involves testing each API endpoint covering important workflows. They often sound a little similar to integration testing but this is done entirely outside of the code, with nothing faked or mocked, so you can make absolutely sure the end-user experience is working as expected.
This could be done in staging, with real emails being sent to a developer account, and real dependencies using a test credit card, or some other sort of switch to put it into "test mode", but it's real code.
Functional testing gives a clearer picture of how users will experience your API, including how errors are handled in real-world scenarios. When testing endpoints, make sure your API provides helpful, user-friendly error messages and logs critical information about the errors.
Logs from functional tests can help identify whether the right information is being captured during failures, which is crucial for investigating and resolving production issues. If you’re looking for ways to ensure your API doesn’t break after changes, take a look at 5 ways to test APIs for breaking changes.
Functional tests are typically performed after integration tests and before releasing to production, and are generally run from outside of the codebase. This could be something like JMeter, Postman, or another test runner that doesn't know anything about the internals of your code. For more insights on this, you can check out automated testing of REST APIs.
Performance Testing
Performance testing evaluates how well your API performs under various conditions, including high load and concurrent users. The goal here is to measure response times, throughput, and resource usage. It will identify bottlenecks, and help determine the API's scalability and capacity limits.
Performance tests are typically conducted after functional testing and before releasing to production. They're also important when scaling an existing API or making significant changes that might affect performance.
By tracking the performance over time you can spot "boiling frogs", where the response creeps up subtly over time, getting slower the more data that comes in.
During performance testing, it’s essential to track not only performance metrics like response times and throughput but also log any errors that occur under high load conditions. If the API starts timing out or fails to respond, those errors need to be logged with enough detail to identify the cause of the bottleneck. By capturing and analyzing these logs, you can pinpoint performance issues and address them before they become problems in production.
You can use dedicated performance testing tools like JMeter, BlazeMeter, etc, but Treblle can help you keep an eye on performance out of the box. It will give you a score out of 100, and even make helpful suggestions on things to improve. Additionally, if you’re looking for an API tool to simplify testing, Treblle's Aspen might be your next favorite.
Security Testing
Security testing aims to identify vulnerabilities in your API that could be exploited by malicious actors. This includes testing for common vulnerabilities like SQL injection, XSS, verifies proper authentication and authorization mechanisms, and helps ensure sensitive data is properly protected.
Security testing should be an ongoing process throughout the API lifecycle, but it's particularly crucial before releasing to production and periodically thereafter.
This can be done by pointing automated tools at staging or production which will poke and prod in various ways once it's deployed. Another complimentary approach is to point tools at your OpenAPI document to highlight security issues in your design, which can help avoid common problems before you go live.
Security testing requires robust error handling and logging. Any security vulnerabilities detected during testing should be logged with enough information to understand the type of attack and the response of the system. Logging these security issues provides valuable data for further analysis and patching vulnerabilities. Detailed logging also helps in auditing and tracking any suspicious activities, ensuring that your API’s security posture remains strong.
Once again, Treblle can help here too, with their API Security feature, giving you a score out of 100 and giving handy suggestions on things to improve to get that score up.
Security is always an ongoing concern and you'll never be "done", and theres no "one simple trick", but by combining a variety of these approaches, and occasionally hiring an expert to see if they can break things, you should be ok.
Bonus: Testing in CI/CD Pipelines
Adding testing to your CI/CD pipeline is a game-changer for catching issues early and ensuring smooth deployments. Here’s how you can do it, with a little help from Treblle:
1. Automate Unit and Integration Tests: Every time code is pushed or a pull request is made, your unit and integration tests should run automatically. This gives fast feedback and keeps broken code from sneaking in.
2. Functional Testing in Staging: Once everything passes, deploy to staging and run functional tests. Using appropriate testing tools, you can ensure your API behaves as expected in real-world scenarios.
3. Scheduled Performance Testing: Running performance tests on every commit isn’t practical, but you should definitely schedule them before big releases. Tools like Treblle can help by giving you real-time insights into response times and error rates, so you know how your API is holding up.
4. Security Testing: Before going live, automate security scans with something like OWASP ZAP to catch vulnerabilities. Treblle can also lend a hand by providing security scores and tips on how to tighten things up.
5. Feedback and Reporting: Make sure you’re getting test results instantly. With Treblle, you get real-time monitoring and error tracking across your environments, so you always know what’s going on with your API.
Conclusion
Each type of testing plays a vital role in ensuring the quality, reliability, and security of your REST API. By incorporating these testing methods throughout your API lifecycle, you can catch issues early, improve the overall quality of your API, and provide a better experience for your users.
You'll never be done with testing. It should be part of the workflow, and involved at every stage of the lifecycle. Some of it may be automated testing, some may be manual, but as your API evolves, so should your testing strategies to ensure continued reliability and performance.
