API Versioning In Python

As we found out in Steve's Article API Versioning is used to introduce changes to our API.

So before we begin, let’s dive into what exactly is API Versioning. It is an approach we as Software Engineers will use to introduce changes to our API, it is as simple as that. We may want to change how the data is structured, or how functionality works, or add extra authorization layers to increase security in areas that were overlooked. There are other approaches we can take to do some of these tasks, such as Role Based Access Control or Feature Flags. However, usually we would want to use these alongside Versioning of our API to get the fine-grained control we desire for our application.

In this article - we're going to deep dive into the preparatory work of How API Versioning can be set up in Python to reduce code overhead, as well as allowing full flexibility.

The Aspects We Will Cover

No matter what type of API versioning we use in Python, we'll want to consider several different aspects of how we handle this in our codebases. We could be changing many different areas of code between versions:

• Request Structure - What your user sends to you
• Response Structure - What your user receives from you
• Business Logic - What your API actually does
• New Endpoints - Something new your user can call
• Remove Endpoints - Taking Something away your user can no longer call
• Deprecations / Sunsetting - Adding warnings to things

And possibly even more than that - but what do we now now? we have a LOT to deal with here.

Project Structure

Project Structuring could look something like the below

root/
│
├── app/
│   │
│   ├── v1/
│   │   │
│   │   ├── endpoints/
│   │   │
│   │   └── models/
│   │
│   ├── v2/
│   │   │
│   │   ├── endpoints/
│   │   │
│   │   └── models/
│   │
│   ├── endpoints/
│   │
│   └── models/
│
└── config/

Now that probably looks scary, right? lots and lots of directories all nested deeply, with some confusing structure - so let's delve in a bit:

root/: This is the root directory of your entire project.

app/: The directory where the core of your FastAPI application resides. It contains the following:

v1/ and v2/: These directories represent different versions of your API. You can organize your API by version to maintain backward compatibility as your API evolves.

endpoints/: This directory holds the API endpoints for each version. Each endpoint can be defined in a separate Python file for better organization.

models/: This directory contains Pydantic models specific to each version. These models define the data structures used in your API.

config/: This directory is used for configuration settings and is typically not version-specific. It contains the following:

A lot less scary now right? So why are we structuring it like this? Well quite simply, we're building the foundation of the tree we want to use later. This tree really does help us to find what we want - in the version we want it - fast.

Imagine, for a moment, a customer raises a bug - "hey, in version 3 of the API, when I submit a request to update a post, I'm getting an issue where it tells me that the field "name" is missing - could you tell me what's going on?"

So that customer raising that bug has given us the tree to go to:

• It's in Version 3
• Its to do with a model
• Its the Request Model for updating Posts

So we can quickly traverse our tree /app/v3/models/UpdatePostModel.py and we can start examining there. Yes, of course other structures also work, this is my personal preference - but others may differ and even have better ideas! But the core principal is make it as easy as possible to find the bit of code you want.

Fetching The Right Model for Request And Response

If we look back at our first point of change earlier, it was the Request Structure, and the response structure - so how do we handle that when we can have one of three options:

• It exists in an API Version directory
• It exists in the base App directory
• It doesn't exist at all

Well, that's where we start with some helper functionality - we're going to have to come up with some way of getting the right model for the right version at the right time.

What I like to do is have a get_model_for_version function in python - it takes 2 parameters:

• a string for the model
• a string for the version

If it can't find it in the specified version, it tries the base directory

If it cant find it in the base directory it throws a custom exception

But here's the cool bit! not only does it work for getting the entity, but also for type hinting!

This is a fairly generic example, yet it's important to note that our discussion extends to Django API versioning, providing insights on how to adeptly handle version control for multiple different frameworks in Python.

from typing import Type, Union, TypeVar, Optional
from importlib import import_module


class ModelNotFound(Exception):
    pass


T = TypeVar('T')


def get_model_version(api_version: str, model_name: str) -> Union[Type[T], None]:
    try:
        model_module = import_module(f'app.{api_version}.models.{model_name}')
        model_class = getattr(model_module, model_name)
        return model_class
    except ImportError:
        try:
            model_module = import_module(f'app.models.{model_name}')
            model_class = getattr(model_module, model_name)
            return model_class
        except ImportError:
            raise ModelNotFound(f"Model '{model_name}' not found in API version '{api_version}' or base version.")

Business Logic

So now we get into some real fun here - what if the business logic needs to be different per version? Enter, Strategy Pattern:

Imagine you have a big box of colorful building blocks at home. These blocks are not just any blocks; they are magic blocks because they can do different things when you put them together. Some blocks are like puzzle pieces that fit together perfectly, some blocks have little wheels that let them roll, and some blocks have magnets that attract each other.

Now, you have a special plan to build different things with these blocks. Sometimes, you want to build a tall tower, like a castle, and you need the puzzle blocks that fit together. Other times, you want to make a car that can move, so you need the blocks with wheels. And when you're building a house, you need blocks with magnets to stick together.

Here's where the Strategy Pattern comes in. Instead of keeping all the blocks mixed up, you organize them into different groups. You put all the puzzle blocks in one box, all the wheeled blocks in another, and the magnetic blocks in a separate box.

Now, when you want to build something, you pick the right box of blocks. If you're building a castle, you grab the puzzle block box. If it's a car, you take the wheeled block box. And for a house, you use the magnetic block box.

The Strategy Pattern is like having these special boxes for your building blocks. It helps you choose the right set of blocks for what you want to build without getting confused. Plus, if you want to build something new that needs a different kind of block, you can always make a new box for those blocks without messing up the others.

So, the Strategy Pattern is like a smart way to use your magic building blocks, making it easy to build all sorts of cool things without any mix-up. It's like having a plan and the right tools for the job!

What, then, does that have to do with API Versioning? well, our business logic can be encapsulated into smaller strategies - that we can call in the endpoints for each version in the right way.

Imagine a loan system, for example. Our API accepts PayPal, CashApp or Credit Card payments. They're all payments, and all about paying it - but they're different methods of paying. You switch out your payment strategy, and you get a whole different approach - just by switching out one thing. The same can apply in your API Business Logic: Switch out the strategy!

Here's an example of the strategy pattern for Python, but again, we're delving into the depths of this more in our other article.

from typing import Protocol


class PaymentStrategy(Protocol):
    def pay(self, amount: float) -> None:
        pass


class CreditCardPayment(PaymentStrategy):
    def __init__(self, card_number: str, expiration_date: str):
        self.card_number = card_number
        self.expiration_date = expiration_date

    def pay(self, amount: float) -> None:
        print(f"Paying ${amount} with credit card {self.card_number}.")


class PayPalPayment(PaymentStrategy):
    def __init__(self, email: str):
        self.email = email

    def pay(self, amount: float) -> None:
        print(f"Paying ${amount} with PayPal account {self.email}.")


class ShoppingCart:
    def __init__(self, payment_strategy: PaymentStrategy):
        self.payment_strategy = payment_strategy

    def checkout(self, total_amount: float):
        self.payment_strategy.pay(total_amount)


# Example usage:

if __name__ == "__main__":
    credit_card_strategy = CreditCardPayment("1234-5678-9876-5432", "12/25")
    paypal_strategy = PayPalPayment("[email protected]")

    cart1 = ShoppingCart(credit_card_strategy)
    cart2 = ShoppingCart(paypal_strategy)

    cart1.checkout(100.0)
    cart2.checkout(50.0)

It's worth noting that this is a contrived example, specifically for showing how the pattern works - in the real world, I would avoid holding the credit card details myself!

Adding A New Endpoint

Imagine, if you will, that Version 2 of your API adds in an entirely new route - never seen in version 1. Your API used to have /hello, /hello/{name} in version 1 - but now you want /goodbye as a new route on version 2. This is actually a common use case - and with our individual routes, and routers - Add a new endpoint function in your v2 endpoints directory, and hook it to the router for that particular version of the API.

But how does that work when you have 20 versions? do you have to copy the file in to each directory? No, friends - that's the beauty of this system! Your version 2 router, ALSO uses your version 1 routes, and your base routes. It cascades down. How do we implement that? Our other article "Framework Differences" explains the intricacies of how - but essentially, it's a case of order of linking them. The next router takes precedence over the first.

Removing An Endpoint

Removing an endpoint becomes very similar to adding one - you're effectively creating a new handler for your route in the version that you want to remove it that returns a 404. It sounds complicated, but you only need to do it once - any future version, that route is removed.

Deprecating Or Sunsetting An Endpoint

My personal preference for this is to use Header Deprecation - let's say in version 3 of our hello API, we want to remove /hello/{name} from our routing:

The steps I take are:

• in version 1 the Route exists as normal
• in version 2 the Route now has a middleware attached that adds x-deprecated to the headers
• in version 3 you create your 404 route.

Pretty easy to understand flow, right?

When To Remove Versions Entirely

In my experience, removing a version of an API should be done in line with your customer's use cases - generally, I like to maintain at least 2 major versions of any API - so in version 3, I'd remove version 1 and move all the functionality we are keeping up a level into version 2.

Your personal use case may require more support than that - but the principal applies. You add a deprecation middleware to all routes that adds x-deprecated-version to the headers, as well as clearly documenting that.

So, with this in place, we now handle all our requirements:

• Request Structure - What your user sends to you
• Response Structure - What your user receives from you
• Business Logic - What your API actually does
• New Endpoints - Something new your user can call
• Remove Endpoints - Taking Something away your user can no longer call
• Deprecations / Sunsetting - Adding warnings to things

And we briefly touched on deprecating an entire version - does that wrap everything up in a nice little package? not really.

Talking To Your Users

Your API users are the most important people when it comes to your API - the more you talk to your users, the more you'll learn about what works for them and what doesn't. Here's the list of questions I try to get answers from my users about:

• Is the API easy for the developers to integrate with?
• Can you predict the responses of your API?
• Can you predict the routes of your API?
• Is there anything you wish the API did that it doesn't?
• Is there anything you wish the API didn't do that it does?

This sort of feedback really helps when designing versioning in your API, as the users can help you form ideas and concepts for future versions, as well as help you identify bugs you don't know about, or developer pain points you don't know about.

Remember - effective API versioning hinges on solid code architecture and robust management practices. It's also crucial to maintain comprehensive API versioning documentation. While it's essential to communicate with your users and provide support for at least two versions during their integration period, never underestimate the power of well-crafted documentation in guiding them through the transition.

API versioning is a crucial aspect of software engineering that allows us to introduce changes to our APIs while maintaining backward compatibility. A well-structured project, like the one discussed, with organized directories for different versions, endpoints, models, and configurations, serves as a solid foundation. Implementing strategies, such as the Strategy Pattern, for handling business logic variations between versions ensures flexibility. Adding new endpoints and gracefully removing or deprecating old ones is made straightforward by cascading routing. Ultimately, successful API versioning goes beyond code; it involves engaging with users to understand their needs, predict responses, and shape future versions. Effective versioning, good code structure, and user feedback together form the pillars of a robust API development strategy.