Open sourcing apiron: A Python package for declarative RESTful API interaction

Open sourcing apiron: A Python package for declarative RESTful API interaction

This post originally appeared&nbsp;<a href="" target="_blank">on Build Smarter</a>.

This post originally appeared on Build Smarter.

At ITHAKA our web teams write applications that each interact with a large handful of services—sometimes as many as ten. Each of those services provide multiple endpoints, each with their own set of path variables and query parameters.

Gathering data from multiple services has become a ubiquitous task for web application developers. The complexity can grow quickly: calling an API endpoint with multiple parameter sets, calling multiple API endpoints, calling multiple endpoints in multiple APIs. While the business logic can get hairy, the code to interact with those APIs doesn’t have to.

We created a module some time ago for low-level HTTP interactions, and use it throughout our code base. For a good while, though, the actual details of each service call—the service name, endpoint path, query parameters—were scattered throughout the code. This inevitably led to duplication as well as a bug or two when we made an update in one place and forgot about the other.

To reduce the pains from this, we eventually took stock of these scattered configurations and centralized them in one registry module. This module essentially contains a giant dictionary of all the services we interact with:

service_endpoints = {
        'SERVICE': 'content-service',
        'METADATA_ENDPOINT': '/content/{id}',
        'CITATION_ENDPOINT': '/citation/{citation_type}/{id}',
        'SERVICE': 'search',
        'SEARCH_ENDPOINT': '/search',
        'EXCERPTS_ENDPOINT': '/excerpt?contentId={content_id}',

Each service has a 'SERVICE' key containing the name of the service used to discover hosts, and some number of '*_ENDPOINT' keys that describe an endpoint and its parameters. Calling these services looks like this:

from http import make_get_request_with_timeout
from services.registry import service_endpoints


determine content_id...

metadata = make_get_request_with_timeout( service_name=CONTENT_SERVICE.get('SERVICE'), endpoint=METADATA_ENDPOINT.format(id=content_id), headers={'Accept': 'application/json'}, request_timeout=5, )

As you can see, there are a variety of shapes to these endpoints. This solved the issue of duplication across the codebase, but we still faced a couple of problems with this approach:

  1. Strings as endpoint descriptors don’t result in structured data. This is pretty difficult to introspect or validate.
  2. Even with fully-formattable strings, sometimes a call needed to exclude a parameter all together, or add a new one. This had to be done ad-hoc after the fact.
  3. Our HTTP module still had a laundry list of methods, each with slightly different behavior and unclear names like make_get_request_fast (how fast?). Many of these methods called the same underlying methods with different default parameters, and the stack got pretty deep sometimes. Choosing the right method for a call was hard.

In order to address the high variability of behaviors and lack of structured data of this problem, we built a new paradigm for HTTP interactions that provided a declarative interface for configuring services. We wanted a few things out of it:

  1. Code describes how a service interaction looks, not the details of how to make the underlying HTTP call happen.
  2. The endpoint descriptors are structured and support introspection.
  3. Default behaviors can be declared in the service configuration, but can also be easily overridden dynamically at call time.

With these desires in mind, we came up with apiron. With apiron the same definition from above looks more like this:

from services import IthakaDiscoverableService
from apiron.endpoint import Endpoint

class ContentService(IthakaDiscoverableService): service_name = 'content-service'

metadata = Endpoint(path='/content/{id}')
citation = Endpoint(path='/citation/{citation_type}/{id}')

And the code to call the service looks more like this:

from apiron.client import ServiceCaller, Timeout
from services import ContentService

CONTENT_SERVICE = ContentService()

determine content_id...

metadata = service=CONTENT_SERVICE, endpoint=CONTENT_SERVICE.metadata, path_kwargs={'content_id': content_id}, headers={'Accept': 'application/json'}, timeout_spec=Timeout(read_timeout=5), )

We can now define what ContentService looks like and easily refer back to that class whenever we need to understand its shape. Service discovery is now a plugin system. Endpoints can be introspected and have their parameters validated and enforced.

With apiron we’ve been able to replace many of our existing service calls quickly and with little pain. The code has become clearer and with the cognitive load out of the way we can begin focusing on other gains like streaming responses and data compression. It’s been nice for us, and we’d like to make it nice for you too.

You can install apiron from PyPI with pip (or your favorite package manager):

$ pip install apiron

By : Dane Hillard

Angular 9 Tutorial: Learn to Build a CRUD Angular App Quickly

What's new in Bootstrap 5 and when Bootstrap 5 release date?

Brave, Chrome, Firefox, Opera or Edge: Which is Better and Faster?

How to Build Progressive Web Apps (PWA) using Angular 9

What is new features in Javascript ES2020 ECMAScript 2020

How to Write Python C Extension Modules using the Python API

There are several ways in which you can extend the functionality of Python. One of these is to write your Python module in C or C++. In this tutorial, you’ll discover how to use the Python API to write Python C extension modules.

Top Python Development Companies | Hire Python Developers

After analyzing clients and market requirements, TopDevelopers has come up with the list of the best Python service providers. These top-rated Python developers are widely appreciated for their professionalism in handling diverse projects. When...

APIs for Beginners - What is an API? How to use an API?

APIs for Beginners, What exactly is an API? How do you use an API? Learn all about APIs in this full course for beginners. Learn how to use Postman and helper libraries in both JavaScript and Python. Lean how to create a project using an API using both Node.js and Flask. Learn what APIs do, why APIs exist, and the many benefits of APIs. Explore APIs online