Connecting with the outside world: API Best practices
As a developer, there is a moment when you have to face an integration with an outside service (JIRA, Toggl, Slack, Github, WordPress, and a million more). It may be done via REST API, GraphQL, SOAP, or the protocol that the developers behind the service are using. This post is not about how to use a specific protocol but about some recommendations that I have found necessary to keep in mind once you are connecting with one or several services in your application and that will make your life easier. To ease the reading, I will use the term “API” to refer to these protocols; but the recommendation are general so feel free to use them with all the integration protocols you may find. As well, all the examples will be written in python, but you should be able to migrate the concept to other programming languages.
So, let’s start!
Read the documentation
Start from the beginning. Usually, the best place to start developing is the service’s documentation. If the service you need to integrate has a way to consume its data, it will have documentation about it. Some documentation will be enough, some will not be; but all documentations will show you at least how to start using the API, how the authentication should be made and what are you able and are not able to do using its API.
I know this recommendation is very obvious, but the amount of developers that never read the documentation and go straight to forums such as Stack Overflow or whatever google returns is incredible. These forums are very useful and almost always you will find relevant information there; but the documentation is the only place where you will find how the API should be used from the service developers’ perspective. They are the ones who developed it, so it is better to believe them in the first place than believing MasterHamster98 recommendations in Stack Overflow.
As well, if the service is paid, they surely have a technical support system. Send them a ticket asking for help, you may get useful information from it.
Search for existing libraries
It is very probable that other developers have needed to integrate the service’s API before and have created a library or a SDK that would help them achieving it. As well, if the service is big enough, they may have created SDKs for several languages. Search those libraries before you start developing. They usually ease and reduce boilerplate code, provide examples, have several tests so you are sure the code works and have support using an issues system or something similar. They also have error handling, so you are able to return an error with detailed information without much effort. Using them, you do not need to start your integration from scratch.
For example, you may find python SDKs for Slack, JIRA, Toggl, GitHub, and many more.
If you do not find a library to integrate the service, it may be a good opportunity to create an open source software project and contribute to the developers community. It’s up to you!
Create a class to encapsulate your code
Probably, you will need a lot of information from the service’s API. It means that you will use more than one if its method or endpoints. This may get out of control if you do not handle it properly. The solution I have found for this situation is to create a class that will be in charge of keep everything organized and smooth.
Use the class init method to initialize all the requirements needed to connect to the API such as setting the Authorization Token, setting the base url, and all other requirements you may have.
class ServiceConnector(object):
"""
Class in charge of retrieving information
from Service
"""
def __init__(self):
self.headers = {'Authorization': settings.SERVICE_API_KEY}
self.url = "{}/api/v1/".format(settings.SERVICE_API_URL)
Using a class, will allow you to know exactly what is your code doing, what are you missing and what is not being used at all. The recommendation is to create methods with significant names and docstrings that will let you know what does each method do. As well, keep it as simple as possible. It is better to have several methods that do one thing or two that to have a big method in charge of everything. Lets see it with an example.
from service_idk import Service
from .skeletons import ValueSkeleton
class ServiceConnector(object):
"""
Class in charge of retrieving information
from Service
"""
def __init__(self):
self.service = Service('MySecretToken')
def calculate(self):
returned_data = self.service.get_data()
total = 0
spent = 0
for data in returned_data:
value = ValueSkeleton()
value.total = data['total_days']
value.price = data['money']['amount']
total += value.total
spent += value.price
average = spent / total
return average
from service_idk import Service
from .skeletons import ValueSkeleton
class ServiceConnector(object):
"""
Class in charge of retrieving information
from Service
"""
def __init__(self):
"""
starts service opbject with the API token
"""
self.service = Service('MySecretToken')
def get_information_from_service(self):
"""
uses service.get_data to retrieve
the important data from service.
Returns the date normalized
"""
returned_data = self.service.get_data()
return [self.normalize_data(data) for data in returned_data]
def normalize_data(self, data):
"""
Gets data from service and normalizes it
to match ValueSkeleton format
"""
value = ValueSkeleton()
value.total = data['total_days']
value.price = data['money']['amount']
return value
def calculate_spent_average(self):
"""
gets the spent average for the total days
from the data retrieved from service
"""
returned_data = self.get_information_from_service()
total = 0
spent = 0
for data in returned_data:
total += data.total
spent += data.price
average = spent / total
return average
On the first example, you may get what you need, the average value, but you will only be able to get that information. If you need to use the data from the service to get extra information, such as the total, you will not be able to do it. Instead, on the second example, you still are able to get the average value but you are also able to get different information, such as the data from the server, with no extra effort.
Get spooky and code a lot of Skeletons!
You may have noticed that on the previous example I used a ValueSkeleton so I will explain to you what it is. The concept of Skeleton was given to me by Andrés González three years ago when I was just starting to work in Swapps. My very first project was a dashboard that integrated a lot of external services that the company uses. This project is still active and I am still in charge of it.
So, Andrés told me “Jose, create a skeleton to know what data you need from each object you get from the API”. I was just starting to develop code so I just followed the instruction. What I did not know then and I do know now is that that advice helped me a lot in the future. It even allowed me to integrate similar services with similar data with no effort.
Basically, a Skeleton is just an object that will contain the properties of the object you are receiving that you are going to use. As well, it is an easy way to provide default values,
So, let’s see an example.
I know it is a simple example, but once you use this concept in something bigger, you will understand its power.
class ValueSkeleton(object):
"""
Normalized version of value
"""
total = 0
price = 0
def __str__(self):
return self.total
This concept is useful specially to avoid noise from the service’s API. Usually, the API returns a lot of data, but you just need to use some of these fields. Using the Skeleton, you know exactly what information you are using and what is its data type. As well, you avoid loading gigantic objects into your memory every time you use the integration. Finally, it is very useful if you are going to retrieve similar data from several services but you want to process it equally.
The unknown is there for us to conquer, not to fear!
I know that integrating external services
So, I really hope they will help you next time you need to integrate an external sistem to your application!
