Effective API Management (Part 1)

I’ll start this post by addressing all the doom and gloom mindset around AI taking everyone’s jobs. I find this particularly funny because there’s still so many problems out there that need solving and so much more work to be done. If you think AI is going to steal everyone’s jobs in the next year, you probably need to step outside, sip some coffee, and touch grass (or snow if you’re in the Midwest like me). Case and point, the irony in me dedicating a post to a specific use case: effective API management. Now I will say that if AI does make API management much easier than that would be lovely, as it is quite tricky and requires a ton of thought and careful considerations. But until then, please feel free to give this a read and come back to reference it whenever your boss asks you why you opted to push REST APIs over GraphQL. Let’s dive in!

So what is an API? API stands for Application Programming Interface. Why are APIs important? Well APIs are a pretty fundamental application concept and continuing to grow in popularity for how various technologies interact with each other. APIs use defined communication protocols to share data with other resources, which we’ll get into shortly. You can find APIs in your everyday life all over the place, some of the most common examples of where you see APIs used would be: Google Maps (whenever you go on a restaurant’s website and you scroll down to see their location displayed where exactly it’s at, that’s Google Maps API being called), Payment APIs such as Stripe or Square (you go to your favorite coffee shop or small business and pay with your card, that transaction gets processed and handled by the API), or one of your favorite social media sites / music streamers (ever seen someone’s Instagram feed or Tik Tok auto scroll on a website? A playlist from Spotify displayed on a personal site? That’s all using APIs to call and display that information!). Hopefully you have a rough idea of what an API is now, but long story short is that APIs are how a lot of information is shared from one app to another! Below is a helpful image from Akamai.

Because this post is focused on effective API management, we’re going to talk about the full lifecycle of an API. We like to do the right thing as engineers, so we follow “best practices” and align our API development to robust SDLC (Software Development Lifecycle) standards. That means we consider all aspects of an API: Planning and Designing, Developing and Testing, Implementing and Deploying, Operating and Maintaining, then finally retirement and decommissioning. Essentially we’re going to do our due diligence to understand the best way to design our API based on our needs, develop it securely so that it’s safe and scalable, make sure it’s usable for developers and maintainers, then when it’s time to retire it we do so in a way it doesn’t break anything.

PLANNING AND DESIGNING PHASE

The first phase of the lifecycle is understanding what is our API’s purpose? What do we want it to do and how are we going to build it? This is the phase where it’s time to think about anything and everything; think about encrypting your API traffic, what communication protocol to use, how your app’s infrastructure is set up to support the API, how to inventory all your APIs, and more! Now this isn’t too different from the planning and design phase for any project, the goal is to understand what you’re building and the best way to execute that.

For the sake of ease, we’re going to use an example of a bank account that can send and transfer money for a user. I chose this because I completed a project (link to Github repo) for this in early 2024, following along with a video on YouTube I found; you can find the video here: How to Build a Complete JSON API in Golang. So the idea here is to build an API that the bank’s users would be able to use to send and transfer money to other banks, either a user at a different bank via connecting to our API or our two users who both use our bank sending money to each other. For our API to do what we need it to do, we’re going to have to define a few things that matter for handling data associated with our users. To start, we’re going to define the fields relevant to our application that are populated by our users. We have the following type “Account” defined:

• type Account struct { - this is our Account struct, it contains all the below data fields associated with it. This is how we manage our users, and each user will have an Account with the below information.
  • ID int json:"id" - this is our ID, whenever a new user joins the bank and signs up for an account they get a unique ID. This is to help us manage their data and avoid duplications where users have the same name for example.
  • FirstName string json:"firstName" - First name of our users.
  • LastName string json:"lastName" - Last name of our users.
  • Number int64 json:"number"- this is the number associated with our account, similar to ID, this is used for handling data for our users, aka when our API is called it knows which account to look at.
  • EncryptedPassword string json:"-"- This is where our users passwords will be stored, as you can see we will encrypt them for security reasons.
  • Balance int64 json:"balance" - This is the account balance for our users, aka how much money they have in their account.
  • CreatedAt time.Time json:"createdAt"} - This is a timestamp of when our user’s account was created.

If you want to become more familiar with Golang or want to learn how to build an API from scratch, I would highly recommend the video series from Anthony GG I hyperlinked. I think he does a really good job of not only talking through the design of the API, but how it functions as well.

Ok, so we have our account fields defined, because as mentioned, this is part of the design and planning phase. You are seeing “JSON” pop up a handful of times, because this is how we define our data format. When other APIs look to communicate with ours, we have to be consistent in how we deliver that information so it doesn’t create issues. If our API does not provide ease of functionality for others to use, people are less likely to use it and thus defeats its purpose. Now the way we deliver our information and how APIs worldwide are designed is typically one of three main ways: REST API, SOAP, and GraphQL. Each of these methods has pros and cons, and should be carefully evaluated before ultimately deciding which makes sense for you. At a high level this how each is defined and how they operate:

• REST API - The most common design and one that you would probably hear the most about if you talked to API developers. REST stands for “Representational State Transfer”. This means that we follow standard HTTP methods (HTTPS if we follow proper security methodologies and have website certificates) to action the data we need to. How we write the code for these APIs to be interacted with should allow users to call our APIs and fetch, create, update, or delete data stored on a server. Those methods are:

  • GET (retrieve data) - this is the method another bank would use to collect the information needed to send to a user, something like GET Account.Number.
  • POST (create data) - this is the method another bank would use to send information to one of our users, something like a transfer request to POST Account.Amount to one of our users.
  • PUT or PATCH (update data) - this is the method that could be used to update existing account information. Say a user gets married and changes their last name, PUT Account.LastName
  • DELETE (remove data) - this method is hopefully a little self explanatory but if you correctly guessed it removes data, then you’re right! This could be for if we needed to delete a user’s account, DELETE Account.

• SOAP - Simple Object Access Protocol Application Programming Interface) is a messaging protocol for exchanging structured information between systems. SOAP is a protocol with strict rules and standards, unlike REST, which is an architectural style.That means that all messages are formatted as XML documents, making it platform-independent. SOAP APIs are described using WSDL (Web Services Description Language), which defines the structure, operations, and protocols of the API and while commonly used with HTTP/HTTPS, SOAP can also work over SMTP, TCP, or other network / communication protocols.

• GraphQL - A query language and runtime for APIs that allows clients to request exactly the data they need and nothing more. It provides a flexible alternative to REST by enabling clients to define the structure of their desired responses. GraphQL was originally an API methodology that began development by Facebook in 2012, before ultimately becoming open source. Some of the considerations to use GraphQL would be for if you want to use a single endpoint for all operations. If you have clients that want to specify exactly what data they need, GraphQL can reduce over-fetching (too much data) and under-fetching (too little data). Lastly, the API is defined by a schema that specifies the types of data and relationships, ensuring predictability.

Now once we’ve made a decision on what methodology makes sense, and you can have multiple depending on business needs, it is time to plan out how the API will mesh with the other phases of the SDLC. We’ll go into more detail in each phase, but the key here would be to make sure you’re aligned enterprise wide as an organization. This means that if all your code is stored in Github Repos, use Github functionalities to review the code for bugs, set up integrations to onboard your API into the central API Management solution (you need one of these), scan for vulnerabilities, make sure API keys are NOT stored in plain text, set up ENV variables for your API keys, and more.

To stay true to our post’s spirit of effective API management, make sure there are clear guidelines on development practices. That means there should be documentation that explains what each API does / should do and can be easily accessible for developers. If your organization has standards specific to APIs, there should be controls that help ensure the standards are followed (we will provide examples throughout the phases). Lastly if there are certain packages or methodologies used in the development of the APIs, make sure they are consistent and followed across the org. This means that we use bcrypt to encrypt a specific application's user passwords, developers for that team use bcrypt and not introduce various other packages. It may seem like it shouldn’t need to be said, but having consistent practices that are introduced at the design and planning phase for all developers to follow, helps prevent issues down the road. I’ll note there is more collaboration that goes on here if you are a part of a larger org, such as working with security to make sure there’s proper monitoring, as well as folks like the Network team to make sure there’s appropriate traffic management and your API is behind proxies and firewalls. Just know there’s a lot of work that goes into making sure everything is set up prior to actually going live and all of it is important. Now is the time to be picky and strict!

DEVELOPING AND TESTING PHASE

So we figured out what we want our API to do and we’re ready to start building it! I should preface that there’s two flavors of APIs that are developed, outside of everything we talked through in the planning and designing phase. There are APIs that are developed for applications we are responsible for with the intent to share information / data we manage and there are APIs that are developed to connect to existing APIs to retrieve information / data other people manage. If we use an example, there is a team at Google responsible for developing and maintaining the Google Maps API, this is scenario one as Google is the owner of Google Maps and they developed an API to share that data. The second scenario is if we have a website that wants to have a feature that displays the location of something, we develop an API based on Google Maps documentation to retrieve that information and display it on our webpage. In both scenarios an API is developed, but the difference is only one is responsible for maintaining the core data (Google Maps in this case). I’m calling this out because if you are in scenario two, please look up the documentation for how the API functions. Any team responsible for owning and managing an API almost always has documentation that goes along with it and explains stuff that makes it much easier to use, oftentimes with examples. It should also be said that if you are scenario one, build documentation that explains how the API should be used and include examples.

So for this phase, we are in the camp of developing our own API that other banks would use to retrieve information from us, aka scenario one from the paragraph above. I can’t go into every single small detail of how the code should be written here, but we are going to cover the key functionality that needs to be developed for this particular API. In our example, we are a bank with an API that serves our clients. That means for our API it needs to at the bare minimum have the following:

• Ability to create and update a user’s account. Think of the Account struct we showed in phase one. That is the crux of our API as that is all the relevant data that is managed by our API.
• Send and receive transfers for money in our user’s accounts. If another user sends me money either via a wire, Zelle, venmo, etc. it should be able to identify our users and send a specified amount.
• Store data relevant to our users in a database. Any sort of situation where someone’s information or data is being handled will typically have a database. We’re not going to dive into databases for this article, just know they’re relevant.
• Handle logins for our users. If our user wants to login either to view their account or perform an action, they need to be able to login with a username and password.
• Encryption and appropriate security methods. We will use JWT (JSON web tokens) for authentication and bcrypt for encrypting our passwords. This provides an extra layer of security so we don’t expose our user’s information to the public internet.

Below is a code snippet of how our API handles each of those tasks with respective functions. see how the first line has "s.handleLogin"? That's where the API handles the login for a user when they are attempting to sign in.

Ok so with all that in mind, we’ll send our requirements to the development team responsible for building this API. As noted previously, there should be coding practices defined at the enterprise level for developing APIs, as well as controls in place to support company standards.Some examples that would apply for the development of an API that we should expect to see almost anywhere are:

• Version control and code reviews. The standard example is using git / github to make changes to the code and review pull requests prior to pushing the code to the main branch.
• Using a Development (DEV) and Test environment for deploying the code prior to any production releases. If this is a part of an application you need to make sure the application actually works before trying to send any sort of requests to the application’s API.
• Testing the API. There are various API tools out there to test request handling. Some of the most popular are Postman, Katalon Studio, Insomnia, and JMeter. There are also extensions available in your IDE to test as well, I use Thunder Client in VScode to test APIs. Note, whichever you use, it should be the same enterprise wide.
• ITSM (IT Service Management) / Change Management approvals. Most mid to large size organizations have teams dedicated to ITSM and change management to ensure proper due diligence is performed prior to any sort of application feature going live.
• Disaster Recovery and Failover testing. If your API fails or something happens to the infrastructure supporting the application, make sure you have data backups in place and a plan to restore functionality timely and effectively.
• Successful onboarding into your APIM solution. Your API Management solution should be able to onboard and inventory your API prior to going live. The benefits of having an APIM are: Enhanced API Security, Scalability and Performance Optimization, Monitoring and Analytics, Centralized API Governance, and Lifecycle Management just to name a few.

There’s one last part I’m going to call out specifically with respect to the API controls due to the importance of it and that is API key management. API keys are critical for establishing secure connections to APIs. They are unique, complex strings generated by an API that serve as authorization credentials, granting access to specific data or services. Proper management of these keys is crucial because if an unauthorized party gains access to your API key, they can exploit your data or impersonate your identity, potentially leading to security breaches.

To mitigate these risks, it's essential to store API keys in a secure location, such as an encrypted vault or environment variable, and follow best practices like rotating keys periodically. By prioritizing API key management, you help protect your data, maintain trust with your API users, and enhance the integrity of your systems. To do your best at properly managing your API keys, you should be doing at a minimum the following:

• Store your keys in a secure vault. If you think about my previous post “What the H*eck is the cloud?” You may remember that there are functionalities available to you from your cloud provider. One of the one’s Azure has is Azure Key Vault. This is a place where you can safely store and manage sensitive information such as your API keys. DO NOT store them in plain text.
• Use ENV variables in your code for where API keys are used. If you are not familiar with ENV variables, please look them up. These type of variables help mask the actual key as they are stored.
• Limit Permissions. Assign only the minimum permissions necessary for each API key and use separate keys for different services.
• Rotate and Expire Keys. Regularly rotate API keys, set expiration dates, and automate updates where possible. In the next phase I will provide an example of how this can be done with Azure Key Vault.
• Restrict Access. Implement IP whitelisting and use scoped or temporary tokens when available.
• Monitor and Respond. Log key usage, monitor for anomalies, and immediately revoke and replace compromised keys. This should be worked out between the APIM team and security to make sure appropriate monitoring and alerting is set up.

So with all that in mind, you should be ready to move on to the implementing and deploying phase. Initially when I started writing this post I thought I would be able to fit all of this in one post, however I’ve quickly realized that’s not going to happen (lol). Stay tuned for part 2, coming out shortly!