Update your Services Without Disrupting Users with the help of the API Mediation Layer

Published in

Zowe

7 min readSep 13, 2021

{Core} As someone who has been following the development of the Open Mainframe Project’s Zowe API Mediation Layer (API ML), you’ve probably already discovered some of the many ways the API ML can leverage REST APIs. Perhaps you’ve even been following some of the new features of the API ML in other blog posts. You may be wondering, though, how can API ML help me with deployments of other services? Let’s take a closer look at this.

As any experienced developer knows, customers, whether internal or external, are no fans of downtime, nor do they welcome the impacts that often go along with release changes. Understandably, the industry as a whole has seen a noticeable push towards minimizing the risk and impact of releases, an effort which, in part, has led to smaller and more frequent releases.

The effect, however, of these smaller and more frequent releases is that there is now even less tolerance for service downtime. Fortunately, the API Mediation Layer can help. Leveraging the API Mediation Layer makes it possible for developers to deploy new versions of an application in a way that does not result in any downtime from the user’s perspective. Let’s take a structural look at how this works.

Multiple instances of a specific service onboarded to the API Mediation Layer

In the image above we see the typical setup of a service behind the API Gateway. There’s an authentication service providing an API in order to get a new token proving authentication. There are multiple instances of the service which prevents any downtime for the service consumer. From the point of view of external API consumers, the fact that there are multiple instances is hidden by the API Gateway. One benefit of instances being hidden by the Gateway is that it is possible to upgrade services at any point without any downtime on the consumer’s side.

Let’s go first through the two typical patterns that can be modeled with the help of the API Mediation Layer.

Red/Black

This is an application release technique limiting downtime of the applications by having two identical release environments. At any point in time only one of these environments is live and serving the production traffic. When there is a need to update the applications, the upgrade is done in the environment that doesn’t serve the production traffic. After the update is done, after any necessary validations the traffic is moved to the other environment.

This pattern can be applied to the service as known within the API Mediation Layer. The onboarded service can have one or more instances. To apply the red/black pattern you would have a set of instances that are to be known as red and serving production traffic and the secondary environment known as black that will contain the same setup and services without serving the production traffic.The image below shows how will the setup look like before the update and after the update. The act of upgrade is just to route the traffic to the new warmed up set of service instances.

This pattern is easy to use and it works well even with larger updates introducing breaking changes to the instances of the services. The potential issue with it is that it concentrates risk. When there is some unexpected issue with the new production instances all the consumers will experience the issue but with good monitoring tools you can spot the problem early and switch back to the previous environment without much effort.

Blue/Green

This technique limits downtime by gradually transferring user traffic from one instance to another where both run within the same production environment. While the application is being upgraded the traffic is going to both versions of the service. The image below shows what the process look like. The previous version of the application can be kept standby in case there is a problem to divert the traffic back to the previous version.

With the effective tooling for monitoring and observation for the services this technique allows you to limit potential impact in case of issue with new version to just a part of the consumers of the service as well as easy revert in case there is a problem with the new version.

API Mediation Layer role

The next step is to understand what tools the API Mediation Layer provides to support the two techniques explained above. The API Mediation Layer provides a combination of Discovery service working together with the API Gateway to make sure that the requests from consumers are routed to an instance of the service in valid state. How it looks is shown in Fig. 1. The Discovery service provides the information about the valid instances and their state and gateway routes the traffic towards the instances that are UP.

Three APIs linked to the functionality:

GET /eureka/apps
This endpoint provides the information about all the services, their instances and the state of the instances. You can leverage this to get a list of the currently known instances.
In the following example appId is GATEWAY and the instanceId is either gateway-service:gateway:10017 or gateway-service-2:gateway:10017

<application>
    <name>GATEWAY</name>
    <instance>
        <instanceId>gateway-service:gateway:10017</instanceId>
    </instance>
    <instance>
        <instanceId>gateway-service-2:gateway:10017</instanceId>
    </instance>
</application>

PUT /eureka/apps/appID/instanceID/status?value=OUT_OF_SERVICE
This endpoint allows you to override the status of a specific instance of a service to OUT_OF_SERVICE. This state is understood by the Discovery service and Gateway as a mark that no traffic should be routed to the service.
DELETE /eureka/apps/appID/instanceID
This endpoint allows you to remove the specific instance of the service from the Discovery service. This is useful as a part of cleanup.

The next part shows how we can use these endpoints to implement the red/black scenario on an example of discoverable client available within the api-layer repository.

Discoverable client example for the Red/Black scenario

To demonstrate the red/black scenario explained above, I prepared docker containers that you can use yourselves to verify the behavior. You will need to have Docker installed and either curl installed directly or another tool to make request that you have a good knowledge of.

Prepare the containers with applications to be used

The following snippet of code creates a docker network within which all the services we will work with will live in. After that it creates the containers for the core services within the API Mediation Layer necessary for the example as well as two sets of discoverable client applications. Two instances for black version and two instances for the red version.

docker network create api-layer-full

docker create --name gateway-service --network api-layer-full -p 10010:10010 -p 10017:10017 ghcr.io/balhar-jakub/gateway-service:black
docker create --name mock-zosmf --network api-layer-full -p 10013:10013 ghcr.io/balhar-jakub/mock-zosmf:black
docker create --name discovery-service --network api-layer-full -p 10011:10011 ghcr.io/balhar-jakub/discovery-service:black

docker create --name discoverable-client-1 --network api-layer-full -e APIML_SERVICE_HOSTNAME=discoverable-client-1 ghcr.io/balhar-jakub/discoverable-client:black
docker create --name discoverable-client-2 --network api-layer-full -e APIML_SERVICE_HOSTNAME=discoverable-client-2 ghcr.io/balhar-jakub/discoverable-client:black

docker create --name discoverable-client-3 --network api-layer-full -e APIML_SERVICE_HOSTNAME=discoverable-client-3 ghcr.io/balhar-jakub/discoverable-client:red
docker create --name discoverable-client-4 --network api-layer-full -e APIML_SERVICE_HOSTNAME=discoverable-client-4 ghcr.io/balhar-jakub/discoverable-client:red

2. Start the first set of containers

Following commands start the core API Mediation Layer services necessary for demonstration together with black version of discoverable client.

docker start mock-zosmf
docker start discovery-service
docker start gateway-service

docker start discoverable-client-1
docker start discoverable-client-2

3. Verify current setup

// Verify that the services are up.
curl -u USER:validPassword -k https://localhost:10011/
// The expected answer contains:

<tr>
    <td><b>DISCOVERABLECLIENT</b></td>
    <td>
        <b>n/a</b> (2)
    </td>
    <td>
        <b></b> (2)
    </td>
    <td>
        <b>UP</b> (2) -
        <a href="https://discoverable-client-1:10012/discoverableclient/application/info" target="_blank">discoverable-client-1:discoverableclient:10012</a>
        ,
        <a href="https://discoverable-client-2:10012/discoverableclient/application/info" target="_blank">discoverable-client-2:discoverableclient:10012</a>

    </td>
</tr>

// Try to access the services.
curl -u USER:validPassword -k https://localhost:10010/discoverableclient/api/v1/greeting

Expected answer: {"date":"2021-08-16T09:04:59.170+00:00","content":"Hello black, world!"}

4. Start new instances of discoverable client and verify

docker start discoverable-client-3
docker start discoverable-client-4
curl -u USER:validPassword -k https://localhost:10011/<tr>
    <td><b>DISCOVERABLECLIENT</b></td>
    <td>
        <b>n/a</b> (4)
    </td>
    <td>
        <b></b> (4)
    </td>
    <td>
        <b>UP</b> (4) -
        <a href="https://discoverable-client-3:10012/discoverableclient/application/info" target="_blank">discoverable-client-3:discoverableclient:10012</a>
        ,
        <a href="https://discoverable-client-1:10012/discoverableclient/application/info" target="_blank">discoverable-client-1:discoverableclient:10012</a>
        ,
        <a href="https://discoverable-client-4:10012/discoverableclient/application/info" target="_blank">discoverable-client-4:discoverableclient:10012</a>
        ,
        <a href="https://discoverable-client-2:10012/discoverableclient/application/info" target="_blank">discoverable-client-2:discoverableclient:10012</a>
    </td>
</tr>

5. Take old services out of service

// Take down the old services. This needs the certificate
curl -k --cert-type P12 --cert all-services.keystore.p12:password --request PUT --url 'https://localhost:10011/eureka/apps/DISCOVERABLECLIENT/discoverable-client-1:discoverableclient:10012/status?value=OUT_OF_SERVICE'curl -k --cert-type P12 --cert all-services.keystore.p12:password --request PUT --url 'https://localhost:10011/eureka/apps/DISCOVERABLECLIENT/discoverable-client-2:discoverableclient:10012/status?value=OUT_OF_SERVICE'

6. Verify the expected behavior

curl -u USER:validPassword -k https://localhost:10011/
curl -u USER:validPassword -k https://localhost:10010/discoverableclient/api/v1/greetingExpected answer: {"date":"2021-08-16T09:04:59.170+00:00","content":"Hello red, world!"}

7. Cleanup the old services

docker stop discoverable-client-1
docker stop discoverable-client-2

// Remove the instances
curl -k --cert-type P12 --cert all-services.keystore.p12:password --request DELETE --url 'https://localhost:10011/eureka/apps/DISCOVERABLECLIENT/discoverable-client-1:discoverableclient:10012'curl -k --cert-type P12 --cert all-services.keystore.p12:password --request DELETE --url 'https://localhost:10011/eureka/apps/DISCOVERABLECLIENT/discoverable-client-2:discoverableclient:10012'

Conclusion

The API Mediation Layer available within Zowe gives you a simple way to upgrade your own services onboarded to the API Mediation Layer without downtime. In the article you saw some of the patterns used to minimize the risk and impact of such deployment together with an exact guide showing how to use the available tools to implement them. Let’s bring the agility it allows to your customers.

“If you enjoyed this blog checkout more Zowe blogs here. Or, ask a question and join the conversation on the Open Mainframe Project Slack Channel #Zowe-dev, #Zowe-user or #Zowe-onboarding. If this is your first time using the OMP slack channel register here.”