Deploying an API into the various stages of a software development pipeline involves not only the aspect of writing (designing) an API specification, but also having or simultaneously deploying a corresponding infrastructure. This article describes possible ideas and steps of a deployment process, starting with the design of an API specification.
OpenAPI Spec
When designing an API, OpenAPI is one possible description form, along with AsyncAPI . Since many people still talk about Swagger, I would like to give OpenAPI some historical context. In 2009, Tony Tam started developing a description language that became known as “Swagger”. The first version (1.0) was released in August 2011. But only version 1.2 (March 2014) ensured a wide distribution. In September 2014, the next milestone, 2.0, was released. While the previous versions still required two files, the now familiar one-document structure started to take shape. With Tony Tam’s move to Smart Bear in September 2015, the Swagger specification was handed over to a new organization under the umbrella of the Linux Foundation, in December of the same year: the OpenAPI initiative. Both specifications were now available in version 2.0 . From then on, further development progressed only for the OpenAPI specification. In July 2017, version number 3.0.0 was published. Three more patches of this release appeared until 2020, before 3.1.0 saw the light of day at the beginning of 2021. In the following, however, only version 3.0.3 plays a role.
Details
Next, let’s take a closer look at the OpenAPI specification. In my talks, I generally use the OpenAPI Map by Arnaud Lauret/@apihandyman to look at the individual objects in the specification. For this article, we’ll walk through some of the elements via a sample specification. An OpenAPI specification basically consists of the following objects:
infoserverspathscomponents
These roughly describe the API functionality, with the representation of each endpoint under paths`.
1paths: 2 /news: 3 get: 4 description: gets latest news 5 operationId: getNews 6 tags: 7 - news 8 responses: 9 '200': 10 description: Expected response to a valid request 11 content: 12 application/json: 13 schema: 14 $ref: '#/components/schemas/ArticleList' 15 '404': 16 description: Unexpected error 17 content: 18 application/json: 19 schema: 20 $ref: '#/components/schemas/Error' 21 parameters: [] 22 summary: ''
In the example above, the /news endpoint, supported by the HTTP GET verb, returns either a successful response in JSON format with status code 200 or a general, unspecified, error with status code 404. Up to this point, everything is fine. However, the goal is also to include configurations for the endpoint, a gateway or a portal in an OpenAPI specification.
OpenAPI Extensions
The OpenAPI Extensions come into play here. They are often referred to as Specification Extensions or Vendor Extensions. But basically they are custom properties that start with the letter “x” followed by a “-”. With these we are now able to describe functionalities that are not part of the standard OpenAPI specification. The extensions can be used at the root level of the specification or in the sections on info, paths, responses, tags and security schemes. By using the properties, we also regain some flexibility in the design of the OpenAPI specification.
1paths: 2 /news: 3 get: 4 description: gets latest news 5 operationId: getNews 6 tags: 7 - news 8 responses: 9 '200': 10 description: Expected response to a valid request 11 content: 12 application/json: 13 schema: 14 $ref: '#/components/schemas/ArticleList' 15 '404': 16 description: Unexpected error 17 content: 18 application/json: 19 schema: 20 $ref: '#/components/schemas/Error' 21 x-amazon-integration: 22 http-method: GET 23 type: mock 24 parameters: [] 25 summary: ''
Using x-amazon as in the example, the possibilities of the extensions can be seen very clearly. Here, the gateway is now informed that the endpoint is not assigned an upstream service, but only enables mocking for the time being. Depending on the mocking technology, defined examples or random values are accessed. Cross-cutting concerns, such as rate limiting, can also be controlled globally or locally for a particular endpoint via extensions. Here it is important to look carefully at the documentation of the respective providers to see to what extent the reading of configuration parameters from the OpenAPI specification is supported.
Spotlight on APIOps deployment phase
By knowing about the extensions, we can also move forward within the process from the design of the specification to the deployment. For this purpose, I would like to look at the APIOps phase model once more.
The specification was placed under version control after or even during the design phase. Within the Continuous Integration subphase, the specification basically goes through a linting and testing process. During linting, validation is performed simultaneously. The agreed rules of the applicable API guidelines form the basis of the rule set for validation.
As mentioned in the post about APIOps, Stoplight Spectral provides a way to be used as a tool for linting and validation. With Spectral it is also possible to create your own rule sets.
1formats: 2 - oas3.0 3extends: 4 - 'spectral:oas' 5rules: 6 set-x-amazon-integration: 7 description: x-amazon-integration must be set. 8 message: x-amazon-integration is missing 9 given: $.pathts[*] 10 then: 11 field: x-amazon-integration 12 function: truthy
With the ruleset as an example, it is now possible to check for the use of an extension in the specification.
This thus provides a path for secure deployment of the API and corresponding infrastructure configurations, the final phase of APIOps. The goal, after all, is to use the specification to infer bases of automation rules that should support operational processes.
This in turn pays off in terms of quality and also in terms of speed. Deploying an API is not just primarily about placing a file in a specific location. At the same time, infrastructure components are also to be provided and configured.
Components of the infrastructure
But what is actually meant by infrastructure components in the case of APIs? This question is relatively easy to answer. After all, the APIs of a service do not exist in a vacuum, but are part of a system that is based on certain infrastructure components. In terms of APIs, an API gateway and portal or hub are added to the existing components. Depending on the architectural conception, we speak of central components or see, for example, a gateway as part of self-contained systems. Looking at the current market, many API gateway vendors talk about control and data planes. Here, the administration layer (control plane) is seen as a global component, whereas the data planes are part of self-contained systems. For the further process we select the view of the central components. Thus we consider the following architecture sketch.
Now that the architecture has been explained in theory, we need to make it possible to apply the configuration information in the specification to the gateway and also to the portal. This paves the way for Configuration as Code or, more precisely, Infrastructure as Code.
Infrastructure as Code
This step, the deployment of the configuration, depends on the components used. The wish would be for a standard to be established that understands the specification as a single source of truth, which has been or is being implemented by some providers (e.g., APISIX, gravitee.io) with the help of a rest API. Others (e.g., AWS, Azure, Kong, Tyk) require further tooling in the form of templates and command line interfaces (CLI). For our deployment process this means that we have to extend it with corresponding technologies. This also increases the complexity of the deployment, including the corresponding dependencies. We will take a closer look at three variants. We will examine the corresponding gateway while focussing on the deployment functionalities.
Rest API Interface
For the consideration of a Rest API interface approach in the context of Continuous Deployment, let’s take a closer look at gravitee.io and APISIX . Both tools make it possible to provide an OpenAPI specification and its configuration via a rest API. Whereby only APISIX can deploy a deep configuration based on the use of extensions. When using gravitee.io, a configuration of the so-called policies has to be done outside the OpenAPI specification.
CLI Technology
In one of the previous sections I had located Kong’s API gateway within the CLI approach. This is not correct, because Kong also allows control via a so-called admin API. However, it is not possible to read out the extensions within the specification. In this case, a different toolchain based on CLI has been established. If you want to work with Kong’s gateway, it has also proven useful to rely on Insomnia as an editor, which ensures seamless integration. A library (openapi-2-kong ) contributes to this. With this library it is possible to transform an OpenAPI specification into a declarative configuration. Thanks to this generated YAML file, it is now possible, with the help of decK , another CLI tool, to securely provide or update the configuration to the gateway. What makes decK special is that it makes it possible to detect drifts within the configuration. Tyk relies on a similar context as decK with Tyk Sync . To wrap up the article, I’d like to take a deeper look at the approach with AWS as an example of a cloud provider.
AWS as an example
At AWS, we find two possible approaches to deploying infrastructure. One is CloudFormation templates and the other is the Cloud Development Kit (CDK ). Corresponding configurations for the gateway or the endpoints are made in the OpenAPI specification as a single source of truth. The API specification shown below serves as the basis for the sample implementation.
1--- 2openapi: 3.0.0 3info: 4 title: API Gateway OpenAPI Example 5 version: 1.0.0 6 7paths: 8 /api/posts: 9 get: 10 summary: List Posts 11 operationId: listPosts 12 requestBody: 13 required: true 14 content: 15 application/json: 16 schema: 17 '$ref': '#/components/schemas/CreatePostRequestBody' 18 responses: 19 '200': 20 description: Retrieve the list of Posts 21 content: 22 application/json: 23 schema: 24 '$ref': '#/components/schemas/ListPostsResponseBody' 25 x-amazon-apigateway-integration: 26 httpMethod: POST 27 type: mock 28 post: 29 summary: Create a new Post 30 operationId: createPost 31 responses: 32 '200': 33 description: Success 34 content: 35 application/json: 36 schema: 37 '$ref': '#/components/schemas/Post' 38 x-amazon-apigateway-integration: 39 httpMethod: POST 40 type: mock 41 42components: 43 schemas: 44 BasePost: 45 type: object 46 required: 47 - title 48 - description 49 - publishedDate 50 - content 51 properties: 52 title: 53 type: string 54 description: 55 type: string 56 publishedDate: 57 type: string 58 format: date-time 59 content: 60 type: string 61 Post: 62 allOf: 63 - $ref: '#/components/schemas/BasePost' 64 - type: object 65 required: 66 - id 67 - createdDate 68 - updatedDate 69 properties: 70 id: 71 type: string 72 createdDate: 73 type: string 74 format: date-time 75 updatedDate: 76 type: string 77 format: date-time 78 CreatePostRequestBody: 79 allOf: 80 - $ref: '#/components/schemas/BasePost' 81 ListPostsResponseBody: 82 type: array 83 items: 84 $ref: '#/components/schemas/Post'
Cloudformation
If we look at it from CloudFormation’s point of view, we need templates to create a stack for a deployment bucket.
1AWSTemplateFormatVersion: 2010-09-09 2 3Resources: 4 ArtifactBucket: 5 Type: AWS::S3::Bucket 6 7Outputs: 8 ArtifactBucket: 9 Description: The name of the artifact bucket 10 Value: !Ref ArtifactBucket 11 Export: 12 Name: !Sub ${AWS::StackName}-artifact-bucket
Now we can make the OpenAPI specification available via the S3 bucket.
1aws s3 cp openapi.yaml s3://<YOUR DEPLOYMENT BUCKET NAME GOES HERE>
Finally, the template to be able to deliver the API gateway. With this we access the specification in the provided bucket.
1AWSTemplateFormatVersion: '2010-09-09' 2 3Parameters: 4 5 ProjectId: 6 Type: String 7 Default: experiment 8 9 Bucket: 10 Type: String 11 Default: api-gateway-openapi-artifact-bucke-artifactbucket-1wmq2pswrxwjw 12 13 OpenAPIS3Key: 14 Type: String 15 Default: openapi.yaml 16 17Resources: 18 19 Api: 20 Type: AWS::ApiGateway::RestApi 21 Properties: 22 Name: !Ref AWS::StackName 23 Description: 'An experimental API' 24 FailOnWarnings: true 25 BodyS3Location: 26 Bucket: !Ref Bucket 27 Key: !Ref OpenAPIS3Key 28 29 ApiDeployment: 30 Type: AWS::ApiGateway::Deployment 31 DependsOn: Api 32 Properties: 33 RestApiId: !Ref Api 34 35 Stage: 36 Type: AWS::ApiGateway::Stage 37 DependsOn: 38 - Api 39 - ApiDeployment 40 Properties: 41 StageName: test 42 RestApiId: !Ref Api 43 DeploymentId: !Ref ApiDeployment 44 MethodSettings: 45 - ResourcePath: '/*' 46 HttpMethod: '*' 47 LoggingLevel: 'INFO'
The deployment is done by the following command:
1aws cloudformation create-stack --stack-name api-experiment --template-body file://api-stack.yaml
Unfortunately, CloudFormation templates do not have a nice developer experience that allows developers to deploy infrastructure with some speed using any of their respective favorite programming languages. And that’s where AWS’ Cloud Development Kit comes into play. I’ll spare us an introduction at this point, instead I’d like to look at how the CloudFormation example can be translated in the direction of CDK. To do this, we first need an initial CDK project, which we start with cdk init app –language java. For this we need to create an appropriate folder for our project in advance. I chose Java because it is the programming language closest to me. Now, after renaming the packages, the project has this structure:
CDK
When using CDK, we have several options. If CloudFormation templates are already available, as in our case, we can use them with the help of
1CfnInclude cfnInclude = CfnInclude.Builder.create(this, id)
2 .templateFile(templatePath)
3 .build();
in our CDK app. Thus, we create a separate Java class for each stack. Now it is possible to perform the three-step deployment supported by CDK. After this works, we can think about refactoring to completely avoid using CloudFormation templates.
1package de.codecentric.softwerker19;
2
3import software.amazon.awscdk.core.*;
4import software.amazon.awscdk.services.apigateway.AssetApiDefinition;
5import software.amazon.awscdk.services.apigateway.InlineApiDefinition;
6import software.amazon.awscdk.services.apigateway.SpecRestApi;
7import software.amazon.awscdk.services.s3.assets.Asset;
8
9import java.util.HashMap;
10import java.util.Map;
11
12public class AgoCdkProjectStack extends Stack {
13 private final String apiName = "Example API";
14 private final String openApiFilePath = "./api/openapi.yaml";
15
16 public AgoCdkProjectStack(final Construct scope, final String id) {
17 this(scope, id, StackProps.builder().env(Environment.builder().region("eu-central-1").build()).build());
18 }
19
20 public AgoCdkProjectStack(final Construct scope, final String id, final StackProps props) {
21 super(scope, id, props);
22
23 Asset asset = Asset.Builder.create(this, "ExampleAsset").path(openApiFilePath).build();
24 Map<String, String> variables = new HashMap<>();
25 variables.put("Location", asset.getS3ObjectUrl());
26 Object data = Fn.transform("AWS::Include", variables);
27
28 InlineApiDefinition apiDefinition = AssetApiDefinition.fromInline(data);
29 SpecRestApi restApi = SpecRestApi.Builder.create(this, apiName)
30 .restApiName(apiName)
31 .apiDefinition(apiDefinition)
32 .deploy(true)
33 .build();
34
35
36
37 }
38}
The new AgoCdkProjectStack class provides a simple refactoring of the CloudFormation templates. However, for this we need to run cdk bootstrap before deployment, since we use an asset object that requires an S3 bucket. The OpenAPI specification is then stored in this bucket. Depending on whether the deployment pipeline clears the bucket, bootstrap only needs to be run once. Following this, we can deploy the AgoCdkProjectStack stack using cdk deploy AgoCdkProjectStack.
Conclusion
In this article we noticed that the deployment of an API needs much more than the mere API specification. We need to think about OpenAPI extensions. At the same time, we need to have a lot of information about the intended infrastructure ready in order to bring about the appropriate automation for the deployment of the API specification and associated infrastructure. If all this fits, the implementation is a challenge that can be solved.
Was this post helpful?
More articles
fromDaniel Kocot
21.8.2024 Your job at codecentric?
Jobs
Agile Developer und Consultant (w/d/m)
Alle Standorte
More articles in this subject area
Discover exciting further topics and let the codecentric world inspire you.
Gemeinsam bessere Projekte umsetzen.
Wir helfen deinem Unternehmen.
Du stehst vor einer großen IT-Herausforderung? Wir sorgen für eine maßgeschneiderte Unterstützung. Informiere dich jetzt.
Hilf uns, noch besser zu werden.
Wir sind immer auf der Suche nach neuen Talenten. Auch für dich ist die passende Stelle dabei.
Blog author
Daniel Kocot
Senior Solution Architect / Head of API Consulting
Do you still have questions? Just send me a message.
Do you still have questions? Just send me a message.