‘API generators’ is an umbrella term for various generators in the context of API development. Some generate code, some generate API descriptions, and some generate tests or documentation.
Generators automate parts of the development process. In general, the goals of integrating generators are to speed up the development process, make it more convenient, or even produce higher consistency and fewer human errors. However, depending on the generator type, these goals can vary.
This blog post introduces the different types of generators, providing details such as their purpose and the challenges associated with each. The aim is to enable readers to form an initial opinion on whether generators are generally suitable for their API project, and if so, which one(s).
Definitions
The terms API specification, API definition, and API description are often used interchangeably. This blog post sticks to the definitions that are used in the documentation of OpenAPI:
- API description: The interface description of a specific API, e.g. the OpenAPI document of a specific API and its references
- API specification: A specific standard to describe APIs, e.g. the OpenAPI standard
Influence of the development cycle on the generator choice
The development cycle affects the choice of the generator type, as some generator types are tightly coupled with either a design first or code first approach. Two prominent examples are the generation of the server code from the API description, which is only applicable when the API design, i.e. the API description, is written first, and the reverse generation of the API description from the server code, which is only possible if the server code is implemented first.
Definitions
- (API) design first: The API interface is designed before anything else is implemented. Implementation is subordinated to the interface design. The interface is the API description, which is seen as a contract between the provider and the consumers, detailing how the API functions and how it can be used. The developers meet API stakeholders to finalise the contract before the development has started. This ensures that the needs of everyone involved, including consumers, are met in the final product. The approach does not only imply that the interface is developed first, but also that the developer’s mindset is focused towards user experience. The goal of design first is to provide a higher level of usability. However, it requires a lot of initial planning and assumes that design decisions can be made early on. This may not always be possible and could delay the start of development.
- Code first: The implementation is done first and the API interface emerges from the code. The main advantage is fast development and prototyping. The developers have a high degree of flexibility as they are not bound to a strict interface contract, which is most beneficial in an environment of unclear or changing requirements. However, this can promote bad API quality and inconsistent interface design, ultimately leading to a bad user experience.
- Without going into too much detail, it is usually recommended to choose API design first to ensure API quality and usability, in particular if different teams or even company-external customers will be using the API.
The section “Generators in the context of design first and code first” illustrates what a development cycle with integrated generators can look like.
Types and Classes of API Generators
There are a lot of generators related to the development of APIs. This section classifies generators based on their input and generated output. The aim is to provide an overview over all types that are currently on the market.
It is possible to generate…
- an API description from a data source
- an API description from a data model
- an API description from server code
- server code from an API description
- client code from an API description
- tests and documentation from an API description.
Figure 1: API generator classes
An important observation is that the API description is central to the API generator topic, as it is involved in each generator type (see figure 1). Therefore, the generator types can be categorized in description generators, which produce an API description, and in description-driven generators which take the API description as an input. Server and client code generation is also known as code generation.
The generator landscape involves many layers of the API application stack, including the data backend, the server codebase and the product front. While most generators are used by providers, there is also a generator that can be used by consumers, which generates the API client (see figure 2).
Figure 2: API generator types categorised by layers of the API application stack and viewed from a provider and consumer perspective.
Though there are many generation possibilities, not all are recommended to take. There is no universal verdict on all generator types. Some are widely used, some are controversial, and some should never be used at all.The following section provides details of each generator type and draws conclusions on that matter.
Each Generator Type in Detail
This section explains the different generator types and serves as an initial guide to choose the right generator type for a specific API project.
For each generator type, it provides a detailed table including the main idea and motivation behind it, how it works, and the most applicable development cycle. It also explains any potential challenges or problems and provides recommendations for usage. Finally, it lists example tools that fall into the specific generator category.
Data source to description
Idea | Define the user interface with the simplest and most naive approach: Just use the same structure that you defined for your internal data source for your API. |
Used by | Providers |
Steps |
|
Applicable in | Never recommended in any development cycle |
Motivation | Simplicity, development speed |
Challenges, Problems |
|
When to use | This is never recommended, as it implies that the external data model equals the internal data model. This is known to be bad practice, as it almost inevitably leads to bad usability and can cause security issues. |
Tools | I won’t provide any links here, as this should not be used. |
External data model to description
Idea | Avoid writing the API description directly in yaml/json files. Instead, the developers can define the external data model with code. As the data model is a big part of the API description, we can enrich it and then generate the API description from it. |
Used by | Providers |
Steps |
|
Applicable in | Design-first development cycles |
Motivation |
|
Challenges, Problems |
|
When to use |
|
Tools | DSLs
Generators
|
Server code to description
Idea | Avoid writing the API description manually. Instead, automatically derive the implicitly defined interface from the server code. |
Used by | Providers |
Steps |
|
Applicable in | Code-first development cycles |
Motivation |
|
Challenges, Problems |
|
When to use |
|
Tools |
Description to server code
Idea | Generate the server stub from the API description, i.e. code that acts as a placeholder for the actual server implementation. It receives requests from the client, unpacks the arguments, calls the actual, manually implemented function on the server, and sends the results back. |
Used by | Providers |
Steps |
|
Applicable in | Design-first development cycles |
Motivation |
|
Challenges, Problems |
|
When to use |
|
Tools |
Description to client code
Idea | Make it more convenient for consumers to use an API by providing a generated client SDK. This way, users won’t have to deal with HTTP requests directly. |
Used by | Providers / Consumers |
Steps |
|
Applicable in | Any development cycle, as the client development is independent from the server development. |
Motivation | Usability / User experience |
Challenges, Problems |
|
When to use |
|
Tools |
Description to test
Idea | Tests can be automatically generated. This saves time to write common test cases that can be directly derived from the API description. |
Used by | Providers |
Steps | As there are various different types of tests like API description tests, contract tests or performance tests, the test generators are quite heterogeneous. However, providing an exhaustive overview is not the goal here. Contract tests (e.g. Portman, Schemathesis)
AI-driven tests:
|
Applicable in | Any development cycle |
Motivation |
|
Challenges, Problems |
|
When to use |
|
Tools |
Description to documentation
Idea | Documentation can be mostly automatically generated based on the API description, as it contains all necessary information to use the API, including human-readable descriptions. |
Used by | Providers |
Steps |
|
Applicable in | Any development cycle |
Motivation |
|
Challenges, Problems |
|
When to use |
|
Tools |
Generators in the context of design first and code first
As server code generation and description generation from server code are both somewhat popular and somewhat controversial, this section is dedicated exclusively to these two, considering them from the perspective of their development cycle.
Important Note: Server code generation makes sense in a design first development cycle. Description generation from server code makes sense in a code first development cycle. Nevertheless, the generation type and the respective development cycle are not synonyms.
- Server code generation and design first: Server code generation does not necessarily imply that the developers think much about the API design. This is a question of mindset. However, server code generation works well with the design-first development cycle.
- Description generation and code first: Description generation and code first do not necessarily imply that developers do not think about the API design. Again, this is a question of mindset. Code first without description generation is often done. Description generation without code first is unusual, but theoretically possible.
Server code generation
The API description is provided by the developers. The server stub is generated, but has to be supplemented by the developers.
The diagram illustrates server code generation in a simplified design first development cycle.
- The API interface is designed first and serves as a contract that the implementation must comply with.
- Using the resulting API description, the server stub can be generated. It is regenerated whenever the API description is changed, but should never be edited manually.
- Instead, developers implement the stub’s dummy methods in a spatially separate, manually created extension of the server codebase. This is usually realised by implementing a generated interface or inheriting from a generated class. The codebase extension is persistent even after server code regeneration.
- The contract tests ensure consistency between the API description and the code. Until all tests succeed, the codebase extension must be completed or corrected according to the test results. API description and generated server stub should not be touched.
Description generation from server code
The server code is implemented by the developers and the API description is automatically derived from it.
The diagram illustrates server code driven generation in a simplified code first development cycle.
- Implement the server code.
- Generate the API description from it.
- Review the API description and validate it (e.g. with Spectral). The validation ensures that the description contains all required or recommended fields based on its current structure. Contract tests are not as helpful here because they test the server implementation based on the API description. However, they don't test the API description against the server implementation. Nevertheless, contract tests might help to detect false annotations in the code, if they were not already identified by static code analysis.
- If the description is incomplete or the tests show problems, go back to the codebase and add e.g. missing annotations.
Summary and Conclusion
The landscape of API generators is quite heterogeneous in purpose, the need for manual work and usefulness tradeoff. Some generators are commonly used and can be integrated quite unhesitatingly, as they also can be easily replaced again by manual work. Others require more careful consideration.
The most common motivations among the generator types is development speed through automation, followed by developer experience by simplifying the development process and reducing cognitive load. Although user experience is not a common direct motivation, it can be improved indirectly by improving the developer experience, resulting in a better product. However, the motivations described in the above tables are not guaranteed to pay out. The challenges of the generator use can diminish the positive effects, e.g. a very buggy and poorly documented tool can negatively impact developer experience and speed more than it helps. In practice, it is important to question the results:
- How much time does the tool actually save?
- Did the developer experience actually improve?
Contrary to popular belief, using a generator does not necessarily make the process straightforward. Some generators require additional manual input, which can be quite time-consuming (see server code generation and external data model to description generation). If the generator tool fails or reaches its limit, the overhead can become uneconomical, requiring cumbersome manual fixes or inefficient workflows to compensate for the tool’s issues. The time saved may not be as significant as initially thought when the project is in its early stages and the requirements are simple.
It is also a myth that developers don’t need to know much about the structure and content of an API description to obtain a high-quality interface description when using a description generator. Description generation often relies on the provision of additional annotations. This applies to both generating descriptions from server code and from an external data model. The developers must understand what information the final description should contain in order to set all the necessary annotations. Consequently, if developers do not care about API description quality when writing the generator's input, or are not even familiar with API descriptions, the quality of the generated API will be poor.
While some challenges of generator use lie within the developer’s mindset and skill, many are also determined by the maturity of the tool, e.g. the functionality span, bug frequency and resolution time, update frequency, support and reliable continuation. Therefore, if a generator is chosen for an API project, the actual tool is as important as the generator type.
In summary, API generators are not a panacea, but they can facilitate the creation of maintainable and consistent interfaces. It is important to recognise them as a tool within the broader design process, rather than as a substitute for careful API design.
Was this post helpful?
More articles
fromMirabell Büscher
15.8.2024 More articles in this subject area
Discover exciting further topics and let the codecentric world inspire you.
Blog author
Mirabell Büscher
Do you still have questions? Just send me a message.
Do you still have questions? Just send me a message.