API testing is a crucial part of the development process that ensures the functionality, reliability, and performance of the API. Testing helps to identify and resolve errors early on, which translates into reduced development costs and improved customer satisfaction. In "Charge your APIs Volume 3 - API Testing," we'll explore contract testing and tools that can help you validate your API and ensure it meets the requirements of your customers. Whether you're new to API testing or an experienced developer, this guide provides valuable insights and practical tips to help you optimise your testing process and create high-quality APIs that deliver optimal performance and user experience.
Introducing Contract Testing
Contract testing is a relatively new strategy in API testing, but it has gained popularity in recent years due to its ability to ensure API functionality and mitigate risks early on in the development process. Unlike other API testing strategies, contract testing focuses on testing the API contracts or agreements between the client and server instead of testing the application's functionality.
Contract testing involves creating and maintaining a contract that defines the expected behaviour of the API. This contract describes what endpoints should be available, what parameters should be accepted, and what responses should be returned. Once the contract is defined, contract testing tools can verify whether the API meets the requirements of the contract.
Compared to other API testing strategies, such as functional testing and integration testing, contract testing is faster and easier to implement. It ensures that the API's contract is met, which is an essential aspect of API testing. Contract testing tools also provide developers with valuable feedback that can help them quickly identify and resolve any issues that arise.
While contract testing is an excellent addition to your API testing strategy, it's not a replacement for functional and integration testing. These testing strategies complement each other, and using them together can provide a comprehensive testing process that ensures the functionality, reliability, and performance of your API.
Testing of Spectral Rulesets
After the short introduction, let's take a look at contract testing in practice. But first, let's take a look at the guidelines and rulesets from last week. In the process described, we write the rulesets on the basis of the guidelines. But so far we have not had the opportunity to put the rulesets under test. Phil Sturgeon already provides a good solution for this based on Jest. You can find the link in the reference below.
Technology overview
Now we continue with contract testing. The term Consumer Driven Contract Tests is often found in the literature. It is precisely this consumer-driven approach that I see as a problem, since compliance with a contract should not be viewed solely from the consumer's point of view. Instead, in terms of product thinking, it must be seen as the most important feature of an API from the producer's point of view. A contract is based on trust.
Nothing should have the effect of compromising this trust. Because of this, an API provider must ensure that this does not happen if there is a small adjustment that can lead to a change.
When we look at tools, we are doing this from the perspective of a provider. Here is an overview of tools that might be useful.
- Pact
- Rest Assured
- Karate
- Schemathesis
- Portman
- Specmatic
As a provider, we would now like to bring the contract to the test phase with as minimal resources as possible. This assumption already drastically reduces the number of possible tools.
- Portman CLI
- Specmatic
Accelerate by using Postman CLI
The focus of this post is on Portman CLI, as we mainly use this within API enablement projects. In order to now carry out the first test with Portman CLI and our test API, we still need to bring a little more tooling power on board. We are still missing a mock server. Here we are now using Prism from Stoplight.
1❯npx @stoplight/prism-cli mock test-api.yaml -p 8080 2› [CLI] … awaiting Starting Prism… 3› [CLI] ℹ info GET http://127.0.0.1:8080/test 4› [CLI] ▶ start Prism is listening on http://127.0.0.1:8080
With this, we have now started a mock server. But we stop it directly. The contract test should now run locally against the mock server so that the provider can see that the requirements of the consumer side have been implemented accordingly.
1❯npx @stoplight/prism-cli mock test-api.yaml -p 8080 2| npx @apideck/portman -l test-api.yaml -b http://127.0.0.1:8080 -n
In the command line we can see that parallel to the mock server Portman CLI is also started with the following parameters.
| Parameter | Description |
|---|---|
-l | fetches local copy of the API definition |
-b | changes the base url |
-n | starts Newman tests for the created Postman Collection |
These parameters now lead to the following result.
1❯npx @stoplight/prism-cli mock test-api.yaml -p 8080 | npx @apideck/portman -l test-api.yaml -b http://127.0.0.1:8080 -n 2============================================================================================================================================================================================================================================================= 3 Local Path: test-api.yaml 4 Portman Config: portman-config.default.json 5 Postman Config: postman-config.default.json 6 Environment: .env 7 Inject Tests: true 8 Run Newman: true 9 Newman Iteration Data: false 10 Upload to Postman: false 11============================================================================================================================================================================================================================================================= 12 ✔ Conversion successful 13============================================================================================================================================================================================================================================================= 14 Run Newman against: http://127.0.0.1:8080 15============================================================================================================================================================================================================================================================= 16newman 17 18Testing API 19 20❏ test 21↳ test 22 GET http://127.0.0.1:8080/test [200 OK, 261B, 24ms] 23 ✓ [GET]::/test - Status code is 2xx 24Collection run completed. 25 26┌─────────────────────────┬──────────────────┬──────────────────┐ 27│ │ executed │ failed │ 28├─────────────────────────┼──────────────────┼──────────────────┤ 29│ iterations │ 1 │ 0 │ 30├─────────────────────────┼──────────────────┼──────────────────┤ 31│ requests │ 1 │ 0 │ 32├─────────────────────────┼──────────────────┼──────────────────┤ 33│ test-scripts │ 1 │ 0 │ 34├─────────────────────────┼──────────────────┼──────────────────┤ 35│ prerequest-scripts │ 0 │ 0 │ 36├─────────────────────────┼──────────────────┼──────────────────┤ 37│ assertions │ 1 │ 0 │ 38├─────────────────────────┴──────────────────┴──────────────────┤ 39│ total run duration: 46ms │ 40├───────────────────────────────────────────────────────────────┤ 41│ total data received: 0B (approx) │ 42├───────────────────────────────────────────────────────────────┤ 43│ average response time: 24ms [min: 24ms, max: 24ms, s.d.: 0µs] │ 44└───────────────────────────────────────────────────────────────┘ 45============================================================================================================================================================================================================================================================= 46🚀 Collection written to: ./tmp/converted/testingApi.json 🚀 47=============================================================================================================================================================================================================================================================
In this case, a successful contract test leads to the creation of a postman collection via Portman CLI. Finally, this is tested with the parameter -n using Newman, which is integrated in Portman.
Conclusion
As we have seen, contract testing is a quick way to fundamentally test a service based on an API. Especially when using a tool like Portman CLI, which allows this by configuration. The test can then be done both on the basis of a mock, as shown, or by using the real service. The design of APIs can be very complex in itself, as we will see. So we need simple means to help make routine tasks like testing easier. And this is exactly what leads us towards automating such a process, which we started last week by creating guidelines.
I'm looking forward to next week when we take a look at Continuous Integration for APIs. See you then!
References
Testing Spectral Style Guides with Jest
Schemathesis: Property-based testing for API schemas — Schemathesis 3.19.2 documentation
Was this post helpful?
More articles
fromDaniel Kocot
16.4.2024 Your job at codecentric?
Jobs
Agile Developer und Consultant (w/d/m)
Alle Standorte
More articles in this subject area
Discover exciting further topics and let the codecentric world inspire you.
Gemeinsam bessere Projekte umsetzen.
Wir helfen deinem Unternehmen.
Du stehst vor einer großen IT-Herausforderung? Wir sorgen für eine maßgeschneiderte Unterstützung. Informiere dich jetzt.
Hilf uns, noch besser zu werden.
Wir sind immer auf der Suche nach neuen Talenten. Auch für dich ist die passende Stelle dabei.
Blog author
Daniel Kocot
Senior Solution Architect / Head of API Consulting
Do you still have questions? Just send me a message.
Do you still have questions? Just send me a message.