Beliebte Suchanfragen

Cloud Native



Agile Methoden



Charge your APIs Volume 3: Optimizing API Testing with Contract Testing

24.5.2023 | 5 minutes of reading time

API testing is a crucial part of the development process that ensures the functionality, reliability, and performance of the API. Testing helps to identify and resolve errors early on, which translates into reduced development costs and improved customer satisfaction. In "Charge your APIs Volume 3 - API Testing," we'll explore contract testing and tools that can help you validate your API and ensure it meets the requirements of your customers. Whether you're new to API testing or an experienced developer, this guide provides valuable insights and practical tips to help you optimise your testing process and create high-quality APIs that deliver optimal performance and user experience.

Introducing Contract Testing

Contract testing is a relatively new strategy in API testing, but it has gained popularity in recent years due to its ability to ensure API functionality and mitigate risks early on in the development process. Unlike other API testing strategies, contract testing focuses on testing the API contracts or agreements between the client and server instead of testing the application's functionality.

Contract testing involves creating and maintaining a contract that defines the expected behaviour of the API. This contract describes what endpoints should be available, what parameters should be accepted, and what responses should be returned. Once the contract is defined, contract testing tools can verify whether the API meets the requirements of the contract.

Compared to other API testing strategies, such as functional testing and integration testing, contract testing is faster and easier to implement. It ensures that the API's contract is met, which is an essential aspect of API testing. Contract testing tools also provide developers with valuable feedback that can help them quickly identify and resolve any issues that arise.

While contract testing is an excellent addition to your API testing strategy, it's not a replacement for functional and integration testing. These testing strategies complement each other, and using them together can provide a comprehensive testing process that ensures the functionality, reliability, and performance of your API.

Testing of Spectral Rulesets

After the short introduction, let's take a look at contract testing in practice. But first, let's take a look at the guidelines and rulesets from last week. In the process described, we write the rulesets on the basis of the guidelines. But so far we have not had the opportunity to put the rulesets under test. Phil Sturgeon already provides a good solution for this based on Jest. You can find the link in the reference below.

Technology overview

Now we continue with contract testing. The term Consumer Driven Contract Tests is often found in the literature. It is precisely this consumer-driven approach that I see as a problem, since compliance with a contract should not be viewed solely from the consumer's point of view. Instead, in terms of product thinking, it must be seen as the most important feature of an API from the producer's point of view. A contract is based on trust.

Nothing should have the effect of compromising this trust. Because of this, an API provider must ensure that this does not happen if there is a small adjustment that can lead to a change.

When we look at tools, we are doing this from the perspective of a provider. Here is an overview of tools that might be useful.

  • Pact
  • Rest Assured
  • Karate
  • Schemathesis
  • Portman
  • Specmatic

As a provider, we would now like to bring the contract to the test phase with as minimal resources as possible. This assumption already drastically reduces the number of possible tools.

  • Portman CLI
  • Specmatic

Accelerate by using Postman CLI

The focus of this post is on Portman CLI, as we mainly use this within API enablement projects. In order to now carry out the first test with Portman CLI and our test API, we still need to bring a little more tooling power on board. We are still missing a mock server. Here we are now using Prism from Stoplight.

1❯npx @stoplight/prism-cli mock test-api.yaml -p 8080
2› [CLI] …  awaiting  Starting Prism…
3› [CLI] ℹ  info      GET
4› [CLI] ▶  start     Prism is listening on

With this, we have now started a mock server. But we stop it directly. The contract test should now run locally against the mock server so that the provider can see that the requirements of the consumer side have been implemented accordingly.

1❯npx @stoplight/prism-cli mock test-api.yaml -p 8080 
2| npx @apideck/portman -l test-api.yaml -b -n

In the command line we can see that parallel to the mock server Portman CLI is also started with the following parameters.

-lfetches local copy of the API definition
-bchanges the base url
-nstarts Newman tests for the created Postman Collection

These parameters now lead to the following result.

1❯npx @stoplight/prism-cli mock test-api.yaml -p 8080 | npx @apideck/portman -l test-api.yaml -b -n
3 Local Path:  		test-api.yaml
4 Portman Config:  	portman-config.default.json
5 Postman Config:  	postman-config.default.json
6 Environment:  		.env
7 Inject Tests:  	true
8 Run Newman:  		true
9 Newman Iteration Data: false
10 Upload to Postman:  	false
12  ✔ Conversion successful
14 Run Newman against:
18Testing API
22  GET [200 OK, 261B, 24ms]
23  ✓  [GET]::/test - Status code is 2xx
24Collection run completed.
27│                         │         executed │           failed │
29│              iterations │                1 │                0 │
31│                requests │                1 │                0 │
33│            test-scripts │                1 │                0 │
35│      prerequest-scripts │                0 │                0 │
37│              assertions │                1 │                0 │
39│ total run duration: 46ms                                      │
41│ total data received: 0B (approx)                              │
43│ average response time: 24ms [min: 24ms, max: 24ms, s.d.: 0µs] │
46🚀 Collection written to: ./tmp/converted/testingApi.json 🚀

In this case, a successful contract test leads to the creation of a postman collection via Portman CLI. Finally, this is tested with the parameter -n using Newman, which is integrated in Portman.


As we have seen, contract testing is a quick way to fundamentally test a service based on an API. Especially when using a tool like Portman CLI, which allows this by configuration. The test can then be done both on the basis of a mock, as shown, or by using the real service. The design of APIs can be very complex in itself, as we will see. So we need simple means to help make routine tasks like testing easier. And this is exactly what leads us towards automating such a process, which we started last week by creating guidelines.

I'm looking forward to next week when we take a look at Continuous Integration for APIs. See you then!


Testing Spectral Style Guides with Jest

GitHub - apideck-libraries/portman: Port OpenAPI Specs to Postman Collections, inject test suite and run via Newman 👨🏽‍🚀

Schemathesis: Property-based testing for API schemas — Schemathesis 3.19.2 documentation


share post




More articles in this subject area

Discover exciting further topics and let the codecentric world inspire you.


Gemeinsam bessere Projekte umsetzen.

Wir helfen deinem Unternehmen.

Du stehst vor einer großen IT-Herausforderung? Wir sorgen für eine maßgeschneiderte Unterstützung. Informiere dich jetzt.

Hilf uns, noch besser zu werden.

Wir sind immer auf der Suche nach neuen Talenten. Auch für dich ist die passende Stelle dabei.