What is Contract Testing?

Overview

To provide an ideal user experience, developers need to ensure that software services can interact according to specific basic requirements, such as responding to a request within a given timeframe. Contract testing helps developers ensure that services meet these kinds of goals. Using contract testing, teams can automatically validate whether interactions between services conform with predefined terms.

What is contract testing?

Contract testing is a type of software test methodology that assesses interactions between software services, with the goal of confirming that they meet the terms of a “contract.”

In this context, a contract refers to a set of rules that define how services should interact. For example, a contract could specify which types of responses a server can send to a client. Contracts can also define performance objectives, like completing a response within a certain number of milliseconds.

The importance of contract testing

Many modern applications consist of discrete services that need to interact with each other on a constant basis. If the interactions don’t happen as intended, the applications will either fail to work at all, or will fall short of expected performance.

For example, a microservices-based, cloud-native application app might include (among other components) one service that renders the frontend and another service that fetches data from a database. In order to fetch data that will appear on the frontend, the frontend service needs to send requests to the backend service, which will collect the data and send it back to the frontend service. If the backend data were to arrive in a format that the frontend service can’t process, the application would experience an error and may fail to render a user interface. Or, if the backend data takes a long time to reach the frontend service, it could cause the app to load slowly, resulting in a poor user experience.

Contract testing helps to prevent issues like these by automatically evaluating software service interactions and determining whether they meet the terms of a predefined contract. That way, developers can quickly confirm that software components are able to communicate effectively and reliably.

Contract testing is all the more valuable given that software services can change frequently – especially in projects that embrace a continuous delivery philosophy, in which developers frequently implement updates and deploy them to production environments. With contract testing, a development team can easily and efficiently run tests prior to deployment to confirm that changes to one service don’t cause the service to fail to interact properly with another service.

How does contract testing work?

Contract testing generally follows this series of steps:

1. Identify services to test: First, a development team chooses which services it wants to test by identifying which services commonly interact.
2. Define a contract: Using code, developers define the terms of the software contract. These terms determine exactly which service interactions to evaluate.
3. Execute the test: With the contract defined, the team can execute the test by generating service requests or responses in a test environment and evaluating how they respond.

A number of tools are available to assist in the process of defining contract code and executing tests. Some popular solutions are discussed below, in the section on how to implement contract testing.

Contract testing vs. integration testing

Another common type of test that developers run to evaluate how well services interact is integration testing.

Contract testing and integration testing are similar in that they are both ways of assessing service communications. However, in most cases, integration testing involves running end-to-end tests. This means that developers deploy a complete application instance, which could include many discrete services. Then, they generate requests to test how well the collection of services is able to handle them.

In contrast, contract testing typically tests interactions between just two specific services. Compared to integration testing, contract testing offers advantages such as:

• Simplicity: Contract testing doesn’t require the deployment of a complete application instance. Teams only need to deploy the specific services they want to evaluate.
• Speed: Because contract tests only evaluate interactions between two services, they typically generate results faster than integration testing.
• Clearer root-cause analysis: By testing just two services, it’s easier to pinpoint the source of a problem, such as a slow response to a certain type of request. With end-to-end integration testing, root causes are not always readily evident because they could arise from any component within an application.
• Ease of maintenance: Contract tests are easier to maintain and update because they have a narrower scope. End-to-end integration tests may require maintenance every time any part of an application changes – and because applications change frequently, developers end up having to update integration tests frequently.

This is not to say that developers should not use integration tests, or that contract tests can fully replace end-to-end integration testing. The latter is useful as a way of evaluating how end-users will experience an application as a whole. As such, end-to-end integration tests often run during the later stages of software development, shortly before deploying an application to production. In contrast, contract testing offers a simpler, faster way of testing interactions between specific parts of an application, making it a valuable means of identifying bugs earlier in the development process.

Contract testing vs. API testing

Contract testing is also comparable in many ways to API testing. In fact, some people consider contract testing to be synonymous with API testing, or at least a subset of it. However, there are some important differences between contract tests and other types of API tests.

API tests evaluate the behavior and performance of Application Programming Interfaces, usually by assessing how an API server responds to a request from a client. Since APIs govern the way services interact, API testing is critical for ensuring that services can communicate with each other effectively.

Contract testing serves a similar purpose in that it also evaluates the interactions of services that, in most cases, communicate through APIs. A difference between these testing methodologies, however, is that contract testing usually focuses only on confirming that services interact in a way that aligns with compatibility and performance goals. API testing is broader in that it can also test for API security risks.

In addition, contract testing sometimes relies on mock API interactions. This means that instead of generating a request to a live API client and viewing the resulting response, contract tests simulate what a request or response would look like, based on the design specifications of an API. API mocking makes it possible to perform contract tests even if developers haven’t fully implemented an API. API testing can take advantage of mocking as well, but it’s more common for API tests to evaluate “real” API interactions.

Types of contract testing

There are two main approaches to contract testing: Consumer-driven and provider-driven.

Consumer-driven tests

In a consumer-driven contract test, the service consumer (meaning the service that generates requests and sends them to another service) is responsible for defining contract terms and determining whether the responding service (known as the provider) meets them.

Consumer-driven testing is beneficial in projects where consumers change more frequently than providers. In that case, developers can run consumer-driven contract tests every time they update a consumer service to ensure that the changes have not introduced compatibility issues with the provider. They can also update the service contract along with the service itself, based on any new requirements introduced by the service changes.

Provider-driven tests

In provider-driven contract testing, the service that receives and responds to requests (the provider) defines contract terms. This approach is most useful when provider code changes more frequently than consumer code; in that case, the provider-driven testing offers a convenient way of validating that an updated provider can still interact with consumers as required.

Provider-driven testing can also help to ensure backward compatibility in systems where a provider needs to interact with multiple versions of a consumer. Using provider-driven testing, developers can confirm that each consumer version interacts properly with the latest version of the provider.

How to implement contract testing

To implement contract testing, teams must first choose a contract testing tool. Popular open source options include:

• Pact: Pact uses a consumer-driven testing approach with support for a variety of popular programming languages.
• Spring Cloud Contract: Spring Cloud Contract focuses on provider-driven testing. It’s Java-centric, and allows developers to define tests using Groovy or YAML.
• Dredd: Dredd enables automated contract testing by assessing API documentation and testing service interactions to validate whether they conform with the behavior defined in the documentation.
• RestAssured: RestAssured is a Java library that can test interactions between services written in Java that use RESTful APIs.
• OpenAPI: OpenAPI (formerly Swagger) is not a contract testing tool per se. It’s a specification format that can define how services should interact. OpenAPI can be used to generate contracts, which developers can then test using a tool like Pact or Dredd.

Contract testing and JFrog

By storing and managing contract tests generated via specialized tools, JFrog makes it easy to integrate contract testing into the software development lifecycle. To learn more, request a demo.