In his formative post from 2007, Martin Fowler describes the differences between stubs, mocks and fakes - all types of Test Doubles, and goes on to discuss the two archetypical TDD schools, dubbed classicist and mockist, or 'Detroit School' and 'London School'. According to Fowler, classicists prefer using the real objects wherever possible, while mockists will mock all dependencies when testing behavior. As a result, classicist tests will tend to be more integrative, but also run longer, because they rely on real implementations, often IO-bound, while mockist tests will run faster but suffer from coupling to implementation details, because they test the internal behavior of the System Under Test rather than the effects of its behavior.
I tend to take a third approach, preferring to write what Dave Farley dubs Acceptance Tests. Tests should have larger scope, asserting business value (features), rather than specific implementation, but they must run as fast as possible to facilitate a short feedback cycle so that developers know early if they broke existing behavior. This type of tests provide the best value-for-money, since they can often be written as progression tests (test-first TDD) rather than regression, and they allow us to change our systems with more ease and confidence, since they aren't coupled to concrete implementation details. The way to achieve this seemingly impossible holy grail of fast, integrative tests is by relying on Fakes.
Fakes Aren't Mocks
Fakes are a type of test double that behaves exactly like the real thing, but are implemented differently. Consider a Data Access Object or Repository, implemented on top of a database:
interface ProductRepository {
findById(productId: string): Promise<Product | undefined>;
create(template: ProductTemplate): Promise<Product>;
}
class MongoDBProductRepository {
private products: Collection<ProductTemplate>;
constructor(db: Db) {
this.products = db.collection("products");
}
async create(fields: ProductTemplate): Promise<Product> {
const id = new ObjectId();
await this.products.insertOne({_id: id, ...fields});
return {
id: id.toString(),
...fields,
}
}
async findById(id: string): Promise<Product | undefined> {
return this.products.findOne({_id: {
$eq: new ObjectId(id)}
}).then(doc => doc && docToProduct(doc))
}
}
It's easy to imagine an in-memory implementation of the same interface:
class MemoryProductRepository {
private products: Product[] = []
constructor(initial: ProductTemplate[] = []) {
initial.forEach(p => this.create(p));
}
async findById(id: string): Promise<Product | undefined> {
return this.products.find(product => product.id === productId);
}
async create(template: Omit<Product, "id">): Promise<Product> {
const product = {...template, id: nanoid()};
this.products.push(product);
return product;
}
}We write this class just once, it's reusable throughout the system, and whenever we want to test code that needs Products, we simply instantiate the system, injecting it with the in-memory implementation instead of the real one - easily done using Hexagonal Architecture. Moreover, unlike a mock, the in-memory fake is internally coherent, fully implementing the API. Whenever I call productRepo.create, I can safely assume that the created product will be returned when I call productRepo.findById.
Using Hexagonal Architecture and in-memory fakes allows us to write tests that encompass as large a part of the system as possible, but that still run in-process, and as such are only memory- and CPU-bound. If our codebase happens to be a TypeScript monolith, we can use Testing Library to write JSDOM-based integrative tests that run in milliseconds, but go all the way to the backend and back:
test("Product search is case-insensitive", async () => {
const moogOne = aProduct({title: "Moog One"});
const minimoog = aProduct({title: "Minimoog"});
const ob8x = aProduct({title: "OB 8x"});
const productRepo = new MemoryProductRepository([
moogOne, minimoog, ob8x]);
const { runInHarness } = await makeApp(productRepo);
await runInHarness(async (app) => {
await userEvent.type(
app.getByPlaceholderText('Search products'), 'moog');
await userEvent.click(
app.getByLabelText('Search'));
expect(app.queryByText(moogOne.title)).toBeInTheDocument();
expect(app.queryByText(minimoog.title)).toBeInTheDocument();
expect(app.queryByText(ob8x.title)).not.toBeInTheDocument();
});
});In the above example, the makeApp function runs a new Express server on a random port, using in-memory fakes, and returns the runInHarness function, which in turn renders the UI application using Testing Library, shutting done the server when the test terminates. Each test creates its own memory-bound system, inserting its own dataset, rather than relying on shared fixtures. As such, these tests are completely independent of each other, can run in parallel, and can safely be changed without affecting other tests.
What Is This Mockery?
At this point, you might be wondering what's the big deal? why not just keep using mocks?
First and foremost, mock programming tends to be repetitive. Each test needs to reprogram the mock using cumbersome syntax:
const productRepo = {
findById: jest.fn(),
create: jest.fn(),
}
productRepo.findById.mockReturnValue([moogOne, minimoog, ob8x]);This results in fragile code - this implementation of the ProductRepository interface is incomplete and violates the principle of least surprise. In addition, due to (mis)use of structural typing, when the interface changes we will have to change multiple sites of setup code.
Since this is tiresome and repetitive, developers are often driven towards reusing setup code, for instance in beforeEach() setup functions. While shared setup code (or shared fixtures) is a DRYer solution, it creates a bigger problem: now our tests are implicitly coupled to each other. When we need to change setup code for one test, we might inadvertently affect other tests. Mature systems with such tests often have long and convoluted fixtures, that only keep growing due to fear of change.
Finally, mocks are designed to allow us to assert that certain behavior has occurred. In acceptance testing, it is preferable to assert an effect (a product has been added to the repository) rather than behavior (the productRepo.create function has been called with the specified product), because relying on effects allow us to change the implementation without changing the tests.
Mocks are still useful, though. I use them with caution, whenever I find myself writing unit or Component Tests. A good example would be testing a registration form that takes a submit function as a dependency. The happy path (valid form) is tested in an acceptance tests, while concrete validation issues (password mismatch, weak password, missing fields, etc.) can be tested in an isolated component test that takes a mocked submit function.
In Fakes We Trust
But how can we trust that our fakes behave exactly like the real implementations? I like to use Contract Tests that run against both the fake and the real implementation, and assert that the both conform to the same interface:
const repositories = [
["mongodb", async () => {
const mongo = await new MongoClient(mongoUrl).connect();
const repo = new MongoDBProductRepository(mongo.db());
return {
repo,
close: mongo.close.bind(mongo)
}
}],
["memory", async () => ({
repo: new MemoryProductRepository(),
close: () => {},
})]
]
describe.each(repositories)('%s product repo', (_, makeRepo) => {
it('finds a product by id', async () => {
const { repo, close } = await makeRepo();
const p1 = await repo.create(aProduct());
await expect(repo.findById(p1.id)).resolves.toEqual(p1);
return close();
});
}) If we make sure that the interface is covered with a comprehensive test suite, we can safely assume that the two implementations are equivalent. If a bug caused by inconsistencies between the fake and real implementation does slip by, we simply add another test case to this suite, thus guaranteeing that the bug will not recur.
Contract tests rely on externally running dependencies (for instance a MongoDB server), and are typically run in the same CI job as our E2E tests in order to make use of the test environment that has already been spawned for these tests.
When Fakes Aren't Enough
There are some types of systems where fast acceptance testing using contract-tested fakes is impractical or impossible. If our system is 95% integration between IO-bound systems, for instance infra-as-code solutions, it might not make any sense to write and maintain fakes - the cost of ownership of these test double would be tenfold the cost of maintaining the production codebase. In these cases, we often rely on a comprehensive E2E test suite, and write mockist tests to assert that our narrow layer of business logic works correctly.
Another example is when the IO-bound implementation is hardware-based, for instance talking to Bluetooth peripherals, or interacting with hardware sensors. In these cases, it might prove impractical to perform contract testing, since they would involve an intricate lab setup with physical machinery to externally trigger the sensors. In this case it might still make sense to write fakes to facilitate acceptance testing, but we might have to rely on manually asserting that the fake and real implementations conform to the same contract (for instance visually inspecting the code, performing manual QA, etc).
In Summary
Fakes are complete implementations of an interface, often used to replace IO-bound adapters with memory-bound doubles using Hexagonal Architecture, to facilitate Acceptance Testing. Contract Tests are used to assert that both implementation conform to the same interface. By writing a comprehensive suite of fast acceptance tests and a narrower (but still comprehensive) suite of contract tests, we can write classicist-type tests (large-scope, feature-oriented) while still enjoying the short feedback cycle offered by mockist-type tests. Instead of having an E2E suite that takes long minutes to run, we can have a suite with the same level of safety running in mere seconds, and a second suite of IO-bound contract tests that run in dozens of seconds at most.
