Skip to main content

Test suites

The Testing section of the Temporal Application development guide describes the frameworks that facilitate Workflow and integration testing.

In the context of Temporal, you can create these types of automated tests:

  • End-to-end: Running a Temporal Server and Worker with all its Workflows and Activities; starting and interacting with Workflows from a Client.
  • Integration: Anything between end-to-end and unit testing.
    • Running Activities with mocked Context and other SDK imports (and usually network requests).
    • Running Workers with mock Activities, and using a Client to start Workflows.
    • Running Workflows with mocked SDK imports.
  • Unit: Running a piece of Workflow or Activity code (a function or method) and mocking any code it calls.

We generally recommend writing the majority of your tests as integration tests.

Because the test server supports skipping time, use the test server for both end-to-end and integration tests with Workers.

Test frameworks​

Some SDKs have support or examples for popular test frameworks, runners, or libraries.

TypeScript has sample tests for Jest and Mocha.

Jest

Mocha

Testing Activities​

An Activity can be tested with a mock Activity environment, which provides a way to mock the Activity context, listen to Heartbeats, and cancel the Activity. This behavior allows you to test the Activity in isolation by calling it directly, without needing to create a Worker to run the Activity.

Run an Activity​

If an Activity references its context, you need to mock that context when testing in isolation.

First, create a MockActivityEnvironment. The constructor accepts an optional partial Activity Info object in case any info fields are needed for the test.

Then use MockActivityEnvironment.run() to run a function in an Activity Context.

import { activityInfo } from '@temporalio/activity';
import { MockActivityEnvironment } from '@temporalio/testing';
import assert from 'assert';

// A function that takes two numbers and returns a promise that resolves to the sum of the two numbers
// and the current attempt.
async function activityFoo(a: number, b: number): Promise<number> {
  return a + b + activityInfo().attempt;
}

// Create a MockActivityEnvironment with attempt set to 2. Run the activityFoo
// function with parameters 5 and 35. Assert that the result is 42.
const env = new MockActivityEnvironment({ attempt: 2 });
const result = await env.run(activityFoo, 5, 35);
assert.equal(result, 42);

Listen to Heartbeats​

When an Activity sends a Heartbeat, be sure that you can see the Heartbeats in your test code so that you can verify them.

MockActivityEnvironment is an EventEmitter that emits a heartbeat event that you can use to listen for Heartbeats emitted by the Activity.

When an Activity is run by a Worker, Heartbeats are throttled to avoid overloading the server. MockActivityEnvironment, however, does not throttle Heartbeats.

import { heartbeat } from '@temporalio/activity';
import assert from 'assert';

async function activityFoo(): Promise<void> {
  heartbeat(6);
}

const env = new MockActivityEnvironment();

env.on('heartbeat', (d: unknown) => {
  assert(d === 6);
});

await env.run(activityFoo);

Cancel an Activity​

If an Activity is supposed to react to a Cancellation, you can test whether it reacts correctly by canceling it.

MockActivityEnvironment exposes a .cancel() method that cancels the Activity Context.

import { CancelledFailure, heatbeat, sleep } from '@temporalio/activity';
import { MockActivityEnvironment } from '@temporalio/testing';
import assert from 'assert';

async function activityFoo(): Promise<void> {
  heartbeat(6);
  // @temporalio/activity's sleep() is Cancellation-aware, which means that on Cancellation,
  // CancelledFailure will be thrown from it.
  await sleep(100);
}

const env = new MockActivityEnvironment();

env.on('heartbeat', (d: unknown) => {
  assert(d === 6);
});

await assert.rejects(env.run(activityFoo), (err) => {
  assert.ok(err instanceof CancelledFailure);
});

Testing Workflows​

How to mock Activities​

Mock the Activity invocation when unit testing your Workflows.

When integration testing Workflows with a Worker, you can mock Activities by providing mock Activity implementations to the Worker.

Implement only the relevant Activities for the Workflow being tested.

import type * as activities from './activities';

// Creating a mock object of the activities.
const mockActivities: Partial<typeof activities> = {
  makeHTTPRequest: async () => '99',
};

// Creating a worker with the mocked activities.
const worker = await Worker.create({
  activities: mockActivities,
  // ...
});

How to skip time​

Some long-running Workflows can persist for months or even years. Implementing the test framework allows your Workflow code to skip time and complete your tests in seconds rather than the Workflow's specified amount.

For example, if you have a Workflow sleep for a day, or have an Activity failure with a long retry interval, you don't need to wait the entire length of the sleep period to test whether the sleep function works. Instead, test the logic that happens after the sleep by skipping forward in time and complete your tests in a timely manner.

The test framework included in most SDKs is an in-memory implementation of Temporal Server that supports skipping time. Time is a global property of an instance of TestWorkflowEnvironment: skipping time (either automatically or manually) applies to all currently running tests. If you need different time behaviors for different tests, run your tests in a series or with separate instances of the test server. For example, you could run all tests with automatic time skipping in parallel, and then all tests with manual time skipping in series, and then all tests without time skipping in parallel.

Set up time skipping​

Learn to set up the time-skipping test framework in the SDK of your choice.

npm install @temporalio/testing

The @temporalio/testing package downloads the test server and exports TestWorkflowEnvironment, which you use to connect the Client and Worker to the test server and interact with the test server.

TestWorkflowEnvironment.createTimeSkipping starts the test server. A typical test suite should set up a single instance of the test environment to be reused in all tests (for example, in a Jest beforeAll hook or a Mocha before() hook).

import { TestWorkflowEnvironment } from '@temporalio/testing';

let testEnv: TestWorkflowEnvironment;

// beforeAll and afterAll are injected by Jest
beforeAll(async () => {
  testEnv = await TestWorkflowEnvironment.createTimeSkipping();
});

afterAll(async () => {
  await testEnv?.teardown();
});

TestWorkflowEnvironment has client and nativeConnection for creating Workers:

import { Worker } from '@temporalio/worker';
import { v4 as uuid4 } from 'uuid';
import { workflowFoo } from './workflows';

test('workflowFoo', async () => {
  const worker = await Worker.create({
    connection: testEnv.nativeConnection,
    taskQueue: 'test',
    ...
  });
  const result = await worker.runUntil(
    testEnv.client.workflow.execute(workflowFoo, {
      workflowId: uuid4(),
      taskQueue: 'test',
    })
  );
  expect(result).toEqual('foo');
});

This test uses the test connection to create a Worker, runs the Worker until the Workflow is complete, and then makes an assertion about the Workflow's result. The Workflow is executed using testEnv.client.workflow, which is connected to the test server.

Skip time automatically​

You can skip time automatically in the SDK of your choice. Start a test server process that skips time as needed. For example, in the time-skipping mode, Timers, which include sleeps and conditional timeouts, are fast-forwarded except when Activities are running.

The test server starts in "normal" time. When you use TestWorkflowEnvironment.client.workflow.execute() or .result(), the test server switches to "skipped" time mode until the Workflow completes. In "skipped" mode, timers (sleep() calls and condition() timeouts) are fast-forwarded except when Activities are running.

workflows.ts

import { sleep } from '@temporalio/workflow';

export async function sleeperWorkflow() {
  await sleep('1 day');
}

test.ts

import { sleeperWorkflow } from './workflows';

test('sleep completes almost immediately', async () => {
  const worker = await Worker.create({
    connection: testEnv.nativeConnection,
    taskQueue: 'test',
    workflowsPath: require.resolve('./workflows'),
  });
  // Does not wait an entire day
  await worker.runUntil(
    testEnv.client.workflow.execute(sleeperWorkflow, {
      workflowId: uuid(),
      taskQueue: 'test',
    }),
  );
});

Skip time manually​

Learn to skip time manually in the SDK of your choice.

You can call testEnv.sleep() from your test code to advance the test server's time. This is useful for testing intermediate states or indefinitely long-running Workflows. However, to use testEnv.sleep(), you need to avoid automatic time skipping by starting the Workflow with .start() instead of .execute() (and not calling .result()).

workflow.ts

import { sleep } from '@temporalio/workflow';
import { defineQuery, setHandler } from '@temporalio/workflow';

export const daysQuery = defineQuery('days');

export async function sleeperWorkflow() {
  let numDays = 0;

  setHandler(daysQuery, () => numDays);

  for (let i = 0; i < 100; i++) {
    await sleep('1 day');
    numDays++;
  }
}

test.ts

test('sleeperWorkflow counts days correctly', async () => {
  const worker = await Worker.create({
    connection: testEnv.nativeConnection,
    taskQueue: 'test',
    workflowsPath: require.resolve('./workflows'),
  });

  // `start()` starts the test server in "normal" mode, not skipped time mode.
  // If you don't advance time using `testEnv.sleep()`, then `sleeperWorkflow()`
  // will run for days.
  handle = await testEnv.client.workflow.start(sleeperWorkflow, {
    workflowId: uuid4(),
    taskQueue,
  });

  worker.run();

  let numDays = await handle.query(daysQuery);
  assert.equal(numDays, 0);

  // Advance the test server's time by 25 hours
  await testEnv.sleep('25 hours');
  numDays = await handle.query(daysQuery);
  assert.equal(numDays, 1);

  await testEnv.sleep('25 hours');
  numDays = await handle.query(daysQuery);
  assert.equal(numDays, 2);
});

Skip time in Activities​

Learn to skip time in Activities in the SDK of your choice.

Call TestWorkflowEnvironment.sleep from the mock Activity.

In the following test, processOrderWorkflow sends a notification to the user after one day. The processOrder mocked Activity calls testEnv.sleep(‘2 days'), during which the Workflow sends email (by calling the sendNotificationEmail Activity).

Then, after the Workflow completes, we assert that sendNotificationEmail was called.

Workflow implementation

timer-examples/src/workflows.ts

export async function processOrderWorkflow({
  orderProcessingMS,
  sendDelayedEmailTimeoutMS,
}: ProcessOrderOptions): Promise<string> {
  let processing = true;
  // Dynamically define the timeout based on given input
  const { processOrder } = proxyActivities<ReturnType<typeof createActivities>>(
    {
      startToCloseTimeout: orderProcessingMS,
    },
  );

  const processOrderPromise = processOrder().then(() => {
    processing = false;
  });

  await Promise.race([processOrderPromise, sleep(sendDelayedEmailTimeoutMS)]);

  if (processing) {
    await sendNotificationEmail();

    await processOrderPromise;
  }

  return 'Order completed!';
}

timer-examples/src/test/workflows.test.ts

it('sends reminder email if processOrder does not complete in time', async () => {
  // This test doesn't actually take days to complete: the TestWorkflowEnvironment starts the
  // Test Server, which automatically skips time when there are no running Activities.
  let emailSent = false;
  const mockActivities: ReturnType<typeof createActivities> = {
    async processOrder() {
      // Test server switches to "normal" time while an Activity is executing.
      // Call `env.sleep` to skip ahead 2 days, by which time sendNotificationEmail
      // should have been called.
      await env.sleep('2 days');
    },
    async sendNotificationEmail() {
      emailSent = true;
    },
  };
  const worker = await Worker.create({
    connection: env.nativeConnection,
    taskQueue: 'test',
    workflowsPath: require.resolve('../workflows'),
    activities: mockActivities,
  });
  await worker.runUntil(
    env.client.workflow.execute(processOrderWorkflow, {
      workflowId: uuid(),
      taskQueue: 'test',
      args: [
        {
          orderProcessingMS: ms('3 days'),
          sendDelayedEmailTimeoutMS: ms('1 day'),
        },
      ],
    }),
  );
  assert.ok(emailSent);
});

Test functions in Workflow context​

For a function or method to run in the Workflow context (where it's possible to get the current Workflow info, or running inside the sandbox in the case of TypeScript or Python), it needs to be run by the Worker as if it were a Workflow.

note

This section is applicable in Python and TypeScript. In Python, we allow testing of Workflows only and not generic Workflow-related code.

To test a function in your Workflow code that isn't a Workflow, put the file it's exported from in WorkerOptions.workflowsPath. Then execute the function as if it were a Workflow:

workflows/file-with-workflow-function-to-test.ts

import { sleep } from '@temporalio/workflow';

export async function functionToTest(): Promise<number> {
  await sleep('1 day');
  return 42;
}

test.ts

const worker = await Worker.create({
  connection: testEnv.nativeConnection,
  workflowsPath: require.resolve(
    './workflows/file-with-workflow-function-to-test',
  ),
});

const result = await worker.runUntil(
  testEnv.client.workflow.execute(functionToTest, workflowOptions),
);

assert.equal(result, 42);

If functionToTest starts a Child Workflow, that Workflow must be exported from the same file (so that the Worker knows about it):

import { sleep } from '@temporalio/workflow';
import { someWorkflowToRunAsChild } from './some-workflow';

export { someWorkflowToRunAsChild };

export async function functionToTest(): Promise<number> {
  const result = await wf.executeChild(someWorkflowToRunAsChild);
  return result + 42;
}

Assert in Workflow​

The assert statement is a convenient way to insert debugging assertions into the Workflow context.

The assert method is available in Python and TypeScript.

The Node.js assert module is included in Workflow bundles.

By default, a failed assert statement throws AssertionError, which causes a Workflow Task to fail and be indefinitely retried.

To prevent this behavior, use workflowInterceptorModules from @temporalio/testing. These interceptors catch an AssertionError and turn it into an ApplicationFailure that fails the entire Workflow Execution (not just the Workflow Task).

workflows/file-with-workflow-function-to-test.ts

import assert from 'assert';

export async function functionToTest() {
  assert.ok(false);
}

test.ts

import {
  TestWorkflowEnvironment,
  workflowInterceptorModules,
} from '@temporalio/testing';

const worker = await Worker.create({
  connection: testEnv.nativeConnection,
  interceptors: {
    workflowModules: workflowInterceptorModules,
  },
  workflowsPath: require.resolve(
    './workflows/file-with-workflow-function-to-test',
  ),
});

await worker.runUntil(
  testEnv.client.workflow.execute(functionToTest, workflowOptions), // throws WorkflowFailedError
);

How to Replay a Workflow Execution​

Replay recreates the exact state of a Workflow Execution. You can replay a Workflow from the beginning of its Event History.

Replay succeeds only if the Workflow Definition is compatible with the provided history from a deterministic point of view.

When you test changes to your Workflow Definitions, we recommend doing the following as part of your CI checks:

  1. Determine which Workflow Types or Task Queues (or both) will be targeted by the Worker code under test.
  2. Download the Event Histories of a representative set of recent open and closed Workflows from each Task Queue, either programmatically using the SDK client or via the Temporal CLI.
  3. Run the Event Histories through replay.
  4. Fail CI if any error is encountered during replay.

The following are examples of fetching and replaying Event Histories:

To replay a single Event History, use worker.runReplayHistory.

When an Event History is replayed and non-determinism is detected (that is, the Workflow code is incompatible with the History), DeterminismViolationError is thrown. If replay fails for any other reason, ReplayError is thrown.

In the following example, a single Event History is loaded from a JSON file on disk (as obtained from the Web UI or the Temporal CLI):

const filePath = './history_file.json';
const history = await JSON.parse(fs.promises.readFile(filePath, 'utf8'));
await Worker.runReplayHistory(
  {
    workflowsPath: require.resolve('./your/workflows'),
  },
  history,
);

Alternatively, we can download the Event History programmatically using a Client:

const connection = await Connection.connect({ address });
const client = new Client({ connection, namespace: 'your-namespace' });
const handle = client.workflow.getHandle('your-workflow-id');
const history = await handle.fetchHistory();
await Worker.runReplayHistory(
  {
    workflowsPath: require.resolve('./your/workflows'),
  },
  history,
);

To gain confidence that changes to a Workflow are safe to deploy, we recommend that you obtain Event Histories from the relevant Task Queue and replay them in bulk. You can do so by combining the Client.workflow.list() and worker.runReplayHistories() APIs.

In the following example (which, as of server 1.18, requires Advanced Visibility to be enabled), Event Histories are downloaded from the server and then replayed by passing in a client and a set of Workflows Executions. The results returned by the async iterator contain information about the Workflow Execution and whether an error occurred during replay.

const executions = client.workflow.list({
  query: 'TaskQueue=foo and StartTime > "2022-01-01T12:00:00"',
});
const histories = executions.intoHistories();
const results = Worker.runReplayHistories(
  {
    workflowsPath: require.resolve('./your/workflows'),
  },
  histories,
);
for await (const result of results) {
  if (result.error) {
    console.error('Replay failed', result);
  }
}