Macros¶

Structure-Level¶

ATC_SUITE(name) { ... }¶

Suites provide namespace-based grouping. They can be nested.

ATC_SUITE("MySuite") {
  ATC_SUITE("NestedSuite") {}
}

ATC_TEST(name) { ... }¶

ATC_TEST(name, CLIENTS(n)) { ... }¶

Defines a test case with an optional number of clients.

When no clients are specified, the number of clients will be determined automatically based on what the task requires.

The test is registered with the Automation Test Framework and can be executed through the Session Frontend window, AutomationTool tool, or ATO.

ATC_TEST(MyFirstTest) { ... }

ATC_COORDINATOR(mask)¶

ATC_COORDINATOR_OVERRIDE(mask)¶

Sets the coordinator mask for the current scope. The coordinator mask determines which coordinator will be used for the test.

ATC_COORDINATOR_OVERRIDE is used within the test scope to override the coordinator mask for a specific test case.

Supported EATCCoordinatorMode types are Dedicated, ListenServer, and Standalone.

ATC_PLAN(planName) { ... }¶

A reusable set of tasks. Can be called by any task at or below its current scope and behaves like an expansion.

ATC_PLAN(LoginFlow) {
  ATC_TASK(Connect);
  ATC_TASK(Authenticate);
}

ATC_RUN_PLAN(planName)¶

Imports the tasks in place – like an include.

ATC_TEST(PrepareWorld) {
  ATC_RUN_PLAN(LoginFlow);
}

Tasks¶

Tasks execute in order of declaration within the test, unless the task is waiting on an explicitly named dependency.

Each task must wait for the task before it to finish executing before it may execute.

Each task will be implicitly dependent on the task before it, unless explicitly set through ATC_DEP(taskName).

ATC_TASK(name, [dependencies...]) { ... }¶

Defines a task that runs on the coordinator.

ATC_PARRALEL {
  ATC_TASK(Initialize);
  ATC_TASK(StartMetrics);
  ATC_TASK(SpawnPlayer, ATC_DEP(StartMetrics), ATC_DEP(Initialize));
}

ATC_LOCAL_PLAYER_TASK(name, [dependencies...]) { ... }¶

Defines a task that runs on the local player.

Only runs when the process is Standalone, Listen, or Client.

ATC_CLIENT_TASK(name, [dependencies...]) { ... }¶

Defines a task that runs on the external client by index 0. Equivalent to ATC_CLIENT_TASK(0, name, [dependencies...]) { ... }

ATC_CLIENT_TASK_FOR(clientIndex, name, [dependencies...]) { ... }¶

Defines a task that runs on the external client by index clientIndex.

ATC_CLIENT_TASK_ALL(name, [dependencies...]) { ... }¶

Defines a task that runs on all external clients simultaneously.

The next task won't execute until all external clients have completed their tasks.

ATC_DEP(taskName)¶

Declares a dependency on another task. ATC_TASK(B, ATC_DEP(A));

SETUP_WORLD and TEARDOWN_WORLD¶

SETUP_WORLD and TEARDOWN_WORLD run on every process. Each process runs these blocks at the same time. The coordinator won't begin the test until each process, including itself, finishes the setup. Likewise, the coordinator won't complete the test until after each process finishes TEARDOWN_WORLD.

These blocks are optional.

ATC_TEST(MyTest) {
  SETUP_WORLD {
    // Tasks here (e.g. ATC_TASK, ATC_CLIENT_TASK)
  }

  ATC_TASK() { ... }

  TEARDOWN_WORLD {
    // Tasks here
  }
}

Parallel Task Execution¶

ATC_PARALLEL { ... }¶

Tasks declared inside this block do not depend on each other to begin execution. Internally, they depend on the task that precedes the ATC_PARALLEL block.

Order of execution is not guaranteed – even on the same target process.

ATC_PARALLEL {
  ATC_TASK(A);
  ATC_TASK(B);
}

Control Flow¶

ATC_SUCCESS()¶

Mark the task as successful.

ATC_FAIL_MSG(message)¶

ATC_FAIL()¶

Immediate fatal failure of the task and test, with optional message.

ATC_SKIP(?message)¶

Skip the current test and continue with the next test, with optional message.

ATC_SKIP_MANY(count, ?message)¶

Skips the current run plus additional future scheduled runs for the same test invocation. A count=0 does nothing and the current run continues.

ATC_SKIP_ALL(?message)¶

Skips the current test and all repeats.

ATC_SKIP_TASK(?message)¶

Skips the current task, with an optional message. This voids any ATC_TASK_RETRY configuration.

Options¶

Most options are settable at any suite level or test level, and their values are inheritable unless overriden.

Any SERVER variants will inherit from their nearest non-server variant if no server override is provided. Because the coordinator always the server, you should never need to use SERVER variants.

All options are optional, and defaults will be used when no value is supplied.

General Options¶

MAP_OPTIONS(options)¶

MAP_OPTIONS_SERVER(options)¶

Specifies options for the map world. MAP_OPTIONS("bUseSplitScreen=1");

GAME_MODE(path)¶

GAME_MODE_SERVER(path)¶

Sets the game mode for the test. GAME_MODE("/Game/Blueprints/BP_GameMode");

GAME_INSTANCE(path)¶

Sets the game instance.

World Type¶

Each test can specify what kind of environment to use for testing. Do you want to open a map for the test, or run a simple barebones actor world where you can place your own actors into?

MAP_WORLD(path)¶

Specifies the path to the map world file to be used for testing. MAP_WORLD("/Game/Maps/TestMap");

MAP_WORLD_SERVER(path)¶

Specifies the path for the server.

Note: MAP_WORLD_SERVER inherits from MAP_WORLD. When running as a server, they will always be the coordinator and setting MAP_WORLD_SERVER is *superfluous.

ACTOR_WORLD()¶

Creates a minimal actor-only world.

Test Options¶

ATC_TIMEOUT(seconds)¶

Sets the amount of time for the test to complete. Exceeding the timeout will result in the test to fail.

ATC_REPEAT(count)¶

Repeats the test count times. ATC_REPEAT(3) will run the test count + 1 times. ATC_REPEAT will continue repeat execution regardless of the previous invocation's result.

If any invocation fails, the test is considered failed.

ATC_SKIP, ATC_SKIP_MANY, and ATC_SKIP_ALL modify how many times the test will be repeated.

ATC_REPEAT_UNTIL_FAIL(maxRuns)¶

Repeats the test until it fails upto maxRuns times. ATC_REPEAT_UNTIL_FAIL(10)

Test Parameters¶

Parameters define a matrix of values to be used for testing within each coordinator.

Each test internally holds the Clients parameter for the number of external clients, with a default value 0. ATC_PARAMETERS(Clients, 0). But this can be overridden by directly setting the value through a matrix.

ATC_PARAMETERS(name, ...)¶

Defines a matrix of values for testing within each coordinator. ATC_PARAMETERS("paramName", "paramValue", "paramValue2") like ATC_PARAMETERS( Speed, "Slow", "Fast");

ATC_MATRIX(...)¶

Enables matrix parameterization for tests.

ATC_MATRIX(
  ATC_PARAMETERS(Speed, "Slow", "Fast");
  ATC_PARAMETERS(Color, "Red", "Blue");
  ATC_PARAMETERS(Clients, 0, 1);
);

Results in 2x2x2=8 tests.

ATC_PARAM_TABLE(names, ...)¶

ATC_PARAM_TABLE(
  ATC_PARAM_NAMES(A, B, Clients),
  ATC_PARAM_ROW("1", "2", 0),
  ATC_PARAM_ROW("3", "4", 1)
);

Results in a matrix of 1+1=2 tests.

Task Options¶

Task options are placed inside the body of a task.

ATC_TASK(TaskName) {
  ATC_TASK_TIMEOUT(5.0f);
  ATC_TASK_RETRY(3);
}

ATC_TASK_TIMEOUT(seconds)¶

Sets the maximum number of seconds the task may run. If the task exceeds the timeout, it will fail.

ATC_TASK_RETRY(count)¶

Retries the task count times if it fails. Retry ends if tests were successful within count or if it fails after count retries. ATC_RETRY(3).

ATC_TASK_RETRY(count, delaySeconds)¶

Retries the current task count times if it fails, waiting delaySeconds between attempts.

Task Flow Control¶

ATC_WAIT(seconds)¶

Waits for the specified number of seconds before continuing execution.

ATC_WAIT_FRAMES(frames)¶

Waits for the specified number of frames (ticks) before continuing execution.

ATC_WAIT_UNTIL(condition)¶

Waits for the specified condition to become true before continuing execution. No timeout on this macro. The test has its own timeout to that can be overridden with ATC_TIMEOUT.

Asset Audit¶

ATC_ASSET_AUDIT(type)¶

ATC_ASSET_AUDIT(type, pathFilter)¶

Registers an asset type for audit.

ATC_ASSET_AUDIT(UStaticMesh, "/Game/Meshes") {
  const int32 NumTris = Asset->GetNumTriangles(0);
  ATC_EXPECT_LT(NumTris, 50000);
}