Runs

The Runs view provides a detailed overview of all your test runs and their results. This page helps you track the execution and outcomes of individual test cases.

Column Descriptions

The Runs view displays several columns that provide important information about each test run:

• env: The environment (windows, android, iOS, console) on which the test was executed.
• build: Hyperlink to the exact build this test run was run on.
• <>: Lets you expand/collapse a test run to see all the different steps of that test.
• actions: You can either click the recording icon to see an in-depth view of what happened in the run, or click on the report for a brief text summary of the important events in this run, or clieck the details button for additional high level run information.
• state: The run’s final state combined with a bug assessment (e.g., success(likely bug)). See Run State Result section below.
• test: The name of the test case that was executed.
• started: Time when the test execution began.
• started by: Who initiated the test execution.
• duration: How long the test execution took to complete.
• tags: Labels assigned to the test for categorization and filtering.

Run State Result

The state column shows how the run ended, followed by a bug assessment label in parentheses.

The run state indicates the completion status:

• success: The run finished in time and reached the given goal.
• max_steps: The agent didn’t reach the goal and ran out of steps. This could mean:
  1. There wasn’t enough time to complete the task.
  2. The agent had trouble fulfilling the task, which indicates poor agent performance or the presence of a bug.
• error: An error occurred, ranging from a game crash to our systems breaking.
• stopped: The run was stopped by the user.

The bug assessment label (in parentheses) indicates how likely the AI Agent thinks a bug was encountered during the run: no_bug, likely_no_bug, uncertain, likely_bug, or def_bug.

Recording

You can check exactly what happened in a run by clicking on the [ recording ] button. The recording view provides a comprehensive playback of the test execution, allowing you to analyze every action, decision, and outcome in detail.

The recording interface is organized into several key sections that provide different perspectives on the test execution.

Game View

Shows you exactly what the game looked like at each moment during the test run.

Test Details

This section displays the goal and expected results for the currently active test step. Expand the test steps to view details for each step in the test case.

AI Agent (Disco, Veri, Summary, Report)

Get insight into the AI Agent’s reasoning and actions. Here, you can review exactly what the AI Agent was thinking (its reasoning) and the actions it took at each step.

Game State

• hint_key: The currently active hint key. See the GameState documentation for more details.
• can_interact: Indicates whether the AI Agent is currently allowed to interact with the game. Learn more.
• game_connected: Shows if the SDK is connected to the game. This should be checked if you expect the AI Agent to interact with your game (not just black-box testing).
• available_funcs: Lists all game functions currently available to the AI Agent.
• game_logs: Displays logs sent from the game (via the SDK) to the AI Agent during the current turn.

Context

• hints: All available hints. If you are using a hint key and a regex, this view shows only the filtered hints.
• chat_message: Displays any messages sent by a human via live chat during the run. More info
• agent_message: In multiplayer tests, AI Agents can broadcast messages to each other. These messages are shown here.

Timeline

At the bottom of the recording view, you’ll find a detailed timeline of the run. Use the left and right arrow keys to navigate between different AI Agents.