Best Practices
Write a test, inspect a run, and refine the points where the agent takes an unintended path. A reliable test gives the agent a clear destination and observable evidence of completion.
Write Observable Checks
For verification tests, list the expected results first. Each check should describe one fact that can pass or fail independently.
| Check | Example |
|---|---|
| Visible UI | The screen header reads SETTINGS. |
| State change | After saving, the username field displays Alex. |
| Variable outcome | The balance increases by the reward amount shown on screen. |
Choose checks that hold on every valid run. If a scenario needs a particular account or item, establish that prerequisite in setup steps.
Group Checks into a Route
Group checks on the same screen or during the same operation into one step. Write the actions needed to reach them in goal/instructions, then order the steps to follow a practical route through the app.
Use a visible endpoint, such as “Follow the tutorial until the Tutorial Complete banner appears.” Keep destructive actions, such as account deletion, at the end or in a separate test.
Group related steps into collections and reuse common setup with Shared Steps.
Put Guidance in the Right Place
- Instructions: Required actions, completion criteria, and explicit conditions for legitimately skipping a scenario.
- Expected results: Observable facts to verify, with one check per row.
- Hints: Navigation, recovery, and interaction guidance. Every situation you describe should include what to do, such as “If a news popup covers the menu, close it.”
- Knowledge: Detailed specs, reference images, or game mechanics. Attach knowledge files and name the relevant file in the instructions.
Recovery Hints
Describe the obstacle, what to do, and how to confirm recovery:
| Situation | Example hint |
|---|---|
| A click misses its target | If Upload stays enabled and no progress appears after clicking, locate the button again in the current view and retry. Confirm upload progress appears. |
| A saved document is absent from the first list read | Search for the exact document name and inspect the result after the list finishes loading. |
| A browser prompt covers the app | Dismiss the password-save prompt, then locate the app’s input or button again. |
Match file-handling hints to the deployment. On Windows, download the file on the Windows machine and attach it through the native file picker. On a browser deployment, use the browser’s upload tools to attach the sandbox file. After recovery, verify the original expected result.
If you help the agent through Live View, add the useful guidance to the saved test. For repeated input combinations, use templating.
Diagnose Inconsistent Runs
Compare a passing and failing run with the same test version, build, deployment config, agent, and template values. For multiplayer tests, also match the players’ deployment choices.
Inspect the affected step’s actions and screenshots in Run Details. Find where the paths diverge: a popup, a different starting state, an interaction that misses its target, or a check performed before the screen is ready.
Refine the instruction or hint around that observation. Prefer a visible readiness condition, such as “Wait until the Save button becomes enabled,” and confirm that the next interaction changes the screen as intended.
Run the revised test several times under the affected configurations and review the original checks again. Keep expected results tied to the required product behavior while refining the execution path.
For a detailed workflow with Nexus MCP tools, use the flaky test refinement skill .