Skip to Content
SDK Integration
Core ConceptsGame State

Writing a Good Game State

The game state system provides structured data about the current state of the game in JSON format. This data supplements the visual information and provides precise details that may not be visible or easily extracted from screenshots.

Framework Required Fields

Every game state implementation must include these two critical fields:

_can_interact (boolean)

Controls whether the agent can interact with the game. Should be false during:

  • Loading screens
  • Cutscenes
  • Dialog sequences (where no interaction is possible)
  • Menu transitions
  • Any non-interactive state

When _can_interact is false, the agent will poll until the state becomes interactive again. This ensures that the agent doesn’t attempt to perform actions when the game is not ready for interaction.

Make sure that this field is only set to true when the game is going to be interactive again without any interaction needed. Otherwise the agent will stall indefinitely!!

_hint_key (string)

Acts as a contextual identifier for the current game state. Used to filter Flayer functions and hints - functions/hints can have a hint regex that only exposes them when there’s a match with the hint key. This creates a limited action space/context that improves agent performance.

Examples: "OVERWORLD_DAY_FOREST", "IN_GAME_TRADER_MENU"

Designing Hint Keys

Hint keys use regex pattern matching to control when Flayer functions and hints become available to the agent. Create sophisticated patterns by combining different game states:

// Basic hint key using just the map name
GameState._hint_key = CurrentLevelName;
 
// Advanced hint key combining multiple states
GameState._hint_key = FString::Printf(TEXT("%s_%s_%s_%s"),
    *CurrentLevelName,
    Character->IsDead() ? TEXT("DEAD") : TEXT("ALIVE"),
    InCombat ? TEXT("COMBAT") : TEXT("PEACEFUL"),
    *CurrentMenuState
);
Common State Components

Build hint keys with these common state indicators:

  • Scene/Level: Current scene name, map name, zone name
  • Combat State: COMBAT, PEACEFUL, AGGRESSIVE, IN_STEALTH
  • Player State: ALIVE, DEAD, LOW_HEALTH, STUNNED
  • UI State: INVENTORY_OPEN, MENU_OPEN, NO_MENU, DIALOG_ACTIVE
  • Character/Role: WIZARD, WARRIOR, ROGUE, class names
  • Time/Environment: DAY, NIGHT, RAIN, INDOORS, OUTDOORS
  • Quest/Progress: QUEST_ACTIVE, IN_DUNGEON, BOSS_FIGHT
Example Hint Key Patterns
  • MainHub_NON_COMBAT_INVENTORY_OPEN_WIZARD: Applies only to wizards with open inventory in the main hub
  • ShooterMap_ALIVE_COMBAT: For combat situations in the shooter map
  • Hub_ALIVE_PEACEFUL_INVENTORY: When in the hub with inventory open

Regex pattern examples:

  • .*INVENTORY_OPEN_WIZARD.*: Matches all wizard inventories regardless of location
  • MainHub_.*(WIZARD|WARRIOR): Defines hints for specific classes in the main hub
  • .*COMBAT_.*: Creates hints for any combat situation across all scenes
  • *_COMBAT_*: Matches any combat situation in any map

Custom Fields

Public Fields

These fields are exposed to the agent and form the core of its decision-making data. Design them to reduce the need for expensive visual operations or unnecessary game actions.

Some examples of public fields include:

  • Inventory and equipment items: helps agents avoid constant inventory opening
  • Interactive Objects: helps agents narrow down what’s interactable
  • Player status: health, stamina, buffs/debuffs: immediate access to player condition
  • Environmental data: nearby threats/enemies, current area/zone -> situational awareness without exploration

Most importantly, the game_state helps the agent with calling flayer functions. For example, if the agent needs to call a function that requires a specific item, it can check the game state for that item before calling the function. If the agent needs to navigate to a specific location, it can check the game state for the coordinates of that location before calling the function.

Private Fields

All private fields start with an underscore (_) and are hidden from the agent. They can be used for debugging as the full game state is always visible in the nexus timeline.

Best Practices

  1. Data Formatting
    • Use human-readable strings for complex data structures
    • Format coordinates as strings: "x: 50, y: 30, z: 20" instead of separate values
    • Include units in value strings: "20kg", "5m/s"
  2. Relevance
    • Prioritize information that saves the agent from performing expensive operations
    • Include data that would otherwise require multiple UI interactions to obtain
    • Provide aggregate data instead of requiring the agent to calculate it
    • Remove irrelevant information during simple states
  3. Performance Considerations
    • Keep the JSON structure flat where possible
    • Avoid deep nesting of objects
  4. Context Awareness
    • Design hint keys to be specific and hierarchical
    • Group related states under common prefixes
    • Only include information the agent actually needs

Example Game States

Here’s a complete example of a well-structured game state:

{
  "_can_interact": true,
  "_hint_key": "OVERWORLD_DAY_FOREST",
 
  "player": {
    "health": 90,
    "stamina": 75,
    "position": "x: 120, y: 45, z: 230",
    "inventory": [
      "5x wood in slot 3",
      "7x stone in slot 5",
      "1x iron sword in slot 1"
    ],
    "equipment": {
      "weapon": "iron sword (durability 80%)",
      "armor": "leather set (protection 15)"
    },
    "status_effects": ["well_fed: 5min remaining", "rested: 2min remaining"]
  },
 
  "environment": {
    "time_of_day": "morning",
    "weather": "rain",
    "temperature": "cold",
    "location": {
      "biome": "forest",
      "zone": "starting_area",
      "landmarks": ["trading_post: 50m north", "cave_entrance: 30m east"]
    }
  },
 
  "interactive_objects": {
    "resources": [
      "iron_ore: x: 125, y: 45, z: 232 (requires mining lvl 2)",
      "oak_tree: x: 118, y: 45, z: 228 (requires axe)"
    ],
    "crafting_stations": [
      "workbench: x: 115, y: 45, z: 225 (in range)",
      "forge: x: 160, y: 45, z: 240 (too far)"
    ]
  },
 
  "entities": {
    "hostile": [
      "skeleton: x: 140, y: 45, z: 235 (aggressive)",
      "wolf: x: 130, y: 45, z: 230 (passive)"
    ],
    "friendly": [
      "villager_merchant: x: 115, y: 45, z: 228 (can trade)",
      "villager_quest: x: 118, y: 45, z: 229 (has quest)"
    ]
  },
 
  "active_quests": [
    {
      "name": "Gather Resources",
      "objective": "Collect 10 wood and 5 iron ore",
      "progress": "5/10 wood, 0/5 iron ore",
      "reward": "50 gold, 1x steel axe",
      "time_remaining": "2 hours"
    }
  ],
 
  "_debug": {
    "_last_input": "move_forward",
    "_physics_states": {
      "_collision_count": 3,
      "_raycast_hits": 2
    }
  }
}
{
  "_can_interact": true,
  "_hint_key": "SHOOTER_MAP_ALIVE_COMBAT",
 
"player": {
"health": 100,
"armor": 50,
"position": "x: 200, y: 150, z: 300",
"inventory": [
"pistol: ammo: 15/30",
"shotgun: ammo: 5/8",
"grenade: x2"
],
"equipment": {
"primary_weapon": "assault_rifle (durability 90%)",
"secondary_weapon": "sniper_rifle (durability 80%)"
},
"status_effects": ["speed_boost: 10s remaining"]
},
 
"environment": {
"time_of_day": "afternoon",
"weather": "clear",
"location": {
"map_name": "downtown_area",
"landmarks": ["building_1", "park"]
}
},
 
"interactive_objects": {
"resources": [
"ammo_box: x: 205, y: 150, z: 305 (requires pickup)",
"health_pack: x: 210, y: 150, z: 310 (requires pickup)"
],
"vehicles": [
"car: x: 220, y: 150, z: 320 (requires key)",
"bike: x: 225, y: 150, z: 325 (available)"
]
},
"entities": {
"hostile": [
"enemy_soldier: x: 230, y: 150, z: 330 (aggressive)",
"enemy_sniper: x: 235, y: 150, z: 335 (passive)"
],
"friendly": [
"teammate_1: x: 240, y: 150, z: 340 (in range)",
"teammate_2: x: 245, y: 150, z: 345 (out of range)"
]
},
"active_quests": [
{
"name": "Secure the Area",
"objective": "Eliminate all enemies in the downtown area",
"progress": "3/5 enemies eliminated",
"reward": "100 gold, 1x health pack",
"time_remaining": "30 minutes"
}
],
"\_debug": {
"\_last_input": "shoot",
"\_physics_states": {
"\_collision_count": 2,
"\_raycast_hits": 1
}
}
}
 
{
  "_can_interact": true,
  "_hint_key": "RPG_MAP_ALIVE_PEACEFUL",
 
  "player": {
    "health": 80,
    "mana": 50,
    "position": "x: 300, y: 200, z: 400",
    "inventory": [
      "health_potion: x5",
      "mana_potion: x3",
      "iron_sword: durability 70%"
    ],
    "equipment": {
      "weapon": "iron_sword (durability 70%)",
      "armor": "chainmail (protection 20)"
    },
    "status_effects": ["blessed: 5min remaining"]
  },
 
  "environment": {
    "time_of_day": "evening",
    "weather": "foggy",
    "location": {
      "biome": "swamp",
      "zone": "abandoned_village"
    }
  },
 
  "interactive_objects": {
    "resources": [
      "herb: x: 305, y: 200, z: 405 (requires gathering)",
      "treasure_chest: x: 310, y: 200, z: 410 (requires opening)"
    ],
    "crafting_stations": [
      "alchemy_table: x: 315, y: 200, z: 415 (in range)",
      "forge: x: 320, y: 200, z: 420 (too far)"
    ]
  },
  "entities": {
    "hostile": [
      "goblin: x: 325, y: 200, z: 425 (aggressive)",
      "wolf: x: 330, y: 200, z: 430 (passive)"
    ],
    "friendly": [
      "villager_merchant: x: 335, y: 200, z: 435 (can trade)",
      "quest_giver: x: 340, y: 200, z: 440 (has quest)"
    ]
  },
  "active_quests": [
    {
      "name": "Collect Herbs",
      "objective": "Gather 10 herbs from the swamp",
      "progress": "3/10 herbs collected",
      "reward": "50 gold, 1x health potion",
      "time_remaining": "1 hour"
    }
  ],
  "_debug": {
    "_last_input": "cast_spell",
    "_physics_states": {
      "_collision_count": 1,
      "_raycast_hits": 0
    }
  }
}
Last updated on
Flayer FunctionsEldritchLink