Flayer Functions (Game Functions)

Flayer functions expand an AI agent’s capabilities beyond basic input simulation, providing direct programmatic interfaces to game functionality. This enables the agent to perform complex game-specific actions that would be difficult or impossible using just keyboard and mouse inputs.

Understanding Flayer Functions

Flayer functions provide a clean, declarative way to expose game functionality to AI agents. These functions follow a simple signature pattern making sure they are easy to use by the AI agent.

Flayer function requirements: - Take primitive type parameters (int, float, string, etc.) - Include a PacketLogger as the final parameter - Always return a primitive type

Core Functions

There’s a list of core functions that are already implemented in the SDK. These functions are hidden from the agent and are only used internally by the framework.

the game_state function returns the current game state as a JSON string.

For more information on the game state check out the Game State section

the pause function pauses or unpauses the game. It takes a boolean argument to determine whether to pause or unpause the game.

Depending on how pause is implemented in your game, you may need to override this function to ensure that the game is paused correctly.

the reset function is called after every agent step. It can be used to clear input buffers, stop navigation functions, or perform any other necessary cleanup tasks.

Custom Flayer Functions

In addition to the core functions, you can write your own Flayer functions to expose game-specific functionality to the AI agent.

For simple 2D mobile games, like Match 3, Hypercasual games, Puzzle games, Social Casino or Idle games, don’t require a deep integration with the nunu SDK. Most of these games can be played completely black box, meaning that the default mobile interface is enough to play the game or the AI agent.

The integration with the nunu SDK here, is mostly to collect logs and expose cheat commands to the agent. Similarly to how a human would test the game, being able to cheat progress, skip levels, to get to the part of the game where we can actually test is gonna speed up the testing process significantly.

Cheat Commands: Expose cheat commands to the agent, to skip levels, unlock items, or modify game state.
Game State: Expose game state information to the agent to verify test cases faster.

For 2D/3D exploration games, like Open World, Life Simulation Games, Sandbox Games, MMORPGs, Action RPGs or FPS games, the integration with the nunu SDK is a bit deeper. To effectively test these games, we usually need to expose some functionality to the agent, like:

Custom Game State: Including coordinates of point of interests, like NPCs, items, enemies, etc.
Navigation: A navigation function that allows the agent to quickly move fron one pont of interest to another.
Camera Control: A camera function that allows the agent to control the camera, look at specific points of interest, or aim at enemies.
Cheat Commands: Expose cheat commands to the agent, to skip levels, unlock items, or modify game state.

The process for writing and registering these functions depends on the game engine being used, but here are some general concepts to keep in mind:

Function Design

Give your custom Flayer functions descriptive names that clearly convey their purpose. Try to keep the functions focused on a single task or closely related tasks. While it’s possible to write complex, long functions, it’s often better to break them down into smaller, more targeted functions. This approach typically results in functions that are easier to maintain, compose in different ways, and use to build agent capabilities.


# Good: descriptive name and parameters
@flayer_expose("move_to_coords", "moves the player to the given coordinates")
def function_name(x: float, y: float, z: float, logger: PacketLogger):
    # Function implementation
    pass
 
# Good: focused on a single task
@flayer_expose("equip_item", "equips the given item in the given slot")
def function_name(item_id: str, slot: int, logger: PacketLogger):
    # Function implementation
    pass
 
# Bad: vague name and parameters
@flayer_expose("manage_inventory", "manages the player's inventory")
def function_name(id: str, quantity: int, logger: PacketLogger):
    # Function implementation
    pass

Usually a good way to tell if a function is well designed is, if a human that never saw the function before can understand what it does and can use it.

Effective Logging

Logging is curcial for good agent performance. It’s the best way to give the agent feedback about what is happening in the game.

Use the PacketLogger to send messages to the AI agent, indicating if the function was successful or if there were any issues. This feedback is essential for the agent to understand if he is doing the right thing or if it needs to change its approach.


@flayer_expose("add_item", "adds the given item to the player's inventory")
def add_item(item_id: str, quantity: int, logger: PacketLogger) -> bool:
    # Check if item exists
    item = inventory.get_item(item_id)
    if not item:
        logger.log(f"Item not found: {item_id}", level=ERROR, send_to_ai=True)
        return False
 
    # Add item to inventory
    success = inventory.add_item(item, quantity)
    if success:
        logger.log(f"Added {quantity} of {item.name} to inventory", level=INFO, send_to_ai=True)
        return True
    else:
        logger.log(f"Failed to add {item.name} to inventory", level=ERROR, send_to_ai=True)
        return False

Implementation Examples

Here are a few examples to illustrate custom Flayer functions in practice:


with timeout duration in ms") def navigate_to_location(x: float, y: float, z:
float, duration: int, logger: PacketLogger): # Verify game state if not
_can_interact: logger.log("Navigation attempted while game not interactable",
level=WARNING, send_to_ai=True) return False if player.is_in_cutscene:
logger.log("Navigation attempted during cutscene", level=WARNING,
send_to_ai=True) return False # Log navigation attempt
logger.log(f"Attempting navigation to ({x}, {y}, {z})", level=INFO) #
Calculate path path = pathfinding.find_path(player.position, target_pos) if
not path: logger.log("No valid path found", level=ERROR,
send_to_ai=True) return False # Start navigation try: success = await
movement_system.follow_path(path, duration) if success: logger.log("Navigation
completed successfully", level=INFO, send_to_ai=True) return True
else: logger.log(f"Navigation timed out after {duration}s", level=WARNING,
send_to_ai=True) return True catch (obstacle_error):
logger.log(f"Navigation failed: {obstacle_error}", level=ERROR,
send_to_ai=True) return False ```
</AccordionItem>
 
<AccordionItem title="Toggle God Mode" icon="ph:shield-check-bold">
```python
@flayer_expose("toggle_god_mode", "toggles god mode for the player")
def toggle_god_mode(enable: bool, logger: PacketLogger):
  # Check if player controller is valid
  if not player_controller:
      logger.log("Player controller not found", level=ERROR, send_to_ai=True)
      return False
 
  # Toggle god mode
  player_controller.god_mode = enable
  if enable:
      logger.log("God mode enabled", level=INFO, send_to_ai=True)
  else:
      logger.log("God mode disabled", level=INFO, send_to_ai=True)
 
  return True


@flayer_expose("game_state", "returns the current game state")
def game_state(logger: PacketLogger) -> str:
 
    # Build game state JSON
    game_state = {
        "_can_interact": !game_manager.is_cutscene_active,
        "_hint_key": f"{current_scene}_{current_menu}_{combat_state}_{player_state}",
        "current_scene": current_scene,
        "player_health": player.health,
        "player_max_health": player.max_health,
        "player_level": player.level,
        "current_objective": current_objective,
        "nearby_interactables": get_nearby_interactables(),
        "inventory": get_inventory_data()
    }
 
    return json.dumps(game_state)