How to Integrate the Nunu SDK into Your Game

This guide provides a step-by-step approach to integrating the Nunu SDK into your game.

Basic Integration

The first step is simply adding the SDK to your game. Depending on the game engine you are using, follow the appropriate getting started guide:

Unity

Start with the Unity nunu SDK

Unreal Engine

Start with the Unreal Engine nunu SDK

Core Gameplay Functions

Next, we want to implement Flayer functions needed to efficiently play the game. The basic functions needed to play a game heavily depend on the game itself.

For a 2D/3D exploration game, like Open World, Life Simulation Games, Sandbox Games, MMORPGs, Action RPGs or FPS games, the integration with the nunu SDK is a bit deeper. To effectively test these games, we usually need to expose some functionality to efficiently move around the game world, like:

Key Functions:

move_to_location(x, y, z, timeout_ms): Move the player to coordinates
move_to_object(object_name, timeout_ms): Move to a specific game object/actor
look_at(x, y, z): Control camera to face specific coordinates
look_at_object(object_name): Turn camera toward a game object

Functionality to look at/move to interactable objects, over name or coordinates, looking at enemies or NPCs etc. is usually needed to effectively test the game.


@flayer_expose("move_to_location", "moves the player to the given coordinates for the given time")
def move_to_location(x: float, y: float, z: float, timeout_ms: int, logger: PacketLogger):
    # Function implementation
    pass
 
@flayer_expose("look_at", "turns the camera to look the given coordinates.")
def look_at(x: float, y: float, z: float, logger: PacketLogger):
    # Function implementation
    pass

For grid based puzzle games, like Match 3 etc, the basic integration could be as simple as exposing a function to help the agent swap tiles in the complex grid. Menuing and most other gameplay can be done black box, it’s really just the fine grained grid interaction that needs to be exposed.

So something like:

Key Functions:

swap_tiles(x1, y1, x2, y2): Swap tiles at grid coordinates


@flayer_expose("swap_tiles", "swaps the tiles at the given coordinates")
def swap_tiles(x1: int, y1: int, x2: int, y2: int, logger: PacketLogger):
    # Function implementation
    pass

Very static/simple 2D mobile games, like Hypercasual games, Puzzle games, Social Casino or Idle games, don’t require a deep integration with the nunu SDK. Most of these games can be played completely black box, meaning that the default mobile interface is enough to play the game or the AI agent.

For more details on how to write Flayer functions, check out the Flayer Functions section

Custom Game State Implementation

Next we want to override the default game state implementation. Having a custom game state implementation:

Allows more fine grained control over the hints and information the agent receives (eg. hints for menu, hints for combat, hints for specific maps etc.)
Allows the agent to better use the Flayer functions we implemented in the previous step (eg. move to specific coordinates, look at specific enemy, etc.)
Allows for better verification of goals or expected results of test cases (eg. check if the player is on a specific map, check if the player has a specific item, etc.)

Once again how much customization is needed depends on the game itself. For most games, just a basic implementation of the hint key and exposing whatever is needed to efficiently call the custom Flayer functions is enough.

For a very in-depth guide on how to implement a custom game state, check out the Game State section


@flayer_expose("game_state", "returns the current game state")
def game_state(logger: PacketLogger) -> str:
    # Build game state JSON
    game_state = {
        "_can_interact": not game_manager.is_cutscene_active,
        "_hint_key": f"{current_scene}_{current_menu}_{combat_state}_{player_state}",
 
        # Game-specific information
        "player": {
            "health": player.health,
            "position": f"x: {player.x}, y: {player.y}, z: {player.z}"
        },
        "interactive_objects": get_nearby_interactables(),
        "inventory": get_inventory_items()
    }
 
    return json.dumps(game_state)

Exposing Cheat Commands

Now that we can play the game, we want to expose some cheat commands to the agent. This is similar to how a human would test the game, being able to cheat progress, skip levels, to get to the part of the game where we can actually test is gonna speed up the testing process significantly.

Cheat functions are usually hidden from the AI agent, and only exposed in certain test cases through a cheat compendium or hints. Otherwise, the agent would be able to use them at any time, which is not ideal, because in most cases you don’t want to cheat unless explicity told to do so.

There’s usually 2 approaches to this:

This approach creates individual, strongly-typed functions for each cheat command. Each function is hidden by default (not exposed to the agent) and can only be used when specifically referenced in a hint or test case.

Clear, type-safe functions with proper parameter validation
Easy to understand purpose and usage of each function
Great for games where you have a limited set of cheats


@flayer_expose("unlock_all_levels", "unlocks all levels", hint_key="cheat_unlock_all_levels", exposed=False)
def unlock_all_levels(logger: PacketLogger) -> bool:
    try:
        game_progression.unlock_all_levels()
        logger.log("All levels unlocked successfully", level=INFO, send_to_ai=True)
        return True
    except Exception as e:
        logger.log(f"Failed to unlock levels: {str(e)}", level=ERROR, send_to_ai=True)
        return False
 
@flayer_expose("add_currency", "adds in-game currency", hint_key="cheat_add_currency", exposed=False)
def add_currency(currency_type: str, amount: int, logger: PacketLogger) -> bool:
    if not economy_manager.is_valid_currency(currency_type):
        logger.log(f"Invalid currency type: {currency_type}", level=ERROR, send_to_ai=True)
        return False
 
    economy_manager.add_currency(currency_type, amount)
    logger.log(f"Added {amount} {currency_type} to player account", level=INFO, send_to_ai=True)
    return True
 
@flayer_expose("skip_to_level", "skips to the specified level", hint_key="cheat_skip_level", exposed=False)
def skip_to_level(level_id: str, logger: PacketLogger) -> bool:
    if not level_manager.level_exists(level_id):
        logger.log(f"Level does not exist: {level_id}", level=ERROR, send_to_ai=True)
        return False
 
    level_manager.load_level(level_id)
    logger.log(f"Skipped to level: {level_id}", level=INFO, send_to_ai=True)
    return True

This approach creates a single exposed function that acts as a command parser, accepting string-based cheat commands with arguments. It’s ideal when integrating with existing cheat/debug console systems or when you have many cheat commands.

Single entry point for all cheat commands
Easy integration with existing debug console systems (SRDebugger or similar)
Simple to add new cheats without changing the interface


@flayer_expose("run_cheat", "executes a cheat command")
def run_cheat(command: str, logger: PacketLogger) -> bool:
    # Log the attempt
    logger.log(f"Attempting to run cheat command: {command}", level=INFO)
 
    # Basic command parsing
    if command.startswith("god"):
        # Toggle god mode
        player_controller = get_player_controller()
        if not player_controller:
            logger.log("Player controller not found", level=ERROR, send_to_ai=True)
            return False
 
        player_controller.set_god_mode(not player_controller.is_god_mode_enabled())
        status = "enabled" if player_controller.is_god_mode_enabled() else "disabled"
        logger.log(f"God mode {status}", level=INFO, send_to_ai=True)
        return True
 
    # Command with arguments
    if command.startswith("give "):
        # Parse arguments (format: "give item_id amount")
        args = command[5:].split()
        if len(args) < 2:
            logger.log("Invalid give command format. Use: give item_id amount", level=ERROR, send_to_ai=True)
            return False
 
        item_id = args[0]
        try:
            amount = int(args[1])
        except ValueError:
            logger.log(f"Invalid amount: {args[1]}", level=ERROR, send_to_ai=True)
            return False
 
        # Add the item
        inventory = get_player_inventory()
        if not inventory:
            logger.log("Player inventory not found", level=ERROR, send_to_ai=True)
            return False
 
        if inventory.add_item(item_id, amount):
            logger.log(f"Added {amount}x {item_id} to inventory", level=INFO, send_to_ai=True)
            return True
        else:
            logger.log(f"Failed to add item: {item_id}", level=ERROR, send_to_ai=True)
            return False
 
    # Level skip command
    if command.startswith("level "):
        # Parse level number
        try:
            level_id = command[6:].strip()
            if game_progression.is_valid_level(level_id):
                game_progression.load_level(level_id)
                logger.log(f"Loaded level: {level_id}", level=INFO, send_to_ai=True)
                return True
            else:
                logger.log(f"Invalid level ID: {level_id}", level=ERROR, send_to_ai=True)
                return False
        except Exception as e:
            logger.log(f"Failed to load level: {str(e)}", level=ERROR, send_to_ai=True)
            return False
 
    # More complex parsing example (format: "Command(arg1, arg2, ...)")
    match = re.match(r"(\w+)\((.*)\)", command)
    if match:
        cmd_name = match.group(1)
        args_str = match.group(2)
        args = [arg.strip() for arg in args_str.split(",")]
 
        if cmd_name == "UnlockAchievement":
            if len(args) == 1:
                achievement_id = args[0]
                achievement_system.unlock_achievement(achievement_id)
                logger.log(f"Unlocked achievement: {achievement_id}", level=INFO, send_to_ai=True)
                return True
 
        elif cmd_name == "SetStat":
            if len(args) == 2:
                stat_name = args[0]
                stat_value = float(args[1])
                player_stats.set_stat(stat_name, stat_value)
                logger.log(f"Set stat {stat_name} to {stat_value}", level=INFO, send_to_ai=True)
                return True
 
    # If we reach here, command was not recognized
    logger.log(f"Unknown cheat command: {command}", level=ERROR, send_to_ai=True)
    return False

For games that already have an internal console or cheat system, you can simply pass the command directly to that system:


@flayer_expose("run_cheat", "executes a cheat command using the game's internal console")
def run_cheat(command: str, logger: PacketLogger) -> bool:
    # Get access to the game's console
    console = get_game_console()
    if not console:
        logger.log("Game console not available", level=ERROR, send_to_ai=True)
        return False
 
    # Log the attempt
    logger.log(f"Running console command: {command}", level=INFO)
 
    # Execute the command and get result
    result = console.execute_command(command)
 
    # Log the result
    if result.success:
        logger.log(f"Command executed successfully: {result.message}", level=INFO, send_to_ai=True)
        return True
    else:
        logger.log(f"Command failed: {result.error}", level=ERROR, send_to_ai=True)
        return False

Compile a Build with the nunu SDK

Once the basic integration is done and tested, we want to compile a build or set up a build pipeline to automatically build the game and run the tests. Check out the Build Pipeline section for more details on how to set up a build pipeline.

With the Nunu SDK, it’s very important to adjust your build pipeline to compile with the USE_NUNU_SDK flag, or the generated builds will not have it enabled properly. To get more information check out the engine specific build configuration documentation.

Advanced Refinement

Now that we have the basic integration and it’s fully automated, we can start refining the integration if needed. This can include:

Improving the hint key
Adding more Flayer functions for situations where the agent struggles
Enhance the game state

Even though the basic integration is able to play the game, it doesn’t mean you shouldn’t refine it further. Refining the integration will speed up the AI agent, and therefore reduce the time it takes to run all the tests.

Improving the integration is an iterative process. Starting out with a basic integration and then refining where you see fit is the best approach. Don’t overdo it at the start, as you most likely don’t know yet what the agent will struggle with.