quantum | v2 switch to V1  

Goal Oriented Action Planning (Beta)

Contents

1. Introduction

The GOAP quantum code and visual editor are both in Beta stage.

1.1 What Is GOAP

Goal-oriented action planning (GOAP) is an AI technique that consists of two parts:

  1. Selection of most suitable goal from a specified set of goals given current situation. Goals (also known as desires) could be: Defeat a target, Heal, Steal enemy flag, Flee from enemy
  2. Finding a set of actions (plan) from a specified actions pool that fulfill this goal. Actions could be: Find object, Find target, Pickup object (e.g. gun, enemy flag, health pack), Go to position, Go to combat range, Shoot

Back To Top
 

1.2 Comparison To Other BotSDK AI Approaches

As opposed to more standard AI approaches like FSM, HFSM and Behavior Trees, there is no exact AI behavior being created by the designer. An AI agent is told what desires it has and what it can do and the exact steps are put together (planned) in runtime based on the current situation (world state).

Back To Top
 

Strengths

  • Handles well complexity Adding new goals and actions that would automatically work with previous AI behavior is straightforward

  • Can speed up development time for larger projects When a large enough set of actions and goals is created it can be reused for many AI agents with different behaviors

  • Natural (fuzzy) behavior When AI behavior is nontrivial it is more difficult for players to spot some patterns if not desired. As with any AI that calculates some utilities/relevancies/costs, there are many values being considered that AI will likely act a little differently in similar situations.

  • Unique solutions

    With a large number of possible actions, AI has many possibilities on how to solve some problems that it can find a unique solution - this behaviour could be both good and bad. On the other hand, having too many actions can impact planning performance, so it needs to be approached carefully.

Back To Top
 

Weaknesses

  • Planning part (process of finding a suitable set of actions) needs to evaluate a lot of possibilities thus GOAP can be slower in general than more lightweight AI solutions (e.g. FSM)
  • More complex to master When designing GOAP AI, users need to figure out how to simplify a complex world into WorldState values and then always think about how GOAP actions will affect this WorldState rather than specifying the behaviour directly.
  • As AI is not told specifically what to do it is cumbersome to do “scripted” AI behavior such as boss fights

In order to best use GOAP strengths, the actions and goals need to be carefully designed. GOAP actions should be atomic - meaning more complex actions should be split into multiple actions if and only if such newly created actions could act in different situations with different actions around. An example could be some complex KillEnemyNest action that could be split into FindNest , GoToCombatRange and Attack actions.If such an approach is not possible or desirable, it is probably better to use a more standard AI solution right away.

Back To Top
 

2. Quantum GOAP Coding

2.1 Implementation

Let’s first analyze the GOAP from the code perspective, mainly to understand how it works internally, with just a few image examples. Then, we will follow up by analyzing it directly on the Unity Editor.

The core of GOAP is located in the GOAPManager. Manager(in GOAP’s context referred to as planner ) chooses the best goal, builds a plan (sequence of actions) that fulfill that selected goal and executes all actions from the plan if possible.

Search for possible sequence of actions is done through backward A* with heuristic being world state difference (how many world state flags are different from the target state).

BotSDK GOAP is very fast and lightweight but can also be further expanded based on specific game needs.

Back To Top
 

2.2 Creating And Updating A GOAP Agent

Here is a step by step for what you need to do to create and update your own agents:

  1. Add the GOAPAgent component to your entity. This can be done directly on Entity Prototypes on Unity, but it can also be done directly from within the quantum code by using the frame API;

  2. Find the desired GOAPRoot asset and use it on the initialization method:

        var goapRoot = frame.FindAsset<GOAPRoot>(playerData.GOAPRoot.Id);
     GOAPManager.Initialize(frame, entity, goapRoot);
  3. Call the Update method, passing the agent entity as a parameter:

        GOAPManager.Update(frame, filter.Entity, frame.DeltaTime);
  4. When needed, manually set the Agent’s current state with:

        goapAgent->CurrentState.SetFlag();

Back To Top
 

2.3 GOAP State

World State

GOAP state represents the state of the game that is relevant for the planner. Each AI agent has its own GOAP state. In our implementation the state is represented as Positive and Negative bitmasks. That means that each world state flag can have three values: true, false and not defined.

Choosing a correct set of flags is very important for good GOAP AI design. Universal rule is that a flag should be present in the world state if and only if the flag can change during planning (= is used as a Condition or an Effect of some GOAP Action).

E.g.: There is no point having the IsHungry flag in the world state if there is no GOAP action that would have IsHungry as a condition or an effect even though other game code uses it. In that case it is better to place the IsHungry flag in your custom agent’s data.

To add flags to the world state you have to edit EWorldState enum in the GOAPState.qtn file.

It is possible to expand the world state with custom data.

Back To Top
 

2.4 GOAP Goal

Goal Sample

Target State State that the planner tries to get to during the planning process.

Start State Usually left blank. When the planner is starting the planning process it takes the current agent’s state and merges it with the goal’s Start State. This merged state then acts as FROM state and Target State acts as TO state.

Note: Usually used to clear some flag from the agent's state to ensure some specific action will be planned.

Validation If a goal is not valid, it will not be considered during the goal selection process.

Relevancy Says how relevant this goal is. Goal with the largest relevancy will be selected by the planner. When two valid goals have the same relevancy, the one that is first in the list of goals will be selected.

Disable Time How long it takes before the goal could be reused.

Note: Disable time is often used when we do not want an AI agent to execute the same goal multiple times in a row even though it is considered as the best goal for the situation (e.g. we might not want an enemy to throw five grenades in a row). Also it is important in situations when a plan cannot be found (some actions are not valid in the current situation) and different goals need to be selected.

IsFinished When the goal is satisfied, the planner deactivates this goal and starts searching for a new goal. Goals are satisfied when either IsFinished is true or when the agent's GOAP state contains the goal's Target State.

Interruption Behaviour The execution of an active goal (plan) can be interrupted. If interruption is enabled, the planner periodically checks if some more relevant goal exists. If so, then execution of the current plan is stopped and a new goal gets planned.

● Always - Always check whether more relevant goal exists
● Never - Plan is never interrupted
● Based On Actions - Active action controls if the plancan be interrupted

OnInitPlanning Actions that are executed before the planning process begins.

OnActivate Actions that are executed when a plan is successfully found and the plan execution starts.

OnDeactivate Actions that are executed when the goal gets deactivated.

Back To Top
 

2.5 GOAP Action

Action Sample

Conditions What state this action needs in order to be considered in the plan.

Effects What effects does this action have on the agent’s state.

Validation Actions that are not valid are not considered during the planning process.

Note: This way you could “disable” certain actions in certain situations without actually complicating world state with unnecessary flags. As mentioned in the world state section, world state should contain only those properties that can change during the planning process itself.

Cost Determines the cost of the action. The planner tries to find a sequence of actions that has the lowest total cost. Cost of an action must be greater than zero. Usually it is best to think about the cost as a time in seconds the action will take but keep some reasonable range (e.g. 0.5 to 5 with most actions being around 1). Too large cost differences can hurt GOAP performance. See Optimization section for more details.

IsDone If an action is completed, the planner continues with the next action in the plan. Action is completed if IsDone is true or automatically when all Effects are present in the agent’s state.

IsFailed When action fails, the whole plan fails and the search for the new best goal starts again.

Interruptible Whether the action can be interrupted. Current active goal needs to have Interruption Behaviour set to Based On Actions in order to this setting be considered.

OnActivate AI actions executed when this GOAP action gets activated.

OnUpdate AI actions executed when this GOAP action is updated.

OnDeactivate AI actions executed when this GOAP action is deactivated.

Plan State Validation Plan state validation node is used for advanced GOAP capabilities. It basically means that it will validate the action during the planning based on the current planning state. It also has the possibility to branch the plan.

Back To Top
 

3. Editor

3.1 Creating A New GOAP

Create Document

On the editor's top bar, click on the (+) button, then select the alternative GOAP.

You will then be prompted to save the HFSM file. Save it wherever you prefer. This will create a data asset used to persist the work that you have done on the visual editor:

GOAP Asset

Note: The name that you choose now will be the name of another data asset generated further when you compile what you have done on the visual editor. That will be the data asset used to actually drive your Bots, so you can already choose a suggestive name.

When you save the file, the main Bot SDK window will show the most basic format of a GOAP document, in which you will see two containers: one named GOAP Goals and the other named GOAP Actions.

GOAP Clean Document

Back To Top
 

3.2 Creating New GOAP Goals

To create a new Goal, right click on any empty space on the Bot SDK window and select a GOAPDefaultGoal. In more advanced settings, it is possible for you to create your own goals, which will also be listed on the right click menu.

Create GOAL

On the root level, a non-modified goal node looks like this:

GOAL Root Level

It is also possible to double click on any goal node in order to see it’s details. The default goal provides you lots of fields that you will have to fulfill in order to express what you want for your AI.

GOAL Detailed

Two core concepts of every goal are the “Start State” and the “Target State”, which are Enums of the type EWorldState that represents the current world state, given a specific GOAP Agent (so this is not a shared/global state). Those states are used in the planning process in order to select the goals that will change the game state in a way that fulfills the agent’s objective.

It is possible to express a world state flag as Positive or Negative. During the planning process, the planner will filter the Goals that properly fulfills the desired world state. Let’s exemplify this a little bit with an example.

Note: Find more details on the definition of the World State on the Quantum GOAP Coding section.

Back To Top
 

3.2.1 Setting The Start/Target States:

  1. Click on the Start State or Target State buttons to define what are the flags that must be contained by the world state during the planning process, either as negative or positive;

    Add World Flag
  2. After selecting the desired states, they will now be contained within the goal node:

    World States Added
  3. By default, the world state will show up with a small green circle. This means that it is expecting the world state flags to contain that world state flag and that it should be Positive. It is then possible to click on the small green circle in order to change it to Negative. The circle will then be colored in red:

    World State Negative
  4. In the sample above, it is expected that the world state flags inform that the agent is in melee range , but is not on recovery , in order for this Goal to be included in the planning process (alongside with other goals). But it will only be included if the Target State’s result will contribute to the Agent’s current objective. In the example above, the result of choosing that Goal is that the target shall be defeated;

  5. It is also possible to see and change the world state flags from the root level. You can go back to the root either by pressing “ESC”, or by clicking on the button “Root”, on the breadcrumbs at the top bar. The root level goals are always shown in a summarized:

    Root Level World Flags

Back To Top
 

3.2.2 Including Goals And Organizing Them

For the Goal nodes to be included in the planning process, it is necessary to include them on the GOAP Goals container (the gray area). In order to achieve this, just drag and drop your goal nodes inside the container:

GOALS Container

It is also possible to rename goal nodes. To do that, select the node and press F2, or right-click on the node and choose the alternative “Rename”:

Goal Renamed

One important thing to notice here is that the node's order matters. Topmost nodes will have higher priority in comparison to the others. The node positioning is the tiebreaker if the planner decides that more than one goal with the same relevancy can be used in the same way to achieve the agent’s goal. It is a good practice to rename the goal nodes and to reorder them properly:

Organised Goals

Back To Top
 

3.3 Creating New GOAP Actions

In terms of creating, editing and organizing nodes, the same concepts above apply to the GOAP Actions. Here is a simple example of some actions included on the Actions container:

Actions Container

Back To Top
 

3.4 Ignoring And Muting Goals/Actions

It is possible to take goals and actions away from the planning process without the need to delete the nodes. This can be useful when debugging and trying alternative routes, without losing your nodes data, in case you decide to add them back again. There are two main ways for doing that:

  1. To ignore a node , just remove it from the container and add it back if it should not be ignored anymore;
  2. To mute a node , right click on it and select the alternative “Mute”. The node will then be a little bit transparent, to express that it is muted. It is useful to mute nodes because you do not lose it’s position/order on the container.

Ignored and muted nodes are skipped during the compilation process, so don’t forget to re-compile when doing that.

Back To Top
 

3.5 Setting Slot Values

When editing GOAP Goals and GOAP Actions (after double clicking on them), you will be able to edit many slots values. Some of them can be hardcoded values, or they can be from other nodes, and some other slots, such as the Action slots, can only be linked to one or more Action nodes.

In this section, we explore a few alternatives on how you can define such values.

Back To Top
 

3.5.1 Primitive Types

To start exemplifying, the GOAP Goals have a few slots with type FP and Boolean. You can define these slots values directly in the editor by clicking on them:

Edit Slot

But besides hard coding values, it is also possible to link the slots with nodes which might either contain a predefined value, or a predefined logic that will run upon polling that slot value. There are many ways for doing so and all of them are explained on this topic:

shared concepts.

Back To Top
 

3.5.2 Action Slots

Both GOAP Goals and GOAP Actions come with some Action slots on it’s right side. These are the micro actions which will be implemented by the users, that will actually change the game

state. These actions can be used to chase a target, to perform an attack, to collect an item and so on. Important: please, do not confound “GOAP Action” with“Action”. These are two different concepts.

Linked Action

The Actions are a common topic as they are also re-used by Bot SDK’s HFSM and UT. Find more information on how to use actions in this topic:

coding actions

Back To Top
 

3.5 Compiling Your GOAP

In order to actually use the GOAP that you created, you have to compile what you have done so you can get the Quantum asset which is used by the Quantum simulation. To compile, you have two options:

Compile Buttons
  • The left side button is used to compile only the currently opened document;
  • The right side button is used to compile every AI document that you have on your project.

Your GOAP files will be located at: Assets/Resources/DB/CircuitExport/GOAP_Assets.

Back To Top
 

3.6 Setting The AI To Be Used By Your Bots

To use the compiled asset, you need to reference it from within the Quantum code. You can do it by loading the asset based on the GUID, or you can just create AssetRefGOAPRoot to point to

the desired AI asset. It can be contained within some Component, some data asset, RuntimeConfig, RuntimePlayer, whatever better fits your code architecture:

GOAP Root Reference

PS: find more details on the Quantum GOAP Coding section.

Back To Top
 

4. Optimizations And Debugging

4.1 Debugging

BotSDK comes with its own debugging tool, but the GOAP editor currently does not make use of it. Adding the debugger is one of the next upcoming features for the GOAP editor.

For the time being, you can use GOAP logging functionality. Simply assign an entity into GameManager.DebugEntity and debug messages will be printed into the console.

GOAP Debug 1

Back To Top
 

4.2 Optimizations

GOAP optimization usually comes down to limiting number of nodes the A* needs to process. To check the numbers and time it took to process, enable entity debugging (see above) and check the log messages:

GOAP Debug 2

Note: The search time values are usually much larger in Unity editor than in actual build of the game. But it is a good reference when trying to improve performance.

Generally speaking you want to keep the number of visited nodes low, but exact number is dependent on the complexity of the AI. If the number of visited nodes is too high (e.g. over 100 for simple AI), chances are that the actions are incorrectly designed. Usual issue is that conditions and effects of action’s are too vague so the planner is considering a lot of options but most of them do not end up valid. Another thing to keep an eye on is actions Cost. A* uses cost to determine what is the best action sequence during the planning process. So having incorrectly set up costs of actions could cause planner to hit a lot of dead ends before returning to the correct branch. Best is to imagine cost as a time in seconds the action will take but keep it in some range (e.g. 0.5 - 5, with most actions being around 1) even though action will realistically take less or more just to not have vast differences of costs between actions.

Back To Top
 

5 Extra Details

5.1 AIParam

Find here more information on using the AIParam, which is useful if you want to have more flexible fields that can be defined in different ways: settings by hand or from Blackboard/Constant/Config Nodes: aiparam.

Back To Top
 

5.2 AIContext

Find here more information on how to pass agent-contextual information as parameter: aicontext.

Back To Top
 

5.3 BotSDKSystem

There is a class which is used to automate some processes such as deallocating Blackboard memory. Find here more information about it: botsdksystem.

Back To Top
 

5.4 Visual Editor Comments

Find here more information on how to create comments on the Visual Editor: visual editor comments.

Back To Top
 

5.5 Changing The Compilation Export Folder

By default, assets generated by Bot SDK's compilation will be placed into the folder Assets/Resources/DB/CircuitExport. See here how you can change the export folder: changing the export folder.

To Document Top