Lecture 18: AI Behavior

# Creación de Videojuegos

### AI Behavior

---

# Game AI

Reminder: The first rule of game AI:

### It does not matter how smart/dumb your AI is, as long as it *plays well*

---

# Ad-hoc behavior
  
  * Let's start simple: We have a bandit that's walking back and forth 
  
  * Our player comes near
  
```C
if (distance(player, bandit) < x) 
{
    attack(bandit, player);
}
```

---

# Ad-hoc behavior

* We play with this a bit, and realize that the player can now bait ("kite") the bandit to town, where the friendly NPCs can kill him without a problem 
  
  * Let's solve that!
  
```C
if (distance(bandit, bandit_camp) > y)
{
   walk_to(bandit, bandit_camp);
}
else if (distance(player, bandit) < x) 
{
    attack(bandit, player)
}>
```

---

# Ad-hoc behavior

* Now the player can bait the bandit to the edge of his follow radius
  
  * What if the player walks back and forth on that line?
  
  * We could add another condition for that ...
  
  * Note: This does not mean that ad-hoc coding of behavior is not bad, and often works well
  
  
---

# AI

## What is AI?

---

# What is AI
  
  * Just like with "game", no one agrees what "AI" really is 
  
  * Commonly, "the science of intelligent agents"
  
  * Or, as Douglas R. Hofstadter said "AI is whatever hasn't been done yet"
  
  * Pathfinding "was" once AI
  
  * For games: Whatever the opponents do
  
---

# What is an agent
  
  * We often talk of "AI agents"
  
  * Each "intelligent" character in your game is an "agent"
  
  * If you play "honestly", each agent only "knows" what a player would know if they could play as that character
  
  * For example: If you have a bandit, it can't see the player if the player is behind an obstacle
  
  * The agent uses their knowledge to make determine which action to use
  
  * Note: It's not mandatory to play "honestly" for the AI agents
  
---

# Some AI techniques

## (that are commonly used in games)

---

# Decision Trees

* Remember the series of if statements from earlier?
  
  * We can represent them as a tree 
  
  * Each branch is one if statement, based on some condition 
  
  * The leaves of the trees are actions the character takes 
  
---

# Decision Trees

---

# Decision Trees: Limitations

* We haven't actually changed anything from the if statements (other than drawing them)
  
  * Designing a decision tree is still a lot of manual work
  
  * There's also no persistence, the agent will decide a new behavior every time the tree is evaluated
  
  * There is one nice thing: Decision trees can (sometimes) be learned with Machine Learning techniques

---

# Finite State Machines

* We can make our code nicer if we separate decisions and behavior 
  
  * For example: Wandering around is one behavior, attacking the player another, etc.
  
  * Each of these behaviors is a "state" of the bandit 
  
  * The bandit decides when to change states depending on the game state

---

# Finite State Machines

---

# Finite State Machines: Limitations

* There's no real concept of "time", it has to be "added"

* If you just want to add one state you have to determine how it relates to every other state

* If you have two Finite State Machines they are hard to compose 
  
  * It's also kind of hard to reuse subparts
  
  * For example: A guard has a routine consisting of multiple states to chase an intruder, the bandit could use that same routine, but it would connect differently to the rest of its behavior

---

# Hierarchical Finite State Machines

* Finite State Machines define the behavior of the agent 
  
  * But we said the nodes are behaviors?!
  
  * We can make each node another sub-machine!
  
  * This leads to *some* reusability, and eases authoring  
  
---

# Behavior Trees

* Let's still use a graph, but make it a tree!
  
  * If we have a subtree, we now only need to worry about one connection: its parent
  
  * The *leaves* of the tree will be the actual actions, while the interior nodes define the decisions 
  
  * Each node can either be successful or not, which is what the interior nodes use for the decisions
  
  * We can have different kinds of nodes for different kinds of decisions 
  
  * This is extensible (new kinds of nodes), easily configurable (just attach different nodes together to make tree) and reusable (subtrees can be used multiple times)
  
---

# Behavior Trees: Common Node types

* Choice: Execute the first child that is successful 
  
  * Sequence: Execute all children until one fails
  
  * Loop: Keep executing child (or children) until one fails 
  
  * Random choice: Execute one of the children at random
  
  * Timing: Execute first child for x seconds, then the second child, etc.
  
---

# Behavior Trees

---

# Let's make a Behavior Tree for a Bandit!

* She can walk somewhere
  
  * She can see the player 
  
  * She can sound the alarm
  
  * She can hear the alarm of the bandit camp
  
  * She can attack the player 
  
---

# Planning

* What if we want the agent to decide on a strategy?
  
  * So far we have only looked at how we can assemble behaviors statically
  
  * Idea: Give the agent a goal and let it figure out how to reach that goal 
  
  * Of course the agent also has to know what it can do, which actions it can perform
  
---

# Planning

The planning problem: Given the current game state, a list of actions, and a goal, find a sequence of actions that leads to the goal

Note: This is basically pathfinding! And what do we use for pathfinding? A*!

---

# Planning

* The current game state is our start vertex 
  
  * Any vertex in which the goal is satisfied is our goal 
  
  * Each vertex has one edge for each action that can be taken in it 
  
  * To use A* you'll have to define a heuristic
  
---

# Planning: Example

---

# Adversaries?

* Planning doesn't really help if another character, or the player, can interfere?
  
  * How do we account for this?
  
  * For example, how can we play chess, if the opponent makes moves that affect our plans?
  
  * Adversarial search!
  
---

# Minimax!

* Let's say we want to get the highest possible score 
  
  * Then our opponent wants us to get the lowest possible score 
  
  * For each of our potential actions, we look at each of the opponents possible actions 
  
  * The opponent will pick the action that gives us the lowest score, and we will pick from our actions the one where the opponent's choice gives us the highest score 
  
  * How does the opponent decide what to pick? The same way!
  
---

# Minimax

# Minimax

---

# Minimax

Let's draw the tree for Tic Tac Toe

---

# Minimax

You and an opponent choose bits until two zeroes or two ones have been chosen in a row, or 5 digits have been chosen in total. If the resulting number is even, you get
that many points, otherwise you lose that many points. Draw the complete game tree for this game. What is your best first move? Example: You pick a 1, then you
opponent picks a 0, then you pick a 0, and the game ends because 2 zeroes have been chosen in a row. The resulting number is 100b = 4, and you get 4 points.

---

# Alpha-beta Pruning

---

# Alpha-beta pruning

* For the max player: Remember the minimum score they will reach in nodes that were already evaluated (alpha)
  
  * For the min player: Remember the maximum score they will reach in nodes that were already evaluated (beta)
  
  * If beta is less than alpha, stop evaluating the subtree

* Example: If the max player can reach 5 points by choosing the left subtree, and the min player finds an action in the right subtree that results in 4 points, they can stop searching.
  
  * If the right subtree was reached, the min player could choose the action that results in 4 points, therefore the max player will never choose the right subtree, because they can get 5 points in the left one
  
---

# Minimax: Limitations

* A tree for Tic Tac Toe is large to draw
  
  * Imagine one for chess 
  
  * Even with Alpha-Beta pruning it's impossible to evaluate all nodes 
  
  * Use a guess! For example: Board value after 3 turns 
  
  * What about unknown information (like a deck that is shuffled)?
  
---

# Unreal Engine

---

# References

* [Finite State Machines for Game AI (ES)](https://gamedevelopment.tutsplus.com/es/tutorials/finite-state-machines-theory-and-implementation--gamedev-11867)
  
  * [Halo 2 AI using Behavior Trees](http://www.gamasutra.com/view/feature/130663/gdc_2005_proceeding_handling_.php)
  
  * [Sliding puzzle using A*](https://blog.goodaudience.com/solving-8-puzzle-using-a-algorithm-7b509c331288)
  
  * [Building a Chess AI with Minimax](https://medium.freecodecamp.org/simple-chess-ai-step-by-step-1d55a9266977)