Lecture 15: Pathfinding

# Creación de Videojuegos

### Pathfinding

---

# Pathfinding

* In many games you have moving, computer-controlled opponents
 
 * These opponents need to determine where they are going 
 
 * Simplest approach: Walk back and forth between walls, fixed points, etc.
 
 * What if we want them to actually find a target?
 
 * Pathfinding!
 
---

# Reminder: PacMan

---

# What can we do?

---

# What can we do?

---

# Graphs

* A graph G = (V,E) consists of *vertices* (nodes) V and *edges* (connections) `\( E \subseteq V \times V \)`
  
  * Graphs can be connected, or have multiple components
  
  * Graphs can be directed (one-way streets) or undirected 
  
  * Edges can have weights (costs) associated with them: `\( w: E \mapsto \mathbb{R} \)`
  
  * We can represent many things in graphs
  
---

# The (undirected) Pathfinding problem

Given a graph G = (V,E), with edge weights w, a start node `\( s \in V \)`, a destination node `\( d \in V \)`, find a sequence of vertices `\( v_1, v_2, \ldots, v_n  \)`, such that `\(v_1 = s, v_n = d \)` and `\( \forall i: (v_i, v_{i+1}) \in E \)`

We call the sequence `\( v_1, v_2, \ldots, v_n \)` a *path*, and the *cost* of the path is `\( \sum_i w((v_i,v_{i+1})) \)`

This means what you would expect: To find a path from a start node to a destination node means to find vertices to walk through that lead from the start to the 
destination by being connected with edges. The cost is the sum of the costs of edges that need to be traversed.

---

# Another example: Romania

---

# How could we find a path?

---

# Every Search Algorithm

All search algorithms (that we are looking at) operate the same way:

* Store a list of nodes to visit, called the "frontier"
  
  * Take the next/"best" node from the frontier, and add all its neighbors to the frontier 
  
  * If the goal is in the frontier, we are done 
  
Different algorithms simply use different data structures to store the frontier. Note: This outline works for trees, for arbitrary graphs we need to handle the case where a node would be added to the frontier a second time.

---

# Depth-First Search

* Pick an order (say, north first, and then clockwise)
  
  * Store frontier as a stack, pushing the nodes in reverse order
  
  * Essentially, what this does:

* Visit the first neighbor
      
      * Visit the first neighbor's first neighbor 
	  
	  * Visit the first neighbor's first neighbor's first neighbor
	  
	  * etc, until we encounter a loop (continue with the second neighbor), or the goal (done)

---

# Depth-First Search

---

# Depth-First Search

<img src="/CI-2700/assets/img/dfs/1.png" width="100%"/>
75

---

# Depth-First Search

<img src="/CI-2700/assets/img/dfs/2.png" width="100%"/>
75 + 71 = 146

---

# Depth-First Search

<img src="/CI-2700/assets/img/dfs/3.png" width="100%"/>
75 + 71 + 151 = 297

---

# Depth-First Search

<img src="/CI-2700/assets/img/dfs/4.png" width="100%"/>
75 + 71 + 151 + 99 = 396

---

# Depth-First Search

<img src="/CI-2700/assets/img/dfs/5.png" width="100%"/>
75 + 71 + 151 + 99 + 211 = 607

---

# Depth-First Search: What if?

---

# Depth-First Search: Revisiting nodes

---

# Depth-First Search: What if?

---

# Depth-First Search: Long way

---

# Breadth-First Search

* Save the "frontier" as a list, initialized with the starting node 
  
  * If the goal is in the frontier, we are done
  
  * Else, for each node in the frontier, add all of its neighbors that to the frontier 
  
  * Repeat
  
  * Basically: Add all neighbors, then add all neighbor's neighbors, then add all neighbor's neighbor's neighbors, etc.
  
---
  
# Breadth-First Search

---

# Breadth-First Search

---

# Breadth-First Search

---

# Breadth-First Search

---

# Limitations

* Depth-First Search can lead to some very long paths 
  
  * If we had an infinite graph, Depth-First Search will probably even fail completely
  
  * Breadth-First Search may need a lot of memory to remember the frontier and paths to get there 
  
  * Both of them ignore costs
  
---

# Some improvements

* We can limit the depth that Depth-First Search should explore
  
     - Now it will "work" on infinite graphs 
     - We may also avoid some of the long paths
  
  * Idea: start with a depth limit of 1, run Depth-First Search, and if it can't find a path, increase the depth limit
  
  * This is called "Iterative Deepening"
  
  * To account for costs, we can use the cost as the order for Depth-First Search, and only make Breadth-First Search expand the node it can reach with the lowest cost in each iteration
  
---

# Heuristic Search

* All of these techniques are called "uninformed"
  
  * What if we can give the path finding algorithm some more information?
  
  * For example, we may not know how to drive everywhere, but we can measure the straight line distance
  
  * This "extra" information is called a "heuristic"
  
  * Search algorithms can use it to "guide" the search process
  
---

# Heuristic Search: General algorithm

* Keep a "frontier" as a priority list, initialized with the start node 
  
  * If the goal is in the frontier, we are done
  
  * Expand "the best" node from the frontier by adding all of its neighbors that have not been visited yet to the frontier 
  
What's "the best" node? What's the "priority" in the priority list?

---

# Greedy Search

* Let's use our heuristic!
  
  * We consider the node with the lowest heuristic to be "the best"
  
  * Heuristic: straight line distance to Bucharest
  
---

# Greedy Search

<table><tr><td>
<img src="/CI-2700/assets/img/pathproblem.png" width="90%"/></td><td> 
Heuristic:<br/>
<table>
<tr><td>Arad</td><td>366</td></tr>
<tr><td>Bucharest</td><td>0</td></tr>
<tr><td>Craiova</td><td>160</td></tr>
<tr><td>Drobeta</td><td>242</td></tr>
<tr><td>Eforie</td><td>161</td></tr>
<tr><td>Fagaras</td><td>176</td></tr>
<tr><td>Giurgiu</td><td>77</td></tr>
<tr><td>Hirsova</td><td>151</td></tr>
<tr><td>Iasi</td><td>226</td></tr>
<tr><td>Lugoj</td><td>244</td></tr>
<tr><td>Mehadia</td><td>241</td></tr>
<tr><td>Neamt</td><td>234</td></tr>
<tr><td>Oradea</td><td>380</td></tr>
<tr><td>Pitesti</td><td>100</td></tr>
<tr><td>Rimnicu Vilcea</td><td>193</td></tr>
<tr><td>Sibiu</td><td>253</td></tr>
<tr><td>Timisoara</td><td>329</td></tr>
<tr><td>Urziceni</td><td>80</td></tr>
<tr><td>Vaslui</td><td>199</td></tr>
<tr><td>Zerind</td><td>374</td></tr>
</table>
</td></tr></table>
Frontier: Arad (366)

---

# Greedy Search

<table><tr><td>
<img src="/CI-2700/assets/img/greedy/1.png" width="90%"/></td><td> 
Heuristic:<br/>
<table>
<tr><td>Arad</td><td>366</td></tr>
<tr><td>Bucharest</td><td>0</td></tr>
<tr><td>Craiova</td><td>160</td></tr>
<tr><td>Drobeta</td><td>242</td></tr>
<tr><td>Eforie</td><td>161</td></tr>
<tr><td>Fagaras</td><td>176</td></tr>
<tr><td>Giurgiu</td><td>77</td></tr>
<tr><td>Hirsova</td><td>151</td></tr>
<tr><td>Iasi</td><td>226</td></tr>
<tr><td>Lugoj</td><td>244</td></tr>
<tr><td>Mehadia</td><td>241</td></tr>
<tr><td>Neamt</td><td>234</td></tr>
<tr><td>Oradea</td><td>380</td></tr>
<tr><td>Pitesti</td><td>100</td></tr>
<tr><td>Rimnicu Vilcea</td><td>193</td></tr>
<tr><td>Sibiu</td><td>253</td></tr>
<tr><td>Timisoara</td><td>329</td></tr>
<tr><td>Urziceni</td><td>80</td></tr>
<tr><td>Vaslui</td><td>199</td></tr>
<tr><td>Zerind</td><td>374</td></tr>
</table>
</td></tr></table>
Frontier: Sibiu (253), Timisoara (329), Zerind (374)

---

# Greedy Search

<table><tr><td>
<img src="/CI-2700/assets/img/greedy/2.png" width="90%"/></td><td> 
Heuristic:<br/>
<table>
<tr><td>Arad</td><td>366</td></tr>
<tr><td>Bucharest</td><td>0</td></tr>
<tr><td>Craiova</td><td>160</td></tr>
<tr><td>Drobeta</td><td>242</td></tr>
<tr><td>Eforie</td><td>161</td></tr>
<tr><td>Fagaras</td><td>176</td></tr>
<tr><td>Giurgiu</td><td>77</td></tr>
<tr><td>Hirsova</td><td>151</td></tr>
<tr><td>Iasi</td><td>226</td></tr>
<tr><td>Lugoj</td><td>244</td></tr>
<tr><td>Mehadia</td><td>241</td></tr>
<tr><td>Neamt</td><td>234</td></tr>
<tr><td>Oradea</td><td>380</td></tr>
<tr><td>Pitesti</td><td>100</td></tr>
<tr><td>Rimnicu Vilcea</td><td>193</td></tr>
<tr><td>Sibiu</td><td>253</td></tr>
<tr><td>Timisoara</td><td>329</td></tr>
<tr><td>Urziceni</td><td>80</td></tr>
<tr><td>Vaslui</td><td>199</td></tr>
<tr><td>Zerind</td><td>374</td></tr>
</table>
</td></tr></table>
Frontier: Fagaras (176), Rimnicu Vilcea (193), Timisoara (329), Zerind (374), Oradea (380)

---

# Greedy Search

<table><tr><td>
<img src="/CI-2700/assets/img/greedy/3.png" width="90%"/></td><td> 
Heuristic:<br/>
<table>
<tr><td>Arad</td><td>366</td></tr>
<tr><td>Bucharest</td><td>0</td></tr>
<tr><td>Craiova</td><td>160</td></tr>
<tr><td>Drobeta</td><td>242</td></tr>
<tr><td>Eforie</td><td>161</td></tr>
<tr><td>Fagaras</td><td>176</td></tr>
<tr><td>Giurgiu</td><td>77</td></tr>
<tr><td>Hirsova</td><td>151</td></tr>
<tr><td>Iasi</td><td>226</td></tr>
<tr><td>Lugoj</td><td>244</td></tr>
<tr><td>Mehadia</td><td>241</td></tr>
<tr><td>Neamt</td><td>234</td></tr>
<tr><td>Oradea</td><td>380</td></tr>
<tr><td>Pitesti</td><td>100</td></tr>
<tr><td>Rimnicu Vilcea</td><td>193</td></tr>
<tr><td>Sibiu</td><td>253</td></tr>
<tr><td>Timisoara</td><td>329</td></tr>
<tr><td>Urziceni</td><td>80</td></tr>
<tr><td>Vaslui</td><td>199</td></tr>
<tr><td>Zerind</td><td>374</td></tr>
</table>
</td></tr></table>
Frontier: Bucharest (0), Rimnicu Vilcea (193), Timisoara (329), Zerind (374), Oradea (380)

---

# A* Search

* Greedy search sometimes does not give us the optimal result 
  
  * It tries to get to the goal as fast as possible, but ignores the cost of actually getting to each node 
  
  * Idea: Instead of using the node with the lowest heuristic value, use the node with the lowest sum of heuristic value and cost to get to
  
  * This is called A* search
  
---

# A* Search

# A* Search

<table><tr><td>
<img src="/CI-2700/assets/img/as/1.png" width="90%"/></td><td> 
Heuristic:<br/>
<table>
<tr><td>Arad</td><td>366</td></tr>
<tr><td>Bucharest</td><td>0</td></tr>
<tr><td>Craiova</td><td>160</td></tr>
<tr><td>Drobeta</td><td>242</td></tr>
<tr><td>Eforie</td><td>161</td></tr>
<tr><td>Fagaras</td><td>176</td></tr>
<tr><td>Giurgiu</td><td>77</td></tr>
<tr><td>Hirsova</td><td>151</td></tr>
<tr><td>Iasi</td><td>226</td></tr>
<tr><td>Lugoj</td><td>244</td></tr>
<tr><td>Mehadia</td><td>241</td></tr>
<tr><td>Neamt</td><td>234</td></tr>
<tr><td>Oradea</td><td>380</td></tr>
<tr><td>Pitesti</td><td>100</td></tr>
<tr><td>Rimnicu Vilcea</td><td>193</td></tr>
<tr><td>Sibiu</td><td>253</td></tr>
<tr><td>Timisoara</td><td>329</td></tr>
<tr><td>Urziceni</td><td>80</td></tr>
<tr><td>Vaslui</td><td>199</td></tr>
<tr><td>Zerind</td><td>374</td></tr>
</table>
</td></tr></table>
Frontier: Sibiu (140 + 253), Timisoara (118 + 329), Zerind (75 + 374)

---

# A* Search

<table><tr><td>
<img src="/CI-2700/assets/img/as/2.png" width="90%"/></td><td> 
Heuristic:<br/>
<table>
<tr><td>Arad</td><td>366</td></tr>
<tr><td>Bucharest</td><td>0</td></tr>
<tr><td>Craiova</td><td>160</td></tr>
<tr><td>Drobeta</td><td>242</td></tr>
<tr><td>Eforie</td><td>161</td></tr>
<tr><td>Fagaras</td><td>176</td></tr>
<tr><td>Giurgiu</td><td>77</td></tr>
<tr><td>Hirsova</td><td>151</td></tr>
<tr><td>Iasi</td><td>226</td></tr>
<tr><td>Lugoj</td><td>244</td></tr>
<tr><td>Mehadia</td><td>241</td></tr>
<tr><td>Neamt</td><td>234</td></tr>
<tr><td>Oradea</td><td>380</td></tr>
<tr><td>Pitesti</td><td>100</td></tr>
<tr><td>Rimnicu Vilcea</td><td>193</td></tr>
<tr><td>Sibiu</td><td>253</td></tr>
<tr><td>Timisoara</td><td>329</td></tr>
<tr><td>Urziceni</td><td>80</td></tr>
<tr><td>Vaslui</td><td>199</td></tr>
<tr><td>Zerind</td><td>374</td></tr>
</table>
</td></tr></table>
Frontier: Rimnicu Vilcea (220 + 193), Fagaras (239 + 176), Timisoara (118 + 329), Zerind (75 + 374), Orodea (291 + 380)

---

# A* Search

<table><tr><td>
<img src="/CI-2700/assets/img/as/3.png" width="90%"/></td><td> 
Heuristic:<br/>
<table>
<tr><td>Arad</td><td>366</td></tr>
<tr><td>Bucharest</td><td>0</td></tr>
<tr><td>Craiova</td><td>160</td></tr>
<tr><td>Drobeta</td><td>242</td></tr>
<tr><td>Eforie</td><td>161</td></tr>
<tr><td>Fagaras</td><td>176</td></tr>
<tr><td>Giurgiu</td><td>77</td></tr>
<tr><td>Hirsova</td><td>151</td></tr>
<tr><td>Iasi</td><td>226</td></tr>
<tr><td>Lugoj</td><td>244</td></tr>
<tr><td>Mehadia</td><td>241</td></tr>
<tr><td>Neamt</td><td>234</td></tr>
<tr><td>Oradea</td><td>380</td></tr>
<tr><td>Pitesti</td><td>100</td></tr>
<tr><td>Rimnicu Vilcea</td><td>193</td></tr>
<tr><td>Sibiu</td><td>253</td></tr>
<tr><td>Timisoara</td><td>329</td></tr>
<tr><td>Urziceni</td><td>80</td></tr>
<tr><td>Vaslui</td><td>199</td></tr>
<tr><td>Zerind</td><td>374</td></tr>
</table>
</td></tr></table>
Frontier: Fagaras (239 + 176), Pitesti (317 + 100), Timisoara (118 + 329), Zerind (75 + 374), Craiova (366 + 160), Orodea (291 + 380)

---

# A* Search

<table><tr><td>
<img src="/CI-2700/assets/img/as/4.png" width="90%"/></td><td> 
Heuristic:<br/>
<table>
<tr><td>Arad</td><td>366</td></tr>
<tr><td>Bucharest</td><td>0</td></tr>
<tr><td>Craiova</td><td>160</td></tr>
<tr><td>Drobeta</td><td>242</td></tr>
<tr><td>Eforie</td><td>161</td></tr>
<tr><td>Fagaras</td><td>176</td></tr>
<tr><td>Giurgiu</td><td>77</td></tr>
<tr><td>Hirsova</td><td>151</td></tr>
<tr><td>Iasi</td><td>226</td></tr>
<tr><td>Lugoj</td><td>244</td></tr>
<tr><td>Mehadia</td><td>241</td></tr>
<tr><td>Neamt</td><td>234</td></tr>
<tr><td>Oradea</td><td>380</td></tr>
<tr><td>Pitesti</td><td>100</td></tr>
<tr><td>Rimnicu Vilcea</td><td>193</td></tr>
<tr><td>Sibiu</td><td>253</td></tr>
<tr><td>Timisoara</td><td>329</td></tr>
<tr><td>Urziceni</td><td>80</td></tr>
<tr><td>Vaslui</td><td>199</td></tr>
<tr><td>Zerind</td><td>374</td></tr>
</table>
</td></tr></table>
Frontier: Pitesti (317 + 100), Timisoara (118 + 329), Zerind (75 + 374), Bucharest (450 + 0), Craiova (366 + 160), Orodea (291 + 380)

---

# A* Search

<table><tr><td>
<img src="/CI-2700/assets/img/as/5.png" width="90%"/></td><td> 
Heuristic:<br/>
<table>
<tr><td>Arad</td><td>366</td></tr>
<tr><td>Bucharest</td><td>0</td></tr>
<tr><td>Craiova</td><td>160</td></tr>
<tr><td>Drobeta</td><td>242</td></tr>
<tr><td>Eforie</td><td>161</td></tr>
<tr><td>Fagaras</td><td>176</td></tr>
<tr><td>Giurgiu</td><td>77</td></tr>
<tr><td>Hirsova</td><td>151</td></tr>
<tr><td>Iasi</td><td>226</td></tr>
<tr><td>Lugoj</td><td>244</td></tr>
<tr><td>Mehadia</td><td>241</td></tr>
<tr><td>Neamt</td><td>234</td></tr>
<tr><td>Oradea</td><td>380</td></tr>
<tr><td>Pitesti</td><td>100</td></tr>
<tr><td>Rimnicu Vilcea</td><td>193</td></tr>
<tr><td>Sibiu</td><td>253</td></tr>
<tr><td>Timisoara</td><td>329</td></tr>
<tr><td>Urziceni</td><td>80</td></tr>
<tr><td>Vaslui</td><td>199</td></tr>
<tr><td>Zerind</td><td>374</td></tr>
</table>
</td></tr></table>
Frontier: Bucharest (418 + 0), Timisoara (118 + 329), Zerind (75 + 374), Craiova (366 + 160), Orodea (291 + 380)

---

# A* Search

* To find *optimal* solution, keep expanding nodes until the goal node is the best node in the frontier
  
  * A* is actually guaranteed to find the optimal solution if the heuristic is:
  
     - Admissible: Never overestimate the cost 
     - Consistent: For a node x and its neighbor y, the heuristic value for x has to be less than or equal to that of y plus the cost of getting from x to y
     - Every consistent heuristic is also admissible
     
  * You can also reduce the memory requirements of A* by using Iterative Deepening
  
---

# Another Example

<table><tr><td>
<img src="/CI-2807/assets/img/pathexample.png" width="90%"/></td><td> 
Heuristic:<br/>
<table>
<tr><td>Arad</td><td>366</td></tr>
<tr><td>Bucharest</td><td>0</td></tr>
<tr><td>Craiova</td><td>160</td></tr>
<tr><td>Drobeta</td><td>242</td></tr>
<tr><td>Eforie</td><td>161</td></tr>
<tr><td>Fagaras</td><td>176</td></tr>
<tr><td>Giurgiu</td><td>77</td></tr>
<tr><td>Hirsova</td><td>151</td></tr>
<tr><td>Iasi</td><td>226</td></tr>
<tr><td>Lugoj</td><td>244</td></tr>
<tr><td>Mehadia</td><td>241</td></tr>
<tr><td>Neamt</td><td>234</td></tr>
<tr><td>Oradea</td><td>380</td></tr>
<tr><td>Pitesti</td><td>100</td></tr>
<tr><td>Rimnicu Vilcea</td><td>193</td></tr>
<tr><td>Sibiu</td><td>253</td></tr>
<tr><td>Timisoara</td><td>329</td></tr>
<tr><td>Urziceni</td><td>80</td></tr>
<tr><td>Vaslui</td><td>199</td></tr>
<tr><td>Zerind</td><td>374</td></tr>
</table>
</td></tr></table>
  
---

# Search

* While we have looked at finding paths in physical spaces so far, there are many other applications 
  
  * Take, for example, Super Mario
  
  * An AI could play the game using A*
  
<img src="/CI-2700/assets/img/marioas.png" width="50%"/>
  
---

# References 
 
  * *Artificial Intelligence: A Modern Approach* by Russel and Norvig, chapter 3: Search
  
  * [An interactive demo of different search algorithms](https://qiao.github.io/PathFinding.js/visual/)
  
  * [The Super Mario 2009 AI competition](http://julian.togelius.com/Togelius2010The.pdf)