Lecture 2: Pathfinding

# Artificial Intelligence: Planning

### Pathfinding

---

# Pathfinding
  
---

# Graphs

* A graph G = (V,E) consists of *vertices* (nodes) V and *edges* (connections) `$ E \subseteq V \times V $`
  
  * Graphs can be connected, or have multiple components
  
  * Graphs can be directed (one-way streets) or undirected 
  
  * Edges can have weights (costs) associated with them: `$ w: E \mapsto \mathbb{R} $`
  
  * We can represent many things in graphs
  
---

# The (undirected) Pathfinding problem

Given a graph G = (V,E), with edge weights w, a start node `$ s \in V $`, a destination node `$ d \in V $`, find a sequence of vertices `$ v_1, v_2, \ldots, v_n  $`, such that `$v_1 = s, v_n = d $` and `$ \forall i: (v_i, v_{i+1}) \in E $`

We call the sequence `$ v_1, v_2, \ldots, v_n $` a *path*, and the *cost* of the path is `$ \sum_i w((v_i,v_{i+1})) $`

This means what you would expect: To find a path from a start node to a destination node means to find vertices to walk through that lead from the start to the 
destination by being connected with edges. The cost is the sum of the costs of edges that need to be traversed.

---

# Another example: Romania

---

# How could we find a path?

---

# Uninformed Search

- The simplest pathfinding algorithm(s) works like this:
  
      - Keep track of which nodes are candidates for expansion (starting with the start node)
      - Take one of these nodes and expand it 
      - If you reach the target, you have found a path
      
  - How do you "keep track" of nodes?
  
      - Use a list/queue: You now have "breadth-first search"
      - Use a stack: You now have "depth-first search"
      
---

# Uninformed Search

<img src="/PF-3341/assets/img/bfsdfs.png" width="100%"/>
  
---

# Heuristic Search
  
  * What if we can give the path finding algorithm some more information?
  
  * For example, we may not know how to drive everywhere, but we can measure the straight line distance
  
  * This "extra" information is called a "heuristic"
  
  * Search algorithms can use it to "guide" the search process
  
---

# Heuristic Search: General algorithm

* We use the same algorithm as above:
  
      - Keep track of which nodes are candidates for expansion (starting with the start node)
      - Take one of these nodes and expand it 
      - If you reach the target, you have found a path
  
  * Instead of using a stack or list, we use a *priority queue*, where the nodes are ordered according to some value derived from the heuristic

* So how do we determine this value?

---

# Greedy Search

* Let's just use our heuristic!
  
  * We order the nodes in the priority queue by heuristic value
  
  * Heuristic: straight line distance to Bucharest
  
---

# Greedy Search

<table><tr><td>
<img src="/CI-2700/assets/img/pathproblem.png" width="90%"/></td><td> 
Heuristic:<br/>
<table>
<tr><td>Arad</td><td>366</td></tr>
<tr><td>Bucharest</td><td>0</td></tr>
<tr><td>Craiova</td><td>160</td></tr>
<tr><td>Drobeta</td><td>242</td></tr>
<tr><td>Eforie</td><td>161</td></tr>
<tr><td>Fagaras</td><td>176</td></tr>
<tr><td>Giurgiu</td><td>77</td></tr>
<tr><td>Hirsova</td><td>151</td></tr>
<tr><td>Iasi</td><td>226</td></tr>
<tr><td>Lugoj</td><td>244</td></tr>
<tr><td>Mehadia</td><td>241</td></tr>
<tr><td>Neamt</td><td>234</td></tr>
<tr><td>Oradea</td><td>380</td></tr>
<tr><td>Pitesti</td><td>100</td></tr>
<tr><td>Rimnicu Vilcea</td><td>193</td></tr>
<tr><td>Sibiu</td><td>253</td></tr>
<tr><td>Timisoara</td><td>329</td></tr>
<tr><td>Urziceni</td><td>80</td></tr>
<tr><td>Vaslui</td><td>199</td></tr>
<tr><td>Zerind</td><td>374</td></tr>
</table>
</td></tr></table>
Frontier: Arad (366)

---

# Greedy Search

<table><tr><td>
<img src="/CI-2700/assets/img/greedy/1.png" width="90%"/></td><td> 
Heuristic:<br/>
<table>
<tr><td>Arad</td><td>366</td></tr>
<tr><td>Bucharest</td><td>0</td></tr>
<tr><td>Craiova</td><td>160</td></tr>
<tr><td>Drobeta</td><td>242</td></tr>
<tr><td>Eforie</td><td>161</td></tr>
<tr><td>Fagaras</td><td>176</td></tr>
<tr><td>Giurgiu</td><td>77</td></tr>
<tr><td>Hirsova</td><td>151</td></tr>
<tr><td>Iasi</td><td>226</td></tr>
<tr><td>Lugoj</td><td>244</td></tr>
<tr><td>Mehadia</td><td>241</td></tr>
<tr><td>Neamt</td><td>234</td></tr>
<tr><td>Oradea</td><td>380</td></tr>
<tr><td>Pitesti</td><td>100</td></tr>
<tr><td>Rimnicu Vilcea</td><td>193</td></tr>
<tr><td>Sibiu</td><td>253</td></tr>
<tr><td>Timisoara</td><td>329</td></tr>
<tr><td>Urziceni</td><td>80</td></tr>
<tr><td>Vaslui</td><td>199</td></tr>
<tr><td>Zerind</td><td>374</td></tr>
</table>
</td></tr></table>
Frontier: Sibiu (253), Timisoara (329), Zerind (374)

---

# Greedy Search

<table><tr><td>
<img src="/CI-2700/assets/img/greedy/2.png" width="90%"/></td><td> 
Heuristic:<br/>
<table>
<tr><td>Arad</td><td>366</td></tr>
<tr><td>Bucharest</td><td>0</td></tr>
<tr><td>Craiova</td><td>160</td></tr>
<tr><td>Drobeta</td><td>242</td></tr>
<tr><td>Eforie</td><td>161</td></tr>
<tr><td>Fagaras</td><td>176</td></tr>
<tr><td>Giurgiu</td><td>77</td></tr>
<tr><td>Hirsova</td><td>151</td></tr>
<tr><td>Iasi</td><td>226</td></tr>
<tr><td>Lugoj</td><td>244</td></tr>
<tr><td>Mehadia</td><td>241</td></tr>
<tr><td>Neamt</td><td>234</td></tr>
<tr><td>Oradea</td><td>380</td></tr>
<tr><td>Pitesti</td><td>100</td></tr>
<tr><td>Rimnicu Vilcea</td><td>193</td></tr>
<tr><td>Sibiu</td><td>253</td></tr>
<tr><td>Timisoara</td><td>329</td></tr>
<tr><td>Urziceni</td><td>80</td></tr>
<tr><td>Vaslui</td><td>199</td></tr>
<tr><td>Zerind</td><td>374</td></tr>
</table>
</td></tr></table>
Frontier: Fagaras (176), Rimnicu Vilcea (193), Timisoara (329), Zerind (374), Oradea (380)

---

# Greedy Search

<table><tr><td>
<img src="/CI-2700/assets/img/greedy/3.png" width="90%"/></td><td> 
Heuristic:<br/>
<table>
<tr><td>Arad</td><td>366</td></tr>
<tr><td>Bucharest</td><td>0</td></tr>
<tr><td>Craiova</td><td>160</td></tr>
<tr><td>Drobeta</td><td>242</td></tr>
<tr><td>Eforie</td><td>161</td></tr>
<tr><td>Fagaras</td><td>176</td></tr>
<tr><td>Giurgiu</td><td>77</td></tr>
<tr><td>Hirsova</td><td>151</td></tr>
<tr><td>Iasi</td><td>226</td></tr>
<tr><td>Lugoj</td><td>244</td></tr>
<tr><td>Mehadia</td><td>241</td></tr>
<tr><td>Neamt</td><td>234</td></tr>
<tr><td>Oradea</td><td>380</td></tr>
<tr><td>Pitesti</td><td>100</td></tr>
<tr><td>Rimnicu Vilcea</td><td>193</td></tr>
<tr><td>Sibiu</td><td>253</td></tr>
<tr><td>Timisoara</td><td>329</td></tr>
<tr><td>Urziceni</td><td>80</td></tr>
<tr><td>Vaslui</td><td>199</td></tr>
<tr><td>Zerind</td><td>374</td></tr>
</table>
</td></tr></table>
Frontier: Bucharest (0), Rimnicu Vilcea (193), Timisoara (329), Zerind (374), Oradea (380)

---

# A* Search

* Greedy search sometimes does not give us the optimal result 
  
  * It tries to get to the goal as fast as possible, but ignores the cost of actually getting to each node 
  
  * Idea: Instead of using the node with the lowest heuristic value, use the node with the lowest sum of heuristic value and cost to get to
  
  * This is called A* search
  
---

# A* Search

# A* Search

<table><tr><td>
<img src="/CI-2700/assets/img/as/1.png" width="90%"/></td><td> 
Heuristic:<br/>
<table>
<tr><td>Arad</td><td>366</td></tr>
<tr><td>Bucharest</td><td>0</td></tr>
<tr><td>Craiova</td><td>160</td></tr>
<tr><td>Drobeta</td><td>242</td></tr>
<tr><td>Eforie</td><td>161</td></tr>
<tr><td>Fagaras</td><td>176</td></tr>
<tr><td>Giurgiu</td><td>77</td></tr>
<tr><td>Hirsova</td><td>151</td></tr>
<tr><td>Iasi</td><td>226</td></tr>
<tr><td>Lugoj</td><td>244</td></tr>
<tr><td>Mehadia</td><td>241</td></tr>
<tr><td>Neamt</td><td>234</td></tr>
<tr><td>Oradea</td><td>380</td></tr>
<tr><td>Pitesti</td><td>100</td></tr>
<tr><td>Rimnicu Vilcea</td><td>193</td></tr>
<tr><td>Sibiu</td><td>253</td></tr>
<tr><td>Timisoara</td><td>329</td></tr>
<tr><td>Urziceni</td><td>80</td></tr>
<tr><td>Vaslui</td><td>199</td></tr>
<tr><td>Zerind</td><td>374</td></tr>
</table>
</td></tr></table>
Frontier: Sibiu (140 + 253), Timisoara (118 + 329), Zerind (75 + 374)

---

# A* Search

<table><tr><td>
<img src="/CI-2700/assets/img/as/2.png" width="90%"/></td><td> 
Heuristic:<br/>
<table>
<tr><td>Arad</td><td>366</td></tr>
<tr><td>Bucharest</td><td>0</td></tr>
<tr><td>Craiova</td><td>160</td></tr>
<tr><td>Drobeta</td><td>242</td></tr>
<tr><td>Eforie</td><td>161</td></tr>
<tr><td>Fagaras</td><td>176</td></tr>
<tr><td>Giurgiu</td><td>77</td></tr>
<tr><td>Hirsova</td><td>151</td></tr>
<tr><td>Iasi</td><td>226</td></tr>
<tr><td>Lugoj</td><td>244</td></tr>
<tr><td>Mehadia</td><td>241</td></tr>
<tr><td>Neamt</td><td>234</td></tr>
<tr><td>Oradea</td><td>380</td></tr>
<tr><td>Pitesti</td><td>100</td></tr>
<tr><td>Rimnicu Vilcea</td><td>193</td></tr>
<tr><td>Sibiu</td><td>253</td></tr>
<tr><td>Timisoara</td><td>329</td></tr>
<tr><td>Urziceni</td><td>80</td></tr>
<tr><td>Vaslui</td><td>199</td></tr>
<tr><td>Zerind</td><td>374</td></tr>
</table>
</td></tr></table>
Frontier: Rimnicu Vilcea (220 + 193), Fagaras (239 + 176), Timisoara (118 + 329), Zerind (75 + 374), Orodea (291 + 380)

---

# A* Search

<table><tr><td>
<img src="/CI-2700/assets/img/as/3.png" width="90%"/></td><td> 
Heuristic:<br/>
<table>
<tr><td>Arad</td><td>366</td></tr>
<tr><td>Bucharest</td><td>0</td></tr>
<tr><td>Craiova</td><td>160</td></tr>
<tr><td>Drobeta</td><td>242</td></tr>
<tr><td>Eforie</td><td>161</td></tr>
<tr><td>Fagaras</td><td>176</td></tr>
<tr><td>Giurgiu</td><td>77</td></tr>
<tr><td>Hirsova</td><td>151</td></tr>
<tr><td>Iasi</td><td>226</td></tr>
<tr><td>Lugoj</td><td>244</td></tr>
<tr><td>Mehadia</td><td>241</td></tr>
<tr><td>Neamt</td><td>234</td></tr>
<tr><td>Oradea</td><td>380</td></tr>
<tr><td>Pitesti</td><td>100</td></tr>
<tr><td>Rimnicu Vilcea</td><td>193</td></tr>
<tr><td>Sibiu</td><td>253</td></tr>
<tr><td>Timisoara</td><td>329</td></tr>
<tr><td>Urziceni</td><td>80</td></tr>
<tr><td>Vaslui</td><td>199</td></tr>
<tr><td>Zerind</td><td>374</td></tr>
</table>
</td></tr></table>
Frontier: Fagaras (239 + 176), Pitesti (317 + 100), Timisoara (118 + 329), Zerind (75 + 374), Craiova (366 + 160), Orodea (291 + 380)

---

# A* Search

<table><tr><td>
<img src="/CI-2700/assets/img/as/4.png" width="90%"/></td><td> 
Heuristic:<br/>
<table>
<tr><td>Arad</td><td>366</td></tr>
<tr><td>Bucharest</td><td>0</td></tr>
<tr><td>Craiova</td><td>160</td></tr>
<tr><td>Drobeta</td><td>242</td></tr>
<tr><td>Eforie</td><td>161</td></tr>
<tr><td>Fagaras</td><td>176</td></tr>
<tr><td>Giurgiu</td><td>77</td></tr>
<tr><td>Hirsova</td><td>151</td></tr>
<tr><td>Iasi</td><td>226</td></tr>
<tr><td>Lugoj</td><td>244</td></tr>
<tr><td>Mehadia</td><td>241</td></tr>
<tr><td>Neamt</td><td>234</td></tr>
<tr><td>Oradea</td><td>380</td></tr>
<tr><td>Pitesti</td><td>100</td></tr>
<tr><td>Rimnicu Vilcea</td><td>193</td></tr>
<tr><td>Sibiu</td><td>253</td></tr>
<tr><td>Timisoara</td><td>329</td></tr>
<tr><td>Urziceni</td><td>80</td></tr>
<tr><td>Vaslui</td><td>199</td></tr>
<tr><td>Zerind</td><td>374</td></tr>
</table>
</td></tr></table>
Frontier: Pitesti (317 + 100), Timisoara (118 + 329), Zerind (75 + 374), Bucharest (450 + 0), Craiova (366 + 160), Orodea (291 + 380)

---

# A* Search

<table><tr><td>
<img src="/CI-2700/assets/img/as/5.png" width="90%"/></td><td> 
Heuristic:<br/>
<table>
<tr><td>Arad</td><td>366</td></tr>
<tr><td>Bucharest</td><td>0</td></tr>
<tr><td>Craiova</td><td>160</td></tr>
<tr><td>Drobeta</td><td>242</td></tr>
<tr><td>Eforie</td><td>161</td></tr>
<tr><td>Fagaras</td><td>176</td></tr>
<tr><td>Giurgiu</td><td>77</td></tr>
<tr><td>Hirsova</td><td>151</td></tr>
<tr><td>Iasi</td><td>226</td></tr>
<tr><td>Lugoj</td><td>244</td></tr>
<tr><td>Mehadia</td><td>241</td></tr>
<tr><td>Neamt</td><td>234</td></tr>
<tr><td>Oradea</td><td>380</td></tr>
<tr><td>Pitesti</td><td>100</td></tr>
<tr><td>Rimnicu Vilcea</td><td>193</td></tr>
<tr><td>Sibiu</td><td>253</td></tr>
<tr><td>Timisoara</td><td>329</td></tr>
<tr><td>Urziceni</td><td>80</td></tr>
<tr><td>Vaslui</td><td>199</td></tr>
<tr><td>Zerind</td><td>374</td></tr>
</table>
</td></tr></table>
Frontier: Bucharest (418 + 0), Timisoara (118 + 329), Zerind (75 + 374), Craiova (366 + 160), Orodea (291 + 380)

---

# A* Search

* To find *optimal* solution, keep expanding nodes until the goal node is the best node in the frontier
  
  * A* is actually guaranteed to find the optimal solution if the heuristic is:
  
     - Admissible: Never overestimate the cost 
     - Consistent: For a node x and its neighbor y, the heuristic value for x has to be less than or equal to that of y plus the cost of getting from x to y
     
  * You can also reduce the memory requirements of A* by using Iterative Deepening:
  
     - Limit search to a particular depth 
     - If no path is found, increase the limit
  
---

# Dijkstra's algorithm

* You may have heard of Dijkstra's algorithm (and its variants) before
  
  * Dijkstra's algorithm is basically A* without using the heuristic 
  
  * In some popular formulations you also let the algorithm compute a path for *every* possible destination 
  
  * This will give you a shortest path tree, which may be useful if you have to repeatedly find a path to different destinations
  
---

# Search

* While we have looked at finding paths in physical spaces so far, there are many other applications 
  
  * Take, for example, Super Mario
  
  * An AI agent could play the game using A*
  
<img src="/CI-2700/assets/img/marioas.png" width="50%"/>

---

# Search Graphs

* What if we have infinite (or at least very large) graphs?
  
  * We can't really store the entire graph in memory 
  
  * But do we need to?
  
---

# Lazy Evaluation

* Some programming languages (like Haskell) use something called *lazy evaluation*
  
  * This means a value is only evaluated when it is needed 
  
  * For example `[1..]` is an infinite list of integers in Haskell 
  
  * But as long as you don't try to access *all* elements, it does not need to generate them 
  
  * We can "simulate" this behavior in other languages using functions
  
---

# Lazy Evaluation for Graphs

* Instead of representing an entire graph in memory, let's say we just store one node 
  
  * Each node can, when requested, give us its neighbors
  
  * We can then even "forget" the original node, if we want to
  
  * The only requirement is that each node only has finitely many neighbors (and preferably not too many)
  
---

# An Infinite Graph

* Let's say we have a graph with one node for each integer

* Each node has 2 to 4 neighbors: The predecessor and successor, twice its value, and half its value (if it is even). All distances are 1

* "1" has the neighbors "0" and "2"

* "3" has the neighbors "2", "4", and "6"

* "4" has the neighbors "3", "5", "8", and "2"

* "16" has the neighbors "15", "17", "32", and "8"

---

# Goal Conditions

* We often also don't want to look for one specific (of our infinitely many) nodes, but rather a node that satisfies some condition

* For example, what if we want a node that is greater than 1000 and a power of 2?

* Each of the nodes "1024", "2048", "4096", etc. would be a valid solution

* Let's use A* starting from "127" to find such a goal node

* We need a heuristic!

---

# Integer graph heuristic

Let's use the following heuristic:
$$
h(n) = \lvert \lfloor \log_2(n) \rfloor - \lfloor \log_2(1024)\rfloor \rvert
$$

What does it do? Look for the next lowest power of two of the current node and the lowest valued goal node and calculate the difference.

Now let's do A*!

---

# Why are we doing this?

* (Almost) anything can be described as a graph

* As long as we have a representation of a "node", a way to get "neighbors", a predicate that tells us what is a "goal" and a heuristic, we can do A*

* In some cases, we can just use a constant 0 for the heuristic (what happens then?)

* In *planning* our nodes will be logical states, the neighbors will be produced by actions and we will be given a logical goal condition

* There has been quite some research in different planning heuristics

---

# For the Project

* For the project, the framework contains a definition of a graph that makes use of this lazy evaluation scheme

```python
class Node:
    def get_neighbors(self):
        # returns a list of edges to neighbors
        return []
        
        
# Each edge contains the target node, the cost of taking 
# that path and a name (particularly useful for planning)
class Edge:
    def __init__(self, target, cost, name):
        self.target = target 
        self.cost = cost
        self.name = name
```

---

# For the Project

* Take a look at `graph.py` to see how these classes are used to set up a finite graph (of Austria), as well as an infinite graph (the integer graph we discussed earlier).

* You do not have to change anything in `graph.py`, but you are encouraged to come up with more examples to test your implementation

---

# For the Project

The actual task is to complete `pathfinding.py` by implementing `astar`:

```python
def astar(start, heuristic, goal):
    # returns: 
    #    path (list of edges)
    #    cost of path (sum of cost of the edges)
    #    number of nodes visited (added to the search frontier)
    #    number of nodes expanded (removed from the search
    #                              frontier, with its neighbors 
    #                              added instead)
    return [],0,0,0
```

Note that the goal and heuristic are functions!

```python
def atgoal(n):
    return n.get_id() == target
```

---

# For the Project

* You can implement A* independently from the logical formula representation

* The deadline for this task is September 27

* The deadline for task 1 is September **20** (one day later than originally stated, to also be on a Friday)
  
---

# Homework

* [Homework 2](/PF-3335/assets/pdf/homework2.pdf) has been posted on the class website
  
  * There are 5 problems involving A*
  
  * There is also a bonus problem involving a (short) proof
  
  * Homework is due **in two weeks**, on Sep 6
  
  * No class next week, because I will be at a conference!

---

# References

* [A* implementations in multiple languages](https://rosettacode.org/wiki/A*_search_algorithm)
  
  * [Sliding puzzle using A*](https://blog.goodaudience.com/solving-8-puzzle-using-a-algorithm-7b509c331288)
  
  * [JavaScript visualization of several search algorithms on a grid](https://qiao.github.io/PathFinding.js/visual/)