Observability

• In some environments, the agent has full knowledge

• Often, though, they can not "see" everything

• For example: In Poker the opponent's cards are hidden

Deterministic or Stochastic

• Deterministic: Each action always does the same thing, and nothing else changes the world

• Stochastic: The state of the world may change by chance, or actions may have random effects

• Note: Poker is technically not stochastic: The deck has a predetermined order (even if it is unknown to the agent)

• Often, however, we abstract away unobservable information as stochasticity (self-driving cars)

Episodic vs. Sequential

• Episodic: Each action is independent from other actions

• Sequential: An action may have effects on future actions

• Most games are sequential: If we perform an action, we change what we do in the future

• Many tasks are episodic, though: Spam filtering, face recognition, path finding, etc.

Static vs. Dynamic

• Static: The environment does not change if the agent does nothing

• Dynamic: While the agent deliberates or acts, the environment may change

• Note: This change may be due to "physics", or other agents

• Semi-dynamic: The environment does not change, but the score does (chess clock)

Discrete vs. Continuous

• Discrete: There is a finite/countable number of states and time moves in steps

• Continuous: There is an uncountable number of states (like positions of a car in the real world), or time moves continuously

• We can often discretize continuous variables

• Many of our AI algorithms' complexity depends on the number of possible states

Known vs. Unknown

• Known: The agent knows how the environment "works" (the "rules")

• Unknown: The agent has to determine what each action does

• Technically this isn't a property of the environment itself, but of the agent

• For example, we can build an agent that knows the rules of Poker, or we could have one that learns them by trial and error

Environments

Environments

• Each of these different properties of environments poses different challenges for the agent

• In this class, we will discuss several different algorithms, each of which addresses some of these challenges

• The first environments we look at will be observable, single-agent, deterministic, episodic, static, discrete and known

Graphs: A Representation!

• A graph G = (V,E) consists of vertices (nodes) V and edges (connections) $E \subseteq V \times V$

• Graphs can be connected, or have multiple components

• Graphs can be directed (one-way streets) or undirected

• Edges can have weights (costs) associated with them: $w: E \mapsto \mathbb{R}$

• We will see that we can represent many things in graphs

Graphs

• We can "walk" around our graph

• Say we start at some node, and follow an edge

• If we can get back to our start node without visiting any edge twice we have found a "cycle"

• Cycles can complicate some things, so we like graphs without them

• These graphs are called "trees"

Trees

• An alternative view: We take some node and call it the root

• All nodes connected to the root are its children, which we put in another layer

• We then add another layer with the children's children, etc.

• In the end, we have the root at the top, with a number of layers below it

Trees

A note on Trees

• Our choice of "root" was completely arbitrary in this case

• We can redraw the same tree with a different node as the root

• For many applications in AI we have some interpretation of the nodes, though

• For example: In Single-player games the root is the current game state, and each edge is an action the player could perform

Sliding Puzzle Tree

Tree Search

• Let's say we want to win this game

• We start at the root node and then we search for a path that leads to a solution state

• What strategies/algorithms could we use to perform this search?

Tree Search

• We start at some node

• We can look at that node's neighbors

• Then we can pick one of these neighbors to continue

• Or we look at all the neighbors and continue from there?

Uninformed Search

The simplest pathfinding algorithm works like this:

• Keep track of which nodes are candidates for expansion (starting with the start node), called the (search) frontier

• Take one of these nodes and expand it, adding all its children to the frontier

• If you reach the target, you have found a path

Trees

Trees

Trees

Tree Search: Summary

Another example: Romania

Open Questions

• Arbitrary graphs?

• Time and space (memory) requirements?

• How do we actually implement this?!

• What about distances, like in the Romania example?