Review: The Planning Problem

A planning problem consists of three parts:

• A definition of the current state of the world

• A definition of a desired state of the world

• A definition of the actions the agent can take

Pathfinding

Recall the pathfinding problem from the second lecture

• We are given a graph, consisting of nodes and edges

• In our implicit graph representation, we could ask one node to generate its neighbors

• We would start at the start node, and generate neighbors "intelligently" until we reached a goal node

Planning As Pathfinding

To use a pathfinding algorithm for planning, we need to formulate the planning problem as a graph. Let's start with the "obvious" choice:

• Each state is a logical state

• Each edge corresponds to an action

• In any state, we can take any action whose preconditions are satisfied to generate a new state

The State-Space

• When we start at the node corresponding to the start state and start expanding actions, we generate the so-called State-Space

• We can then use a standard A* algorithm to do pathfinding

• In fact, many planners do exactly that, with different heuristics

State Space Example

Planning Problem in State Space

For the Project

• Parse PDDL files like discussed last week

• Extract all types, objects, operators, the initial state and the goal condition

• Ground actions

• Plan

Grounding Actions

(:action up
  :parameters (?f1 - floor ?f2 - floor)
  :precondition (and (lift-at ?f1) (above ?f1 ?f2))
  :effect (and (lift-at ?f2) (not (lift-at ?f1))))

For each parameter, replace with all possible values.

Save all ground actions in a list.

You may end up with many ground actions.

State Representation

You can create the world for the initial state by passing all atoms and the type dictionary to make_world. However, for pathfinding you will need a Node subclass, e.g. PlanningState.

Each PlanningState contains a reference to a logical world, as well as to the list of all actions

Note: You don't have to use make_world, and could instead store the state information directly in PlanningState

get_neighbors

• checks which actions can be applied in the logical world, i.e. which actions' preconditions are satisfied using models

• Generates a new logical world for each of these actions by applying its effects using apply

• Returns Edge objects containing new PlanningState objects for each generated logical world, with a cost of 1, and a string representing the action that was taken

Pathfinding

• We can now use astar as our planning algorithm!

• Pass it a PlanningState corresponding to the initial state (which can generate more states by applying actions)

• For the goal condition, write a isgoal function that uses models on the state and the goal condition

• Call pathfinding.astar(start, h, isgoal)

A short timeline

• 1995: Graphplan (next lecture)

• 1997/1998: HSP (Heuristic Search Planner)

• 2000/2001: FastForward

• 2005/2006: FastDownward

Planning Heuristics

• In planning, our nodes are (logical) states, and our edges are actions

• When we expand a state by applying all actions, we compute a heuristic value for each of the successor states

• This heuristic value should estimate how long a plan from that state to a goal state is (at most)

• In other words, we want to estimate how "difficult" it is to satisfy the goal from each state in our search frontier

Heuristics

• So how do we define a heuristic for such an abstract process?

• Here is an idea: Solve a simplified/approximate (relaxed) version of the problem, and use the cost of the solution as the heuristic

• How can we simplify the problem?

Heuristic Search Planning (HSP)

• Idea: Ignore delete lists

• Why is this simpler? Because the state always grows

• With a larger state, more actions can be applied, and eventually we will have added all atoms that can possibly be added

Note: This does not work without modification for non-STRIPS domains that have negative preconditions.

HSP Heuristic Example: Pickup

Positive Preconditions: $$\{\text{free}(), \text{clear}(X)\}$$ Negative Preconditions: None

Add List: $$\{\text{holds}(X)\}$$

Delete List: $$\{\text{free}(), \text{clear}(X)\}$$

HSP Heuristic Example

Relaxed blocksworld Pickup:

Put down:

Current State:

HSP

• HSP was the first widely successful planner to use heuristic search on STRIPS problems

• It won the 1998 planning competition

• There are updated versions which work with similar strategies

• FastForward was inspired by HSP

FastForward

• We said earlier that solving the relaxed problem is NP-Hard

• But that depends on which relaxed problem we mean

• Specificially, finding the optimal plan is NP-hard, but determining a plan is in P

• Of course, this may overestimate

FastDownward

• FastDownward is a "planning system", that implements several different search procedures and heuristics

• Additionally, FastDownward compiles the planning problem to a different representation

• Multi-Valued Planning Task: Instead of binary predicates we have value assignments

FastDownward

• As mentioned, FastDownward actually implements several different approaches

• The translation from a propositional planning problem (in PDDL) to an MPT is actually a separate component

• The algorithms we are discussing for propositional planning usually still work on MPTs with some slight modifications

Different Types of Heuristics

• Relaxation

• Abstraction

• Critical Paths

• Landmarks

• Network Flow

Relaxation

• So far all heuristics we have talked about have been relaxation heuristics

• They relax the planning problem in some way

• Removing negative effects is the most common way to do this

• You could also remove other parts of actions, or the goal, or split up actions

Critical Paths

• We can part of the goal, and construct a "critical path" backwards

• For example, to get B on the table, we have to put-on-table(B), for that we first have to pickup(B), etc.

• The number of actions on this critical path can serve as a heuristic

• We may want to try different parts of the goal to get a better estimate

Landmarks

• In many cases, we can find actions that have to happen (Landmarks)

• For example: If we want B to be on the table, we have to have an action put-on-table(B)

• Our heuristic value can be constructed from counting all of these landmark actions

• The more accurate our count is, the more accurate is our heuristic

Network Flow

• We can look at actions as producing and consuming facts

• The number of productions and consumptions has to be the same

• That means, for every time we pick up a block, we have to put one down as well

• We may be able to use this fact to determine how many "switches" have to be made at least

For the Project

• Task 4 is the most "creative" part of the project

• You are supposed to come up with your own heuristic

• Think about the different types, and how you would implement them

• The FastForward Heuristic may be a good starting point

Other Optimizations

• Combining Multiple Heuristics

• State-Space Pruning

• Invariant Synthesis

• Action preferences

• Other search strategies

Combining Multiple Heuristics

• No single heuristic is good for all planning problems

• Just calculate multiple different ones, and combine the results

• Pick the highest, lowest, average, sum, etc.

• Of course, this is just another heuristic and will not be "perfect" either

• But it may overcome limitations of others

Action Preference

• When we calculate a heuristic by solving a relaxed problem, we actually get a "plan"

• This plan can be used to give us a clue what a "good" next action might be

• Rather than considering all actions the same, we prefer these actions

• Can be used to augment the heuristic, or simply try this action first and if things don't work out fall back to "full" search

Other Search Strategies

Remember: Everything in AI is either representation or search

• Searching through state space is just one possible approach to planning

• Other planners may search through a different kind of space (different representation)

• Yet others may use a very different search procedure when expanding the states

• We will look at two such approaches in the next two weeks

Homework

• Homework 5 has been posted on the class website

• There are 5 problems using the planning domains from the repository of http://planning.domains

• You will be calculating heuristic values and solving relaxed problems

• When you do this, take note of how the different heuristics perform their estimations

Resources

• Planning as Heuristic Search

• HSP: Heuristic Search Planner

• FastForward

• FastDownward

• A Beginner's Introduction to Heuristic Search Planning