class: center, middle # Artificial Intelligence ### Planning --- class: center, middle # The Planning Problem --- # The Planning Problem A planning problem consists of three parts: * A definition of the current state of the world * A definition of a desired state of the world * A definition of the actions the agent can take All of these definitions are done in "some" formal language. -- Logic is a good choice for a formal language! --- # STRIPS .left-column[ <img src="/PF-3341/assets/img/Shakey.png" width="100%"/> ] .right-column[ * Shakey the Robot could solve problems given by a goal state * Various sensors let it build a model of the current world state * The Stanford Research Institute Problem Solver was the planner responsible for finding a solution ] --- class: small # STRIPS planning * A state (and the goal!) is defined as a set of atomic propositions of what is true in the world * The goal is reached when it is a **subset** of a state * Each action consists of four sets of atomic propositions: * Two sets of preconditions: positive and negative * An add list: What the action makes true * A delete list: What the action makes false * To check if an action can be executed in a state, the planner checks if all positive preconditions are in the state and all negative ones are not * To execute the action, a new state is produced as the union of the old state with the add list, and then the delete list is removed --- # Beyond STRIPS * STRIPS was one of the first instances of automated problem solving * To this day, planning still uses many of the same ideas * There are many different variations, though * Let's look at a concrete problem --- # An Example Planning Problem: Blocks World * We have three blocks, A, B, and C * Blocks A and B are on the table, block C is on top of block B * Blocks can be stacked * At the moment, all blocks are on the table next to each other * We want the blocks to be stacked, with C on top of B, and B on top of A --- # An Example Planning Problem: Blocks World <img src="/PF-3341/assets/img/problemstatement.png" width="100%"/> --- # Formalizing the Problem Current state: $$ \text{on}(A, \mathit{table}) \wedge\\\\ \text{on}(B, \mathit{table}) \wedge \\\\ \text{on}(C, B)\\\\ $$ Goal: $$ \text{on}(A, \mathit{table}) \wedge \\\\ \text{on}(B, A) \wedge\\\\ \text{on}(C, B)\\\\ $$ --- # Actions? What can the agent do? * Let's say our agent is a simple robot * It has a gripper * It can pick up a block, and also put it down (on the table, or on top of another block) --- # Formalizing Actions? * First, we need to represent that the robot is holding some block X somehow: `\(\text{holds}(X)\)` * Picking up a block then has a **precondition** and an **effect** * The precondition defines *if* an action can even be taken * The effect defines what the action *changes* in the world --- # Picking Up a Block X Precondition: $$ \forall Y \in \mathit{blocks}: \neg \text{holds}(Y) \wedge \\\\ \forall Y \in \mathit{blocks}: \neg \text{on}(Y, X) $$ Effect: $$ \forall Y \in \mathit{blocks}: \neg \text{on}(X, Y) \wedge \\\\ \neg \text{on}(X, \mathit{table}) \wedge \\\\ \text{holds}(X) $$ --- class: medium # STRIPS? * Logic is nice, but this seems like a pretty complex problem to solve? * Whenever we want to execute an action we have to check against all blocks to see if the agent is already holding something, or if there's already something on the target block * We can use an encoding trick to simplify the problem: Have predicates for negative conditions, too * For example: There is a predicate `holds` and a predicate `free` for the gripper, and a predicate `on` and a predicate `clear` for the blocks * That's how one could encode pretty complex problems in STRIPS --- # Picking Up a Block X in STRIPS Positive Preconditions: $$ \\{\text{free}(), \text{clear}(X)\\} $$ Negative Preconditions: None Add List: $$ \\{\text{holds}(X)\\} $$ Delete List: $$ \\{\text{free}(), \text{clear}(X)\\} $$ --- # Beyond STRIPS * Recall from last time: We restricted our actions to "effect formulas" * Effect formulas were: Positive and negative literals, foralls and when-expressions * As we just saw, we could reduce this even further * ... but we don't actually have to (anymore) --- # Putting a Block X on Top of a Block Y Precondition: $$ \text{holds}(X) \wedge \\\\ \forall Z \in \mathit{blocks}: \neg \text{on}(Z, Y) $$ Effect: $$ \text{on}(X, Y) \wedge \\\\ \neg \text{holds}(X) $$ --- class: small # The Planning Problem Given: * An initial state `\(s_0\)` * Action definitions `\(a = (p,e)\)` consisting of a precondition and an effect * A goal formula `\(g\)` Find a sequence of actions `\(a_0, a_1, \ldots, a_n\)`, such that: * `\(s_i \models p_i\)` * `\(s_{i+1} =\:\text{apply}\:\:e_i\:\:\text{to}\:\:s_i\)` * `\(s_{n+1} \models g\)` Each action can be applied in the state preceding it, and generates a new state using its effect. In the final state the goal condition holds. --- # Planners - A *planner* is an algorithm that solves the planning problem - Two important properties for a planner are: + **Soundness**: Any plan the planner produces is valid, i.e. satisfies the three conditions + **Completeness**: For any planning problem that has a solution, the planner will (eventually) produce one - These properties are "nice to have", but not always necessary! --- class: medium # Actions vs. Action Schemata * Technically "Picking up a block X" (`pickup(X)`) is not an action, because there is no "Block X" in our domain * We used "X" as a free variable to refer to "any block" * `pickup(X)` is an action schema, called **operator**, that will need to be turned into ground actions (without free variables) `pickup(A)`, `pickup(B)`, and `pickup(C)` --- # The Sussman Anomaly Let's look at a slightly different problem: <img src="/PF-3341/assets/img/sussman.png" width="100%"/> --- class: small # The Sussman Anomaly There are two goals: `on(A,B)` and `on(B,C)`. If the planner divides the goal into subgoals: - If it tries to reach a state with A on top of B first, it will put C aside, then put A on B, and then can not fulfill the other goal without undoing the first (by removing A). - If it tries to reach a state with B on top of C first, it will just put B on top of the pile, and then can't satisfy the other goal. A "proper" solution has to interleave solving these two goals: First remove C, then put B on C, then put A on B. This requirement for interleaving is called "the Sussman Anomaly", discovered in the 1970s. Modern Planners do not run into this problem anymore. --- class: center, middle # Planning as Pathfinding --- # Planning As Pathfinding To use a pathfinding algorithm for planning, we need to formulate the planning problem as a graph. Let's start with the "obvious" choice: * Each state is a logical state * Each edge corresponds to an action * In any state, we can take any action whose preconditions are satisfied to generate a new state --- # The State-Space - When we start at the node corresponding to the start state and start expanding actions, we generate the so-called *State-Space* - We can then use a standard A* algorithm to do pathfinding - In fact, many planners do exactly that, with different heuristics --- # State Space Example <img src="/PF-3341/assets/img/planspace.png" width="100%"/> --- # Planning Problem in State Space <img src="/PF-3341/assets/img/problem.png" width="100%"/> --- class: mmedium # Planning Heuristics - In planning, our nodes are (logical) states, and our edges are actions - When we expand a state by applying all actions, we compute a heuristic value for each of the successor states - This heuristic value should estimate how long a plan from that state to a goal state is (at most) - In other words, we want to estimate how "difficult" it is to satisfy the goal from each state in our search frontier - Many planners solve a reduced ("relaxed") version of the planning problem to calculate a heuristic value --- # (Some) Planning Heuristics - HSP: Ignore negative literals in effects - FastForward: Do a "Breadth-First Search" with merging of states - FastDownward: Convert Logic into a different representation, eliminate mutual exclusive states, and calculate possible state changes --- class: center, middle # Planning is PSPACE-complete --- # Planning Complexity * You may know the problem of P vs. NP? * PSPACE is (probably) even worse than NP $$ P \subseteq \mathit{NP} \subseteq \mathit{PSPACE} \subseteq \mathit{EXP} $$ (Any of these subsets may or may not be proper, but at least one of them is) --- # Why is Planning Hard? * We just said that planning is pathfinding on a graph consisting of the states * Pathfinding is solvable reasonably quickly * Planning is not? Why? -- * For a pathfinding problem, the input is the graph. For a planning problem the input is the initial state, the goal condition and the actions. The graph is **implicit** (and potentially exponentially large) --- # It gets worse ... <img src="/PF-3335/assets/img/plancomplexity.png" width="80%"/> --- class: medium # The Good News * [All PSPACE-complete planning problems are equal, but some are more equal than others](https://www.aaai.org/ocs/index.php/SOCS/SOCS11/paper/view/4009/4353) * Many interesting planning problems, i.e. the ones that show up in practice, are tractable, even if their generalizations are theoretically hard to solve * Another way to think about it: Since the solutions we usually *want* are not going to be exponential in length, we shouldn't have to generate a graph of exponential size * Also nice: Adding more convenience features doesn't actually make planning harder --- class: center, middle # Applications --- # Some Applications - Logistics (RoboCup Logistics League) - Ride sharing - Video Games - Narrative Generation --- # Goal-Oriented Action Planning * F.E.A.R. (and several other games) used a variant of planning for its enemy AI * Instead of arbitrary goals and states, it greatly restricts what can be represented * This restriction allowed it to be used in a real-time environment * The developers stated that using planning allowed them to easily add new action types --- # Story Generation - We have an initial state of the world - We have actions the different characters can perform - The author defines a goal for the story - For example: In a detective story the goal is that there is a murder and that the detective identifies the murderer --- # Story Example * Sherlock Holmes goes to Irene Adler's house * Sherlock Holmes stabs Irene Adler * Sherlock Holmes identifies himself as the killer * The End. --- # Story Generation as Planning - Planners are usually good at finding efficient solutions - Stories are not efficient! - However, each *character* should act rationally, i.e. efficiently - We could use one planner per character ... --- # Intentional Planning - With one planner per character we may never find a path to the story goal - Instead, we use a "trick": We tell the planner what each character wants to achieve - The planner may then only use actions for a character that helps them achieve their goal - But the overall plan still has to solve the story goal! - Implementations: IPOCL, CPOCL --- # Discourse Planning - Once we have a story (plot), we want to tell it - There are several different media, such as film, text, comics, or combinations thereof - As we discussed during our lecture on Hierarchical Planning, we can use planning to generate this discourse --- # Camera Planning - Suppose we want to make a movie - Each scene is supposed to convey some particular aspect of the story - There are different camera shots, angles, lenses, etc. that we can use - We can frame this as a planning problem by defining the different shots as operators --- # The Battle for the Rune [Link](https://drive.google.com/open?id=1VweROPFFeFyPv4M1rf_FPnZR9NDIuD23) --- class: medium # Resources * [Shakey the Robot](https://www.youtube.com/watch?v=qXdn6ynwpiI) * [STRIPS: A New Approach to the Application of Theorem Proving to Problem Solving](http://ai.stanford.edu/users/nilsson/OnlinePubs-Nils/PublishedPapers/strips.pdf) * [Planning as Heuristic Search](https://www.sciencedirect.com/science/article/pii/S0004370201001084) * [The Computational Complexity of Propositional Strips Planning](https://ai.dmi.unibas.ch/research/reading_group/bylander-aij1994.pdf) * [All PSPACE-complete planning problems are equal, but some are more equal than others](https://www.aaai.org/ocs/index.php/SOCS/SOCS11/paper/view/4009/4353) * [Introduction to STRIPS Planning and Applications in Video-games](https://www.dis.uniroma1.it/~degiacom/didattica/dottorato-stavros-vassos/) --- class: medium # Resources * [ICAPS 2019 Proceedings](https://aaai.org/ojs/index.php/ICAPS) * [Goal-Oriented Action Planning](http://alumni.media.mit.edu/~jorkin/goap.html) * [Narrative Planning: Compilations to Classical Planning](https://arxiv.org/abs/1401.5856) * [Bardic: Generating Multimedia Narrative Reports for Game Logs](https://www.semanticscholar.org/paper/Bardic-%3A-Generating-Multimedia-Narrative-Reports-Barot-Branon/3087daa75239c7990bfc993a5919d14e69373b42)