class: center, middle # CS-3110: Formal Languages and Automata ## Computability ### Chapter 5 --- # Turing Machines Formally, Turing Machine is a 4-tuple: `\(M = (Q, \Lambda, q_0, \delta)\)` * `\(Q\)` is a set of states, which includes a special final (halting) state `h` * `\(\Lambda\)` is the tape alphabet, which includes a special symbol for "blank" (#) * `\(q_0 \in Q\)` is the initial state * `\(\delta: (Q\setminus \{h\})\times \Lambda \mapsto \Lambda \times \{L,R\} \times Q\)` is the transition function --- # Turing Machines * So how do we compute anything with this? * Let's start with a simple problem: Adding two binary numbers * Input: Two binary numbers, separated by a plus (+) * Output: The sum of the two numbers --- # Turing Machine: Intuition * How do we add two binary numbers? * We go digit by digit, starting from the right * 0+0 = 0, 1+0 = 1, 1+0 = 1, 1+1 = 0, carry a 1 * Our Turing Machine has to move back and forth --- # Turing Machine: Picking up the last digit <center> <img src="/CS3110/assets/img/pickupdigit.png" width="60%"/> </center> 1. Move right until you get to a blank 2. Move one to the left 3. Read digit and change into one of two states --- # Turing Machine: Adding Digits <center> <img src="/CS3110/assets/img/digitadd.png" width="100%"/> </center> --- class: medium # Turing Machine: The Rest * Then we move to the left, until we have passed both numbers * Write the next digit of the sum * Move right again * Repeat * One detail: You need to add another case when you start with a carried 1 --- # The Entire Machine <center> <img src="/CS3110/assets/img/turingadder.png" width="100%"/> </center> --- # The Entire Machine * Download the JFLAP file [/CS3110/assets/adder.jff](/CS3110/assets/adder.jff) * You can input things like "1011+1101=" and it will add the two numbers * Note: It only works with two numbers of the same length (why?) * Demo Time! --- class: center, middle # Computability --- # Limits of Turing Machines? * We can do pretty complex tasks with Turing Machines * But what are the limits? * Or asked another way: How could we extend this model of computation? * First, is there anything we can't possible compute? --- # The Halting Problem * Say you wrote a program X doing some complex calculation * You give it some input and then you wait ... and wait ... and wait * You want to know: Will this ever finish its computation? * Maybe your IDE or debugger could tell you? --- # The Halting Problem * What we want: A program H that reads a program (e.g. X), and some input and tells you if it will ever terminate with that input * H should work for *any* program as input, not just some special cases * And if it's a program, we can run it from within other programs * So let's try something --- # A Strange Program * Let's write a program D that gets a program M as input and then does: * Call H on M, with M as the input (this is not a typo!) * If H says that M terminates with M as input, go into an infinite loop * Otherwise terminate --- # A Strange Program * Let's write a program D that gets a program M as input and then does: * Call H on M, with M as the input (this is not a typo!) * If H says that M terminates with M as input, go into an infinite loop * Otherwise terminate What happens if we call D with itself as input? --- # The Halting Problem * We appear to have reached a contradiction: D(D) does not terminate if H says that D(D) terminates (and vice versa) * We didn't even talk about Automata or Turing Machines * There is *no way* we could implement H * This means there are programs that we can describe but not implement, they are **undecidable** --- # Other Undecideable Problems * Program equivalence: Do two programs produce the exact same output, for all possible inputs * Does a program terminate for *every* possible input * Given a set of axioms, can we prove a given theorem * etc. --- # Undecideable * What Undecideable means is that we can not decide between a "yes" and a "no" answer * Technically, we could write a program that says "yes", but may not be able to reach a "no" answer * This is called "semi-decideable" * What does this have to do with Turing Machines? --- class: mmedium # Church-Turing Thesis * Thesis: Turing Machines can compute anything that is computable * Other models are equivalent to Turing Machines: - Lambda Calculus - General Grammars - Register Machines - Most programming languages (**assuming infinite memory**) --- class: mmedium # Brainf*ck BF is an esoteric programming language. It runs on a "tape" (similar to a Turing machine) of integers, and has only 8 instructions: * `+` and `-`: Add or subtract one from the current cell * < and >: Move one space left or right * `.` and `,`: Write and read the current cell as a character * `[` and `]`: Loop as long as the current cell is not zero Hello World: ``` ++++++++ [>++++ [>++>+++>+++>+<<<<-] >+>+>->>+ [<]<-] >>.>---.+++++++..+++.>>.<-.<.+++.------.--------.>>+.>++. ``` --- # Turing Completeness ### Beware of the Turing tar-pit in which everything is possible but nothing of interest is easy. Alan J. Perlis, Yale University *[Epigrams on Programming](http://pu.inf.uni-tuebingen.de/users/klaeren/epigrams.html)* --- class: center, middle # Reality --- # Efficient Computation * We briefly mentioned non-deterministic Turing-machines * They do not solve the decideability-problem either * But they do change one thing: Efficiency * Basically, they can try options "in parallel" --- # A Simple Problem * Say there are some cities connected by highways * You know the distances between the cities * Someone gives you a maximum distance, and you have to determine if you can visit all cities by driving less (in total) than that distance * Can you do this efficiently? --- class: mmedium # Efficient Computation * What do we mean by "efficiently" (in this case)? * At this point we care about the "general" case, where we have n cities * Say we have a program that solves this problem in "quadratic time" * This means, when we have twice as many cities, it takes 4 times as long as before * Another program may need 9 times as long for twice as many cities, and use cubic time, etc. * Whichever (**fixed**) exponent we have, we call this *polynomial time* --- class: medium # Exponential Time * What if we have a program that needs exponential time, e.g. `\(2^n\)`? * If we add **one** city, our program now needs **twice** as long * If we add **10** cities, we need **1024** times as long * If we add **100** cities, the sun will be extinguished long before we even make it halfway through --- # Polynomial Time vs. Exponential Time * There are some problems that can not be solved in polynomial time, for example determining if a program (given as a binary) will terminate within k steps * There are some problems which we have polynomial time algorithms for (sorting an array, for example) * But there is an important third class: problems for which we do not *have* a polynomial time algorithm, but don't know whether one exists or not --- # Traveling Salesman * Take our city tour example, called the Traveling Salesman Problem * A naive solution would be to try every possible ordering, but `\(n! > 2^n\)` * There are some smarter algorithms, but they also require exponential time * What no one knows: Is there a polynomial time algorithm --- # Solution Verification * Now imagine someone *gives* you a potential solution * You can easily (= in polynomial time) **verify** if it is an actual solution * While we don't know if computing a solution can be done efficiently, we know that verifying it can --- # Non-Determinism * Now recall Non-Deterministic Turing Machines * We can use the non-determinism to "try" all possible solutions "in parallel" * Then we *check* each solution and return one that works (or none) * So we **can** solve this problem efficiently on a Non-Deterministic Turing Machine --- class: mmedium # P vs. NP * `\(P\)` is the set of all problems that can be solved efficiently on a Deterministic Turing Machine * `\(NP\)` is the set of all problems that can be solved efficiently on a Non-Deterministic Turing Machine * Question: Is P = NP? * In other words: Can we simulate a Non-Deterministic Turing Machine **efficiently** on a Deterministic Turing Machine * Or: If we know that we can **verify** a solution of a problem efficiently, does that mean that we can also compute one efficiently, or not? --- # P vs. NP * It is unknown whether P = NP or not * If you figure it out, you can get $1 000 000 (+ probably life-time appointment at a university or research institute of your choice) $$ P \subseteq \mathit{NP} \subseteq \mathit{PSPACE} \subseteq \mathit{EXPTIME} $$ We only know for sure that P is not the same as EXPTIME --- # Traveling Salesman <img src="https://imgs.xkcd.com/comics/travelling_salesman_problem.png" width="100%"/> ([Source](https://xkcd.com/399/)) --- # Real Reality * Probably not in P (we're not sure, but it's very likely): Discrete Logarithm, Integer Factorization * Since we don't think these problems can be solved efficiently, they form the basis of most modern encryption * In PSPACE: AI Planning; widely used in practice * Conclusion: Theory is helpful, but in practice we care more about actual execution times! --- # References * [Online Brainf*ck Interpreter](https://copy.sh/brainfuck/) * [Epigrams on Programming](http://pu.inf.uni-tuebingen.de/users/klaeren/epigrams.html) * [P vs. NP Problem](https://www.claymath.org/millennium-problems/p-vs-np-problem) * [All PSPACE-complete Planning Problems are Equal but some are more Equal than Others](https://www.ida.liu.se/~chrba09/Papers/socs11.pdf)