Turing Machines

Formally, Turing Machine is a 4-tuple: $M = (Q, \Lambda, q_0, \delta)$

• $Q$ is a set of states, which includes a special final (halting) state h

• $\Lambda$ is the tape alphabet, which includes a special symbol for "blank" (#)

• $q_0 \in Q$ is the initial state

• $\delta: (Q\setminus \{h\})\times \Lambda \mapsto \Lambda \times \{L,R\} \times Q$ is the transition function

Turing Machines

• So how do we compute anything with this?

• Let's start with a simple problem: Adding two binary numbers

• Input: Two binary numbers, separated by a plus (+)

• Output: The sum of the two numbers

Turing Machine: Intuition

• How do we add two binary numbers?

• We go digit by digit, starting from the right

• 0+0 = 0, 1+0 = 1, 1+0 = 1, 1+1 = 0, carry a 1

• Our Turing Machine has to move back and forth

Turing Machine: Picking up the last digit

1. Move right until you get to a blank

2. Move one to the left

3. Read digit and change into one of two states

Turing Machine: Adding Digits

The Entire Machine

The Entire Machine

• Download the JFLAP file /CS3110/assets/adder.jff

• You can input things like "1011+1101=" and it will add the two numbers

• Note: It only works with two numbers of the same length (why?)

• Demo Time!

Limits of Turing Machines?

• We can do pretty complex tasks with Turing Machines

• But what are the limits?

• Or asked another way: How could we extend this model of computation?

• First, is there anything we can't possible compute?

The Halting Problem

• Say you wrote a program X doing some complex calculation

• You give it some input and then you wait ... and wait ... and wait

• You want to know: Will this ever finish its computation?

• Maybe your IDE or debugger could tell you?

The Halting Problem

• What we want: A program H that reads a program (e.g. X), and some input and tells you if it will ever terminate with that input

• H should work for any program as input, not just some special cases

• And if it's a program, we can run it from within other programs

• So let's try something

A Strange Program

• Let's write a program D that gets a program M as input and then does:

• Call H on M, with M as the input (this is not a typo!)

• If H says that M terminates with M as input, go into an infinite loop

• Otherwise terminate

A Strange Program

• Let's write a program D that gets a program M as input and then does:

• Call H on M, with M as the input (this is not a typo!)

• If H says that M terminates with M as input, go into an infinite loop

• Otherwise terminate

What happens if we call D with itself as input?

The Halting Problem

• We appear to have reached a contradiction: D(D) does not terminate if H says that D(D) terminates (and vice versa)

• We didn't even talk about Automata or Turing Machines

• There is no way we could implement H

• This means there are programs that we can describe but not implement, they are undecidable

Other Undecideable Problems

• Program equivalence: Do two programs produce the exact same output, for all possible inputs

• Does a program terminate for every possible input

• Given a set of axioms, can we prove a given theorem

• etc.

Undecideable

• What Undecideable means is that we can not decide between a "yes" and a "no" answer

• Technically, we could write a program that says "yes", but may not be able to reach a "no" answer

• This is called "semi-decideable"

• What does this have to do with Turing Machines?

Turing Completeness

Beware of the Turing tar-pit in which everything is possible but nothing of interest is easy.

Alan J. Perlis, Yale University

Epigrams on Programming

Efficient Computation

• We briefly mentioned non-deterministic Turing-machines

• They do not solve the decideability-problem either

• But they do change one thing: Efficiency

• Basically, they can try options "in parallel"

A Simple Problem

• Say there are some cities connected by highways

• You know the distances between the cities

• Someone gives you a maximum distance, and you have to determine if you can visit all cities by driving less (in total) than that distance

• Can you do this efficiently?

Polynomial Time vs. Exponential Time

• There are some problems that can not be solved in polynomial time, for example determining if a program (given as a binary) will terminate within k steps

• There are some problems which we have polynomial time algorithms for (sorting an array, for example)

• But there is an important third class: problems for which we do not have a polynomial time algorithm, but don't know whether one exists or not

Traveling Salesman

• Take our city tour example, called the Traveling Salesman Problem

• A naive solution would be to try every possible ordering, but $n! > 2^n$

• There are some smarter algorithms, but they also require exponential time

• What no one knows: Is there a polynomial time algorithm

Solution Verification

• Now imagine someone gives you a potential solution

• You can easily (= in polynomial time) verify if it is an actual solution

• While we don't know if computing a solution can be done efficiently, we know that verifying it can

Non-Determinism

• Now recall Non-Deterministic Turing Machines

• We can use the non-determinism to "try" all possible solutions "in parallel"

• Then we check each solution and return one that works (or none)

• So we can solve this problem efficiently on a Non-Deterministic Turing Machine

P vs. NP

• It is unknown whether P = NP or not

• If you figure it out, you can get $1 000 000 (+ probably life-time appointment at a university or research institute of your choice)

$$P \subseteq \mathit{NP} \subseteq \mathit{PSPACE} \subseteq \mathit{EXPTIME}$$

We only know for sure that P is not the same as EXPTIME

Traveling Salesman

(Source)

Real Reality

• Probably not in P (we're not sure, but it's very likely): Discrete Logarithm, Integer Factorization

• Since we don't think these problems can be solved efficiently, they form the basis of most modern encryption

• In PSPACE: AI Planning; widely used in practice

• Conclusion: Theory is helpful, but in practice we care more about actual execution times!

References

• Online Brainf*ck Interpreter

• Epigrams on Programming

• P vs. NP Problem

• All PSPACE-complete Planning Problems are Equal but some are more Equal than Others