]> Turing Machines
[HOME][NEXT][INDEX][BIBLIOGRAPHY]

1 Turing Machines

1.1 Problems that Computers Cannot Solve

It is important to know whether a program is correct, namely that it does what we expect. It is easy to see that the following Java program

public class Main {
  void main(String [] args)  {
    System.out.println("hello, world");
  }
}

prints hello, world and terminates.

But about following program:

public class Main {
  int exp(int i, int n) {
    int r;
    for (int j = 1; j <= n; j++) {
      r *= i;
    }
    return r;
  }

  void main(String [] args)  {
    int n = 123456;
    int t = 254;
    while (true) {
      for int x = 1; x <= t - 2; x++) {
        for (int y = 1; y <= t - x -1; y++) {
          for (z = 1; z <= t - x - y; z++) {
            if (exp(x,n) + exp(y,n) == exp(z,n) {
              System.out.println("hello, world");
            }
          }
        }
      } 
    }
  }
}

Given an input n, it prints hello, world only if the equation

xn+yn=zn

has a solution where x, y, and z are integers. We know nowadays that it will print hello, world for n=2, and loop forever for n>2.

It took mathematicians 300 years to prove this so-called Fermat’s last theorem1. Can we expect to write a program H that solves the general problem of telling whether any given program P, on any given input I, eventually prints hello, world or not?

1.1.1 The hypothetical hello, world tester

Proof by contradiction that H is impossible to write. Suppose that H exists:

Hello-world tester H I yes no P

H takes as input a program P and an input I, and tells whether P with input I prints hello, world. In particular, the only output H makes is either to print yes or to print no.

We modify the response no of H to hello, world, getting a program H1:

Hello-world I yes P tester H1 hello, world

We modify H1 to take P and I as a single input, getting a program H2:

Hello-world yes P hello, world P tester H2

We provide H2 as input to H2:

Hello-world yes hello, world tester H2 H2 H2

If H2 prints yes, then it should have printed hello, world.

If H2 prints hello, world, then it should have printed yes.

So H2 and hence H cannot exist.

Hence we have an undecidable problem. It is similar to the language Ld we will see later.

1.1.2 Undecidable Problems

Definition 1.1 [Undecidable]

A problem is undecidable if no program can solve it.

1.1.2.1 Gödel Incompleteness Theorem

In 1931, Gödel published his famous incompleteness theorems in Über formal unentscheidbare Sätze der Principia Mathematica und verwandter Systeme [Gö31]. In that article, he proved for any computable axiomatic system that is powerful enough to describe the arithmetic of the natural numbers, that:

  1. If the system is consistent, it cannot be complete.

  2. The consistency of the axioms cannot be proved within the system.

In hindsight, the basic idea at the heart of the incompleteness theorem is rather simple. Gödel essentially constructed a formula that claims that it is unprovable in a given formal system. If it were provable, it would be false, which contradicts the fact that in a consistent system, provable statements are always true. Thus there will always be at least one true but unprovable statement. That is, for any computably enumerable set of axioms for arithmetic (that is, a set that can in principle be printed out by an idealized computer with unlimited resources), there is a formula that obtains in arithmetic, but which is not provable in that system. To make this precise, however, Gödel needed to solve several technical issues, such as encoding statements, proofs, and the very concept of provability into the natural numbers. He did this using a process known as Gödel numbering.

1.1.3 Problem Reduction

A reduction is a transformation of one problem into another problem. Depending on the transformation used this can be used to define complexity classes on a set of problems.

Intuitively, problem A is reducible to problem B if solutions to B exist and give solutions to A whenever A has solutions. Thus, solving A cannot be harder than solving B. We write AB, usually with a subscript on the to indicate the type of reduction being used.

Often we find ourselves trying to solve a problem that is similar to a problem we've already solved. In these cases, often a quick way of solving the new problem is to transform each instance of the new problem into instances of the old problem, solve these using our existing solution, and then use these to obtain our final solution. This is perhaps the most obvious use of reductions.

Another, more subtle use is this: suppose we have a problem that we've proved is hard to solve, and we have a similar new problem. We might suspect that it, too, is hard to solve. We argue by contradiction: suppose the new problem is easy to solve. Then, if we can show that every instance of the old problem can be solved easily by transforming it into instances of the new problem and solving those, we have a contradiction. This establishes that the new problem is also hard.

Example 1.1 [Multiplication/Squaring]

A very simple example of a reduction is from multiplication to squaring. Suppose all we know how to do is to add, subtract, take squares, and divide by two. We can use this knowledge, combined with the following formula, to obtain the product of any two numbers:

a×b=a+b2-a2-b22

We also have a reduction in the other direction; obviously, if we can multiply two numbers, we can square a number. This seems to imply that these two problems are equally hard.

Example 1.2 [Function call]

Does a program Q, given input q, ever call function foo()?

This problem is undecidable.

We construct a program R and an input z such that R with input z, calls foo() if and only if Q with input y prints hello, world:

  1. If Q has a function called foo(), rename it and all calls to that function. Clearly the new program Q1 does exactly what Q does.

  2. Add to Q1 a function foo(). This function does nothing, and is not called. The resulting program is Q2.

  3. Modify Q2 to remember the first 12 characters that it prints, storing them in a global array A. The resulting program is Q3.

  4. Modify Q3 so that whenever it executes any output statement, it then checks in the array A to see if it has written 12 characters or more, and so, whether hello, world are the first 12 characters. In that case call the new function foo(). The resulting program is R, and input z is the same as y.

Now suppose that Q with input y prints hello, world as its first output. Then R as constructed will call foo(). However, if Q with input y does not print hello, world as its first output, then R will never call foo(). If we can decide whether R with input z calls foo(), then we also know whether Q with input y (remember y=z) prints hello, world. Since we know that no algorithm to decide the hello-world problem exists, so the assumption that there is a calls-foo tester is wrong. Thus the calls-foo problem is undecidable.

1.2 Turing Machines

A Turing machine is a theoretical device that manipulates symbols contained on a strip of tape. Despite its simplicity, a Turing machine can be adapted to simulate the logic of any computer algorithm, and is particularly useful in explaining the functions of a CPU inside of a computer. The Turing machine was described in 1936 by Alan Turing3. Turing machines are not intended as a practical computing technology, but rather as a thought experiment representing a computing machine (thus they have never actually been constructed).

B B X2 X1 Xi Xn B B Finite Control

A move of a Turing machine (TM) is a function of the state of the finite control and the tape symbol just scanned.

In one move, the Turing machine will:

  1. Change state.

  2. Write a tape symbol in the cell scanned.

  3. Move the tape head left or right.

1.2.1 Alan Turing and the Entscheidungsproblem

In his momentous paper On Computable Numbers, with an Application to the Entscheidungsproblem [Tur37], Turing reformulated Kurt Gödel's 1931 results on the limits of proof and computation, replacing Gödel's universal arithmetic-based formal language with what are now called Turing machines, formal and simple devices. He proved that some such machine would be capable of performing any conceivable mathematical computation if it were representable as an algorithm.

Turing machines are to this day the central object of study in theory of computation. He went on to prove that there was no solution to the Entscheidungsproblem by first showing that the halting problem for Turing machines is undecidable: it is not possible to decide, in general, algorithmically whether a given Turing machine will ever halt. While his proof was published subsequent to Alonzo Church's equivalent proof in respect to his lambda calculus, Turing's work is considerably more accessible and intuitive. It was also novel in its notion of a Universal Machine, the idea that such a machine could perform the tasks of any other machine. Universal in this context means what is now called programmable. The paper also introduces the notion of definable numbers.

1.3 Turing Machines: Formal Definition

Definition 1.2 [Turing Machine]

Formally, a Turing machine is a 7-tuple

M=(Q,Σ,Γ,δ,q0,B,F)

where:

  1. Q is the finite set of states of the finite control.

  2. Σ is the finite set of input symbols.

  3. Γ is the finite set of tape symbols; ΣΓ.

  4. δ:Q×ΓQ×Γ×{L,R} is the transition function, which is a partial function.

    The arguments of δ(q,X) are a state q and a tape symbol X. The value of δ(q,X), if it is defined is a triple (p,Y,D), where:

    1. p is the next state, in Q.

    2. Y is the symbol, in Γ, written in the cell being scanned, replacing whatever symbol was there.

    3. D is a direction, either L or R, standing for left or right, respectively, and telling us the direction in which the head moves.

  5. q0Q is the start state.

  6. BΓ is the blank symbol. BΣ.

  7. FQ is the set of final or accepting states.

  8. At the begining the TM ist in the start state and the head points to the leftmost input symbol on the tape.

Notation 1.1 [Blank]

We denote the blank symbol B by .

Definition 1.3 [Output]

The output of a TM consists in the longest string w(Γ-{})* immediately following on the right of the actual tape position.

1.3.1 Instantaneous Description and Moves

A Turing machine changes its configuration upon each move.

We use instantaneous descriptions (IDs) for describing such configurations.

Definition 1.4 [Instantaneous Description]

An instantaneous description of a TM is a string of the form

X1X2Xi-1qXiXi+1Xn

where

  1. q is the state of the Turing machine.

  2. The tape head is scanning the ith symbol from the left.

  3. X1X2Xn is the portion of the tape between the leftmost and rightmost nonblanks.

We use M to designate a move of a Turing machine M from one ID to another.

Definition 1.5 [Move]

If δ(q,Xi)=(p,Y,L), then:

X1X2Xi-1qXiXi+1XnMX1X2Xi-2pXi-1YXi+1Xn

If δ(q,Xi)=(p,Y,R), then:

X1X2Xi-1qXiXi+1XnMX1X2Xi-1YpXi-+Xi+2Xn

The reflexive-transitive closure of M is denoted M*.

1.3.2 Language of a TM

Definition 1.6 [Language]

A Turing machine M=(Q,Σ,Γ,δ,q0,B,F) accepts the language

L(M)={wΣ*:q0wM*αpβ,pF,α,βΓ*}

Example 1.3 [TM for L01]

M=({q0,q1,q2,q3,q4},{0,1},{0,1,X,Y,},δ,q0,,{q4}) where δ is given by the following table:

01XY
q0 (q1,X,R) (q3,Y,R)
q1 (q1,0,R)(q2,Y,L) (q1,Y,R)
q2 (q2,0,L) (q0,X,R) (q2,Y,L)
q3 (q3,Y,R) (q4,,R)
*q4

We can also represent M by the following transition diagram:

⬜; ⬜, R X; X, R 0; X, R Y; Y, R Y; Y, R Y; Y, R 0; 0, R Y; Y, L 0; 0, L 1; Y, L q0 q1 q2 q3 q4

As M performs its computation, the portion of the tape, where M's tape head has visited, will always be a sequence of symbols described by the regular expression X*0*Y*1*.

State q0 is the initial state, and M also enters state q0 every time it returns to the leftmost remaining 0. If M is in state q0 and scanning a 0, it will go to state q1 and change the 0 to an X. Once in state q1, M keeps moving right over all 0's and Y's remaining in that state. If M sees an X or a , it dies. If M sees a 1 when in state q1 it changes that 1 to a Y, enters state q2, and starts moving left.

In state q2, M moves left over 0's and Y's, remaining in state q2. When M reaches the rightmost X, which marks the right end of the block of 0's that have already been changed to X, M returns to state q0 and moves right. There are two cases:

  1. If M sees a 0, the it repeats the matching cycle we have just described.

  2. If M sees a Y, then it has changed all 0's to X's. If all 1's have been changed to Y's, then the input was of the form 0n1n, and M should accept. Thus M enters state q3, and starts moving right, over Y's. If the first symbol other than Y that M sees is a blank, then indeed there were an equal number of 0's and 1's, so M enters state q4 and accepts. On the other hand, if M encounters another 1, then there are too many 1's ant M dies without accepting. If it encounters a0, then the input was of the wrong form, and M also dies.

Example 1.4 [Reverse]

Following TM reverses and input string of 0's and 1's.

⬜; ⬜, R ⬜; 0, L 0; 1, R ⬜; 1, L ⬜; 0, L ⬜; ⬜, L 1; 1, R 0; 1, R 1; 0, R 1; 1, L 0; 0, L 0; 0, R 1; ⬜, R 1; 1, L 0; 0, L 0; 0, L 1; 1, L 0; ⬜, R 1; 1, R 0; 0, R 1; 1, R ⬜; ⬜, S 0; 0, R 1; 1, R 0; 0, R ⬜; ⬜, R ⬜; ⬜, R ⬜; 1, L 1; 0, R ⬜; ⬜, R 1 2 3 4 5 6 7 8 9 10 11

In state 1, M reads the leftmost symbol and replaces it by a blank. If the symbol was a 0, M enters state 2 and moves to the right until it reaches the first blank symbol, otherwise M enters state 3 and moves to the right until it reaches the first blank symbol. States 2 and 3 are used to remember the value of the leftmost symbol.

The leftmost symbol is written in state 4 or 5 after the first blank on the right of the input. This may overwrite an already copied leftmost symbol. If it is the case, that symbol will be copied to the right (states 6 and 7). This process continues until the rightmost blank it reached.

The TM now returns to the next leftmost non blank symbol (states 8, 9 and 10). If there is no more such symbol, the TM enters accepting state 11 and halts.

Example 1.5 [Parity]

The input of following TM is a string of 0's and 1's. If the input string has an even number of 1's, the TM will write a 0 at the end of the input and halt, otherwise it will write a 1 at the end of the input and halt.

⬜; 1, S 0; 0, R 0; 0, R ⬜; 0, S 1; 1, R 1; 1, R q0 q1 q2

The TM always moves to the right. If the TM is in state q0, then it has read an even number of 1' so far, otherwise the machine will be in state q1. At the end of the input, the Tm will replace the first blank by a 0 if it was in state q0 and enter q2, otherwise (the TM was in state q1) the Tm will replace the first blank by a 1 and enter q2. The TM halts in accepting state q2.

1.3.3 Acceptance by Halting

A Turing machine halts if it enters a state q, scanning a tape symbol X, and there is no move in this situation, i.e., δ(q,X) is undefined.

We can always assume that a Turing machine halts if it accepts, as we can make δ(q,X) undefined whenever q is an accepting state. We will always assume that a TM halts when it is in an accepting state.

Definition 1.7 [Recursively enumerable Languages]

The set of languages we can accept using a Turing machine is called the the recursively enumerable languages 4 or RE.

Unfortunately, it is not always possible to require that a Turing machine halts even if it does not accept.

Definition 1.8 [Recursive Language]

A recursive language5 is a language corresponding to a TM that eventually halts, regardless of whether or not it is accepting the input.

Remark 1.1 [Recursive Language]

If a TM M always halts, regardless of whether or not it accepts, we can assume that M has two accepting states qaccept and qreject.

1.4 Programming Techniques for Turing Machines

The goal of this chapter is to show what kind of problems a TM can solve. In particular we will show, that a TM can perform calculations on other TM that we saw performed in section 1.1.2 by a program that examined other programs.

In chapter 4 we will show that a TM is exactly as powerful as a conventional computer.

To make the ability of a TM clearer, we shall present a number of examples of how we may think of the tape and the finite control of the TM. None of the following tricks extend the basic model of the TM.

1.4.1 Storage in the State

We can use the finite control of the TM to hold a finite amount of data. The next figure suggests this technique as well as multiple tracks (see next section).

In the figure we see that the finite control not only consits of a state q but also of three data elements A, B and C. The technique requires no extension to the TM model. We merely think of the state as the tuple [q,A,B,C]. Regarding states this way allows to describe transitions in a more systematic way and often to simplify the strategy of the programm.

Example 1.6 [The language (01*)|(10*)]

To recognize the language (01*)|(10*) we can design the following TM

M=(Q,{0,1},{0,1,},δ,[q0,],,{[q1,]})Q={q0,q1}×{0,1,}

The transition function δ of M is as follows:

  1. δ([q0,],a)=([q1,a],a,R) for a=0,1

  2. δ([q1,a],a])=([q1,a],a,R)
    a is the ``complement'' of a (0 if a=1 and 1 if a=0).

  3. δ([q1,a],)=([q1,],R) for a=0,1

Notice that δ([q1,a],a) is undefined. Thus if M encounters a second occurence of the first symbol in the input, it halts witout accepting.

1.4.2 Multiple Tracks

Another usefull trick is to divide the tape of the TM in several tracks. Each track can hold one symbol and the alphabet of the TM consists of tuples, with one component for each track (see the last figure). This technique does not extend the model of the TM. It is simply a way to view tape symbols and to imagine that they have a useful structure. A common use of multiple track is to use one track to mark cells and the second track as the data.

Example 1.7 [Multiple Tracks]

In this example we will use two tracks to recognize the non-context-free language

Lwcw=wcw|w is in (0|1)*M=(Q,Σ,Γ,δ,[q1,)],[,],{[q9,]}
Q:
The set of states is {q0,...q9}×{0,1,}
Σ:
the input symbols are {[,0],[,1][,c]}
Γ:
the set of tape symbols is {,*}×{0,1,c,} we will use the * symbol to check off symbols of the first or second group of 0's and 1's.

The transition function δ is defined by the following rules, in which a and b each my stand for either 0 or 1.

  1. δ([q1,],[,a])=([q2,a],[*,a],R) save the next symbol and ``check off'' the sanned symbol.

  2. δ([q2,a],[,b])=([q2,a],[,b],R) move right searching symbol c.

  3. δ([q2,a],[,c])=([q3,a],[,c],R) symbol c is found.

  4. δ([q3,a],[*,b])=([q3,a],[*,b],R) move right searching the next unchecked symbol.

  5. δ([q3,a],[,a])=([q4,],[*,a],L) the next unchecked symbol matches the symbol in the finite control.

  6. δ([q4,],[*,a])=([q4,],[*,a],L) move left searching for c

  7. δ([q4,],[,c])=([q5,],[,c],L) symbol c is found here we will branch to the begin if the symbol left from c is unchecked else test if all symbols right of c are ``check off''.

  8. δ([q5,],[,a])=([q6,],[,a],L)

    1. δ([q6,],[,a])=([q6,],[,a],L) move left searching the first ``check off'' symbol

    2. δ([q6,],[*,a])=([q1,],[*,a],R) ``check off'' symbol found M enters q1 to check the next symbol

  9. δ([q5,],[*,a])=([q7,],[*,a],R) no more unchecked symbols left of c

    1. δ([q7,],[,c])=([q8,],[,c],R) move over c

    2. δ([q8,],[*,a])=([q8,],[*,a],R) move right over all checked symbols

    3. δ([q8,],[,])=([q9,],[,],R) reaches end of input so accept.

1.4.3 Soubroutines

As with programs in general it is helpful to think of TM as built from a collection of interacting components (subroutines). A TM subroutine is a set of states that perform some useful process. The subroutine consists of a start state and a state that has no moves the ``return'' state. The subroutine starts, when the TM enters its initial state and ends in the ``return'' state.
Since the TM has no mecanisme to remember a ``return address'' we have to copy and rename the states of the subroutine every time it is called.

Example 1.8 [Multiplication]

We shall design a TM for multiplication. The input is a string of the form 0m10n1 and the output is 0mn. Here is a strategy for a TM

  1. The tape will, in general contain a string of the form 0i10n10kn for some k

  2. In one basic step we change the first 0 of the first group to and coppy n 0's to the end of the string. The resulting string is 0i-110n10(k+1)n.

  3. When there is no more 0's in the first group we change we have to change the leading 10n1 to blanks, and we are done.

We will implement the second step as a subroutine Copy with the following ID specification.

0m-k1q00n10(k-1)n   Copy*  0m-k1q40n10kn

The Copy routine and the program are given in the next figures.

1.5 Extensions to the Basic Turing Machine

In this section we will introduce computer models that are related to the TM and have the same language-recognition power as the basic model.
One of these, the multitape TM is important, because it is easier to see how a multitape TM can simulate a conventional computer (or other kind of TM). Yet the extra tapes of the multitape TM adds no power to the model, as far as the ability to accept languages is concerned.

Another model is the nondeterministic TM, an extension of the basic model that is allowed to choose among a finite set of different moves at any state. Also this extension add no power to the model.

1.5.1 Multitape Turing Machine

A multitape TM has a finite control and some finite number k of tapes. Each tape is divided into cells, and each cell can hold any symbol of the finite tape alphabet. For each tape there is a head. The heads can all move independently.

[Initial state:]
  1. The finite input is placed on the first tape.

  2. All other cells of all tapes hold the blank.

  3. The finite control is in the initial state.

  4. The head of the first tape is at the left end of the input.

  5. All other tape heads are at some arbitrary cell.

A move of the multitape TM:

A move of the multitape TM depends on the state and the symbols scanned by each of the tape heads. In one step the multitape TM does the following:
  1. The control enters a new state.

  2. On each tape a new symbol is written on the cell scanned.

  3. Each of the tape heads makes a move, which can be left, right or stationary.

Accepting:

A multitape TM like one-tape TM accept an input by entering an accepting state.

We will see examples of multitape TM later in this document.

1.5.2 Equivalence of One-Tape and Multitape TM's

Recall (see definition 1.7) that the recursively enumerable languages are defined to be those accepted by a one-tape TM. Surely a multitape TM accept the recursively enumerable languages (a one-tape TM is a multitape TM). The qestion is, are there languages that are not recursively enumerable, yet are accepted by a multi-tape TM? The answer is no.

Theorem 1.1 [Multitape TM]

Every language accepted by a multitape TM is recursively enumerable.

Proof: The proof is suggested by the next figure. Suppose the language L is accepted by a k-tape TM M. We simulate M by a one-tape TM N with 2k tracks and k+2 storage components in the finite controll. Half of the tracks hold the tapes of M. The other tracks each hold a single marker that indicates the position of the head for the coresponding tape of M.

To simulate a move of M, N has must visit the k head markers. In a component of the finite control N stores at all time the number of head markers to its left. After visiting all head markers and saving the the scanned symbols in components of its finite control, N knows which symbols are scanned by the k heads of M. N also stores the actual state of M in its finite control and so N knows what move M will make.

Now N revisits each of the head markers, change the symbol and moves the head markers left or right if necessary. Finally, N changes the state of M as recorded in its own finite control. At this point N has simulated one move of M.

The accepting states of N are all states that record M's state as one of the accepting states of M.

We will introduce now the concepts of ``time complexity'' and ``running time'' of a TM.

Definition 1.9 [Running Time]

The running time of TM M on input w is the number of steps that M makes before halting. If M doesn't halt on w then the running time is infinite. We write RTM(w).

Definition 1.10 [Time complexity]

The time complexity of a TM M is a function TM(n)

TM(n)=max{w|wΣ*length(w)=n}(RTM(w))Mwlength(w)=nTM(n)

The next theorem shows that if a multitape TM has a polynomial time complexity, then the simulating one-tape TM from theorem 1.1 also has polynomial complexity.

Theorem 1.2 [Multitape TM and Complexity]

The time taken by the one-tape TM N of theorem 1.1 to simulate n moves of the coresponding k-tape TM M is O(n2).

Proof: After n moves of M, the tape head markers cannot have separated by more than 2n cells. Thus we need no more than O(3n) moves of N to find all head markers. To write the new symbols we need no more than 2n moves and no more than 2k moves to move the head markers to the right.
Thus, the number of moves of N to simulate one step of M is no more than 5n+4k. Since k is a constant the number of moves is O(n). To simulate n moves of M requires no more than O(n2) moves of N.

1.5.3 Nondeterministic TM

A nondeterministic TM (NTM) differs from a deterministic TM in the transition function δ. The transition function is defined as

δ:Q×ΓP(Q×Γ×{L,R}) where P is the power set. These means for every state q and tape symbol X, δ(X) is a set of triples {(q1,Y1,D1),(q2,Y2,D2),...,(qk,Yk,Dk)} where k is any finite integer. The NTM can choose, at any step, any of the triples to be the next move.
The language of an NTM is defined in the expected manner, M accepts an input w if there is a sequence of possible moves that leads fom the initial instantaneous description (ID) to an ID with an accepting state. The existence of other sequences that do not to an accepting state are irrelevant.

The NTM's accept no languages not accepted by a deterministic TM (DTM for short).

Theorem 1.3 [NTM and DTM]

If MN is a NTM, then there is a DTM MD such that

L(MN)=L(MD)

Proof: We design MD as a multitape TM, sketched in the next figure. The first tape of MD holds a sequence of instantaneous descriptions (ID) of MN. One ID is marked as the current ID. In the figure the current ID is ID3 marked by an x along with the intern ID separator, which is a *. All ID's to the left of the marker have been explored and can be ignored subsequently.

To process the current ID, MD does the following:

  1. MD examines the state and the scanned symbol of the current ID. If the state is accepting then MD accepts.

  2. If the state is not accepting, and the state symbol combination has k possible moves, then MD use the second tape to write k copies of the current ID to the end of the sequence of ID's on tape 1.

  3. MD modifies each of those k ID's to simulate the possible k moves of the current ID.

  4. MD returns to the marked ID, erase the mark, and moves the mark to the next ID to the right. The cycle then repeats with step 1.

It should be clear, that MD will only accept if it finds that MN can enter an accepting ID.
We must show that if MN enters an accepting ID after a sequence of n steps, then MD will eventually make that ID the current and accept.
Let m be the maximum number of choices MN has in any configuration. Then there is one initial ID of MN, at most m ID's that MN can reach after one move, m2 ID's MN can reach after two moves, and so on. Those after n moves, MN can reach at most i=0i=nmi ID's. This number is at most nmn.
The order in which MD examines the ID's is breadth first; that is MD will consider all ID's of MN reachable by up to n moves before considering any ID's that are only reachable by more than n moves.
as a consequence, the accepting ID of MN will be considered by MD among the first nmn ID's. Thus if MN accepts, then so does MD. We can conclude that L(MN)=L(MD)

Notice that the constructed DTM may take exponentially more time than the NTM.

1.5.4 TM With Semi-infinite Tapes

While we have allowed the tape head of a TM to move either left or right from its initial position, it is only necessary that the TM's head be allowed to move within the positions at and rigth of the initial head position. In fact, we can assume the tape is semi-infinite, that is, there are no cells to the left of the initial head position.
We shall make one more restriction to our TM: it never writes a blank.
This simple restrictions means that the tape is at all times a prefix of nonblank symbols followed by an infinity of blanks. We shall see later how usefull it is to assume ID's have this form.

Theorem 1.4 [Semi-infinite Tape]

Every language accepted by a TM M2 is also accpted by a TM M1 with the following restrictions:

  1. M1's head never moves left of its initial position.

  2. M1 never writes a blank.

Proof: See exercises.

1.6 Undecidability

Recall that a language L is recursively enumerable if L=L(M) for some TM M (definition 1.7). Also recall that the language is recursive if it is accepted by a TM that always halts, regardless of whether or not it accepts (definition 1.8). In this case we also say that the language is decidable.

Remark 1.2 [undecidable languages]

Because the set of all TM with finite controll is denumerable and the set of all languages is nondenumerable there are a nondenumerable number of languages which are not recursively enumerable.

The goal of this section is to find langues that are not recursively enumerable and languages that are recursively enumerable but not recursive or decidable.

1.6.1 Codes for Turing machines

We now want to represent Turing machines with input alphabet {0,1} by binary strings, so that we can identify Turing machines with integers and refer to the ith Turing machine as Mi.

In what follows, we shall need to assign a unique integer to each binary string.
If w is a binary string, treat 1w as a binary integer i. Then we shall call w the i-th string. Examples:

ε 1 1. String
0 10 2. String
1 11 3. String
00 100 4. Strin
01 101 5. String
...

To represent a Turing machine

M=(Q,{0,1},Γ,δ,q1,,F)

as a binary string, we must first assign integers to the states, tape symbols, and directions L and R:

Encode the transition rule

δ(qi,Xj)=(qk,Xl,Dm) by 0i10j10k10l10m

Note that there are no two consecutive 1's.

Encode an entire Turing machine by concatenating, in any order, the codes Ci of its transition rules, with separating 11:

C111C211...Cn-111Cn

Example 1.9 [TM Code]

Let

M=({q1,q2,q3},{0,1},{0,1,},δ,q1,,{q2})

where δ is defined by:

δ(q1,1)=(q3,0,R)
δ(q3,0)=(q1,1,R)
δ(q3,1)=(q2,0,R)
δ(q3,B)=(q3,1,L).

Codes for the transitions:

0100100010100, 0001010100100, 00010010010100 and 0001000100010010

Code for M:

01001000101001100010101001001100010010010100110001000100010010

Later we shall have need to code pairs consisting of a TM and a binary string (M,w). For this pair we use the code of M followed by 111, followed by w. Note that since no valid code for a TM contains thre 1's in a row, we can be sure that the first occurence of 111 separates M from w.

Definition 1.11 [ith Turing Machine]

Let M be a Turing machine with code wi. We associate the integer i to wi. The ith Turing machine is referred to as Mi.

Many integers do not correspond to any Turing machine, e.g. 1101. If wi is not a valid TM code, then we shall take Mi to be the Turing machine (with one state and no transitions) that immediately halts on any input. Hence L(Mi)= if wi is not a valid TM code.

1.6.2 The Diagonalisation Language

Definition 1.12 [Diagonalisation Language Ld]

The language Ld, the diagonalisation language, is the set of strings wi such that the string wi is not in L(Mi).

That is, Ld consists of all strings w such that the TM M whose code is w does not accept when given w as input.

The reason why Ld is called a ``diagonalisation'' language can be seen if we consider the next figure. This table tells for all i and j, wether the TM Mi accepts input string wj. The i-th row is the characteristic vector for the language L(Mi).

The diagonal values tell whether Mi accepts wi. The complement of the diagonal is the characteristic vetor of the language Ld. The trick with the diagonal is that this characteristic vector disagrees in some columns with every row of our table. Thus the complement of the diagonal cannot be the characteristic vector of any TM.

Theorem 1.5 [Ld]

Ld is not a recursively enumerable language. That is, there is no TM that accepts Ld.

Proof: Suppose Ld where L(M) for some TM M. Since Ld is a language over the alphabet {0,1}, there is at least one code for M, say i; that is M=Mi.
Now ask if wi (the code of Mi) is in Ld

Since wi can neither be in Ld nor fail to be in Ld, we conclude that there is a contradiction to the assumtion that M exists. That is, Ld is not a recursively enumerable language.

1.6.3 A Undecidable Problem that is Recursively Enumerable

Our next goal is to refine the structure of the recursively enumerable languages into two classes. One class, which coresponds to what we think of as an algorithm, has a TM that not only recognize the language, but it also tells us when a string is not in the language. Such a TM always halts eventually, regardless of wether or not it reaches an accepting state.

The second class of languages consists of those recursively enumerable languages that are not accepted by any TM with the guarantee of halting. That is if the input is in the language, we will eventually know that, but in the other case the TM may run forever. An example of such a language is the set of coded pairs (M,w) such that TM M accepts w.

The next figure shows the relationship between the recursive, recursively enumerable and non recursively enumerable languages.

1.6.3.1 Complements of Recursive and Recursively Enumerable Languages

A powerfull tool in proving languages to belong in the second ring of the last figure is to consider the complement of the language. Recursive languages are closed under complementation. Thus if a language L is recursively enumerable, but L¯, the complement of L, is not recursively enumerable, then we know L cannot be recursive.

Theorem 1.6 [Complement]

If L is a recursive language, so is L¯.

Proof: Let L=L(M) for some TM M that always halts. We construct a TM M¯ such that L¯=L¯(M¯) as suggested by the following figure

M is modified as follows to create M¯.

  1. The accepting states of M are changed to non accepting states of M¯ with no transitions; i.e. in these states M¯ will halt without accepting.

  2. M¯ has a new accepting state r.

  3. For each combination of a nonaccepting state of M and a tape symbol of M such that M has no transition, add a transition to r in M¯

Since M is garanteed to halt, we know that M¯ is also garanteed to halt. Moreover M¯ accepts exactly the strings rejected by M. Thus M¯ accepts L¯.

Theorem 1.7 [Complement of RE]

If both a language L and it's complement L¯ are recursively enumerable, then L is recursive. Note that then by theorem 1.6 L¯ is recursive as well.

Proof: The proof is suggested by the next figure. Let L=L(M1) and L¯=L(M2). Both M1 and M2 are simualted in parallel by a TM M. We can make M a two tape TM. The first tape simulates the tape of M1 and the second tape simulates the tape of M2. The states of M1 and M2 are each components of the state of M.

If input w to M is in L then M1 will eventually accept. If so M accepts and halt. If w is not in L, then M2 will accept. In this case M rejects and halt. Thus on all inputs M halts and L(M)=L. We can conclude, that L is recursive.

We can resume theorem 1.6 and 1.7 as follows. For a language L and it's complement L¯ we have the following possibilities:

  1. Both L and L¯ are recursive.

  2. Neither L nor L¯ are recursively enumerable.

  3. L is recursively enumerable but not recursive and L¯ is not recursively enumerable.

  4. L¯ is recursively enumerable but not recursive and L is not recursively enumerable.

1.6.4 The Universal Language

Definition 1.13 [Universal]

We define Lu the universal language, to be the set of binary strings that encode a pair (M,w) where M is a TM with the binary input alphabet and w is a binary string such that wL(M).

We will show that there is a TM U, called the universal TM such that Lu=L(M). Since the input to U is a binary string U is in fact some Mj in the list of binary-input TM (see section 1.6.1)

Theorem 1.8 [Universal Language]

The universal language Lu is recursively enumerable.

Proof: It is easiest to describe U as the multitape TM in the next figure.

M and w are stored on the first tape. The second tape will be used to hold the simulated tape of M, using the same format as for the code of M. That is, tape symbol Xi will be represented by 0i, and the symbols are separated by single 1's. The third tape holds the state of M with state qi represented by 0i
The operation of U can be summarized as follows:

  1. Examine the input to make sure that the code for M is a legal TM. If not halts without accepting.

  2. Initialize the second tape of U. For each 0 of w write 10 on the second tape, and for each 1 of w write 100.

  3. Write 0 (the start state of M) on the third tape and move the head of the second tape to the first simulated cell.

  4. To simulate a move of M, U searches on the first tape for a transition 0i10j10k10l10m such that 0i is the state on tape 3 and 0j is the symbol of M that begins at the position on tape 2 scanned by U. Now simulate the transition:

    1. [(a)] Change the content of tape 3 to 0k

    2. [(b)] Replace 0j on tape 2 by 0l. Use the scratch tape if jl

    3. [(c)] Move the head on tape 2 to the position of the next 1 to the left or right depending on the value of m.

  5. If M has no transition that matches the simulated state and tape symbol, then no transition will be found. Thus, M halts in the simulated configuration, and U must do likewise.

  6. If M enters its accepting state, the U accepts.

In this manner, U simulates M on w. U accepts the coded pair (M,w) if and only if M accepts w.

We can now exhibit a problem that is recursively enumerable but not recursive.

Theorem 1.9 [The Universal Language is not recursive]

Lu is recursively enumerable but not recursive.

Proof: We already know (theorem 1.8) that Lu is recursively enumerable. Suppose Lu were recursive. Then by theorem 1.6 Lu¯ the complement of Lu, would also be recursive. However, if we have a TM M to accept Lu¯, then we can construct a TM to accept Ld. The TM M' is shown in the next figure.

  1. Given string w on its input, M' changes the input to w111w

  2. M' simulates M the TM for Lu on the new input. If w is wi in our enumeration, then M' determines wether Mi accepts wi. Since M accepts Lu¯, it will accept if and only if Mi does not accept wi; .e., wi is in Ld.

Thus M' accepts w if and only if w is in Ld. Since we know M' cannot exist by theorem 1.5, we conclude that Lu is not recursive.

1.7 Mapping Reducibility

We can use reducibility to prove that problems are undecidable.
The notion of reducing one problem to another may be defined formally in one of several ways. Our choice is a simple type of reducibility called mapping reducibility. To do this we first introduce the notion of Turing computable function.

Definition 1.14 [Turing computable function]

A function f:Σ*Σ* is a Turing computable function if some TM M, on every input w, halts with just f(w) on its tape.

Example 1.10 [Turing computable function]

All usual arithmetic operations on integres are computable functions. We can build a TM that takes input m,n and returns m+n or mn.

Computable functions may be transformations of TM descriptions. For example, one computable function take input w and return the description of a TM M' if w=M is an encoding of TM M. The TM M' recognize the same input than M, but never attempts to move its head to the left of the input. The function f accomplishes this task by adding some states to the description of M. f returns ε if w is not a legal encoding of a TM.

Definition 1.15 [mapping reducibility]

Language A is mapping reducible to language B, written AmB, if there is a computable function f:Σ*Σ*, where for every w,

wAf(w)BreductionAB

Theorem 1.10 [decidable language]

  1. [a)] If AmB and B is recursive, then A is recursive.

  2. [b)] If AmB and A is not recursive, then B is not recurcive.

Proof:

  1. [a)] Let M be a TM that recognizes B and f be the reduction from A to B. We construct a TM N for A as follows.

    1. On Input w the TM N first computes f(w).

    2. Now run M on input f(w) and output whatever M outputs.

    Clearly, if wA then f(w)B because f is a reduction from A to B. Thus M accepts f(w) whenever wA. Therefore N works as desired.

  2. [b)] Follows immediatly from a).

As an example, we will show, that the halting problem is not decidable. We define the following Language

HALTTM={M,w|M is a TM and M halts on input w}

Theorem 1.11 [Halting problem]

HALTTM is not recursive

Proof: We show, that we can find a reduction from the universal Language Lu to HALTTM. To do so we must present a computable function f that takes input of form M,w and return output of the form M',w' where

M,wLuM',w'HALTTM The following TM computes a reduction f.
  1. On input M,w construct the following machine M',x

    1. Run M on x

    2. If M accepts then M' accepts

    3. If M rejects then M' enters an infinite loop

  2. Output M',w

We know from theorem 1.9 that Lu is not recursiv so by theorem 1.10 it follows that HALTTM is not recursive.

Theorem 1.12 [recursively enumerable language]

  1. [a)] If AmB and B is recursively enumerable, then A is recursively enumerable.

  2. [b)] If AmB and A is not recursively enumerable, then B is not recurcively enumerable.

Proof: The proof is the same than the proof of theorem 1.10

Because Lu is recursively enumerable (theorem 1.8) it folows from theorem 1.12 that HALTTM is recursively enumerable.


1 The conjecture has been proved by Andrew Wiles in year 1994

2 Kurt Gödel (1906 - 1978) was an Austrian-American logician, mathematician and philosopher. One of the most significant logicians of all time, Gödel made an immense impact upon scientific and philosophical thinking in the 20th century, a time when many, such as Bertrand Russell, A. N. Whitehead and David Hilbert, were pioneering the use of logic and set theory to understand the foundations of mathematics.

3 Alan Turing, 23 June 1912 - 7 June 1954), was an English mathematician, logician, cryptanalyst, and computer scientist. He was influential in the development of computer science and provided an influential formalisation of the concept of the algorithm and computation with the Turing machine. During the Second World War, Turing worked for the Government Code and Cypher School at Bletchley Park, Britain's codebreaking centre. For a time he was head of Hut 8, the section responsible for German naval cryptanalysis. He devised a number of techniques for breaking German ciphers, including the method of the bombe, an electromechanical machine that could find settings for the Enigma machine.

4 A recursively enumerable language is also called Turing-recognizable

5 A recursive language is also called Turing-decidable or simply decidable


[HOME][NEXT][INDEX][BIBLIOGRAPHY]