Lambda Calculus

Lambda calculus is a model of computation. It was introduced in the 1930s by Alonzo Church. He was trying to develop formal foundations for mathematics. His attempts were unsuccessful, as several of his systems turned out to be inconsistent. Nevertheless, he invented lambda calculus as a fragment of his formal system. Using this calculus, he solved the famous Hilbert's Entscheidungsproblem by showing that there are questions that cannot be solved algorithmically. Later, lambda calculus became a formal tool for analyzing computability and functional programming languages.^[1]

Lambda calculus is probably the simplest universal programming language. Many of the key concepts that make functional programming so powerful today were invented for, or are inspired by lambda calculus. This includes for example currying, recursion, and closures. You will also see were all this parenthesis in Racket come from!

Lambda calculus can be viewed as a minimal programming language in which computations can be described and as a mathematical object about which rigorous statements can be proved. It serves as a core language that allows one to specify and study programming language features by translating them into the core language. We will discuss only the untyped variant of lambda calculus. There is also a typed version used in the study of type systems.^[2]

Even though the syntax and semantics of lambda calculus are extremely simple, lambda calculus is Turing complete. In other words, it is possible to simulate any Turing machine with a program in lambda calculus and vice versa.

Syntax

The syntax of lambda calculus has only three types of terms: a variable (denoted by lowercase letters $x,y,z,\dots$), the abstraction of a variable $x$ from a term $t$ defining a function (denoted $\lambda x.t$), and the application of a term $t_1$ to a term $t_2$. Formally, we define lambda terms by the following grammar:

$$\begin{array}{rl}\mathtt{term}& \to \mathtt{var}\mid \mathtt{func}\mid \mathtt{app}\\ \mathtt{func}& \to (\lambda \mathtt{var}.\mathtt{term})\\ \mathtt{app}& \to (\mathtt{term}\mathtt{term})\end{array}$$

The abstraction $\lambda x.t$ defines an anonymous function as in Racket. The variable $x$ is its parameter, and $t$ is its body. The application represents a function call.

Note that each abstraction and application introduce parentheses. To simplify the notation, we use several conventions to remove some of the parentheses.

• We often leave the outermost parentheses.
• The application is left-associative, e.g. $e_1{e}_2{e}_3{e}_4$ is $(((e_1{e}_2)e_3)e_4)$
• The bodies of functions extends to the right as far as possible.

Using the above conventions, we can simplify the following term

$$(\lambda x.((\lambda y.((xy)x))z))$$

as follows:

$$\lambda x.(\lambda y.xyx)z$$

The red parentheses are the only ones that remain because they determine the body of the anonymous function defined by $\lambda y$.

The parentheses are necessary if we want to represent lambda terms as sequences of characters. Alternatively, we avoid parentheses if we understand lambda terms as abstract syntax trees. The above lambda term $\lambda x.(\lambda y.xyx)z$ corresponds to the following tree (the application nodes are denoted by the symbol @):

Before focusing on semantics, we need to introduce the scopes of variables. An occurrence of a variable $x$ is said to be bound if it occurs in the body $t$ of an abstraction $\lambda x.t$. The term $t$ is called scope of the binder $\lambda x$. An occurrence of $x$ is free if it is not bound. To find out whether an occurrence of $x$ is bound or free, we can also track the path in the abstract syntax tree from the occurrence of $x$ to the root. The occurrence is bound if we find the node $\lambda x$ on the path. If not, it is free. In the above term $\lambda x.(\lambda y.xyx)z$, all the occurrences are bound except for the only occurrence of $z$.^[3] A lambda term is called closed (aka combinator) if all its variable occurrences are bound. Otherwise, it is called open.

Semantics

In lambda calculus, programs (i.e., lambda terms) consist of anonymous functions created by abstraction and function calls. The computation of such a program is the simplification process reducing the program to a lambda term that cannot be reduced anymore. This term is the value computed by the program.

A lambda term $\lambda x.t$ represents an anonymous function with an argument $x$ and body $t$. The term $\lambda x.t$ corresponds to the following expression in Racket:

(lambda (x) t)

We can call the function $\lambda x.t$ by applying it to another term $e$: $(\lambda x.t)e$. A lambda term of the form $(\lambda x.t)e$ is called a redex because it is reducible by a substitution rule known as $\beta$-reduction. When we reduce a redex $(\lambda x.t)e$, we substitute $e$ for all free occurrences of $x$ in $t$. The resulting term is denoted $t[x:=e]$.

The $\beta$-reduction can be applied to any redex occurring in a lambda term. It represents a single step in the computation. We denote the relation that a term is $\beta$-reducible to another term by ${\to }^{\beta }$.^[4] For example, $(\lambda x.t)e{\to }^{\beta }t[x:=e]$

Let us see some examples. We denote the equality on lambda terms by $\equiv$.

$$(\lambda x.x)(\lambda y.y){\to }^{\beta }x[x:=(\lambda y.y)]\equiv (\lambda y.y)$$$$\begin{array}{rl}(\lambda x.xx)(\lambda y.y)& {\to }^{\beta }(xx)[x:=(\lambda y.y)]\equiv (\lambda y.y)(\lambda y.y)\\ & {\to }^{\beta }y[y:=(\lambda y.y)]\equiv (\lambda y.y)\end{array}$$$$(\lambda x.x(\lambda x.x))y{\to }^{\beta }(x(\lambda x.x))[x:=y]\equiv y(\lambda x.x)$$

Note that we substitute only for the free occurrence of $x$ in the last example.

The substitution rule has one more caveat regarding the variable names. Consider a redex $(\lambda x.t)e$. A free occurrence of a variable in $e$ may become bound after the $\beta$-reduction. For example,

$$(\lambda x.(\lambda y.xy))y{\to }^{\beta }\lambda y.xy[x:=y]\equiv \lambda y.yy$$

The above reduction step is not valid. The reason is that functions do not depend on the names of their arguments. Consider the following Racket program:

(define y 5)
(define (cadd x) (lambda (y) (+ x y)))

If we now call

(cadd y)

the resulting function cannot be

(lambda (y) (+ y y))

which would be doubling its argument. Instead, it should be the function adding $y=5$ to its argument. To see that, we can simple rename the argument $y$, for instance, to $z$:

> (define (cadd x) (lambda (z) (+ x z)))
> (cadd y) => (lambda (z) (+ y z))
(lambda (z) (+ 5 z))

Thus the correct reduction of the above lambda term should proceed as follows:

$$(\lambda x.(\lambda y.xy))y\equiv (\lambda x.(\lambda z.xz))y{\to }^{\beta }\lambda z.xz[x:=y]\equiv \lambda z.yz$$

Whenever we want to substitute a term $e$ for a variable in a term $t$, we must first check if any free occurrence in $e$ becomes bound. If so, we must rename the clashing bound variable in $t$ to a new name. The renaming of the function arguments is known as $\alpha$-conversion. We will consider two lambda terms that are the same up to renaming a bound variable equivalent. For example,

$$\lambda x.xz\equiv \lambda y.yz$$

Formally, we can define the substitution by induction as follows:

$$\begin{array}{rl}x[x:=e]& =e\\ y[x:=e]& =y\text{if }y\ne x\\ (t_1{t}_2)[x:=e]& =(t_1[x:=e]t_2[x:=e])\\ (\lambda x.t)[x:=e]& =(\lambda x.t)\\ (\lambda y.t)[x:=e]& =(\lambda y.t[x:=e])\text{if }y\ne x\text{ and }y\text{ is not free in }e\\ (\lambda y.t)[x:=e]& =(\lambda z.t[y:=z][x:=e])\text{if }y\ne x\text{ and }y\text{ is free in }e\text{; }z\text{ is a fresh variable}\end{array}$$

The last case is the one where we need to do the $\alpha$-conversion and rename the function argument to a fresh name $\lambda z.t[y:=z]$ before we proceed with the substitution $[x:=e]$.

Church-Rosser Theorems

The computation in lambda calculus is performed by $\beta$-reduction. Given a lambda term, we reduce it as much as possible by repetitive applications of $\beta$-reduction. The resulting irreducible lambda term is the computed value. However, a lambda term can contain several redexes, for example:

$$\underset{\text{redex}}{\underbrace{(\lambda x.x)(\stackrel{\text{redex}}{\overbrace{(\lambda y.y)z}})}}$$

This leads to natural questions: Does it matter which redex we reduce first? Can different reduction orders lead to different final values? Can some reduction orders terminate, whereas some do not? The answers to these questions are provided by the Church-Rosser theorems.

Before we state the Church-Rosser theorems, let us discuss a few examples showing what might happen. First of all, it may happen that the reduction process does not terminate no matter which reduction order is used! Consider the following term containing only a single redex:

$$(\lambda x.xx)(\lambda x.xx){\to }^{\beta }(\lambda x.xx)(\lambda x.xx)$$

The above lambda term corresponds, in fact, to an infinite loop.

For some expressions, the reduction process terminates for some reduction orders and diverges for others. For example, the term

$$(\lambda x.y)((\lambda x.xx)(\lambda x.xx))$$

contains two redexes. The inner one whose reduction does not terminate. If we reduce the redex applying $(\lambda x.y)$ to the rest of the expression, the reduction stops immediately:

$$(\lambda x.y)((\lambda x.xx)(\lambda x.xx)){\to }^{\beta }y$$

A lambda term is said to be in normal form if it is irreducible, i.e., it contains no redexes. Reduction orders are also called evaluation strategies. Several of them were investigated. Let us introduce two of them:

1. Normal order reduces the leftmost outermost redex first at each step.
2. Applicative order reduces the leftmost innermost redex first at each step.

We say a redex is to the left of another redex if its lambda appears further left. The leftmost outermost redex is the leftmost redex not contained in any other redex. The leftmost innermost redex is the leftmost redex, not containing any other redex. For example, consider the following lambda term:

$$(\lambda y.y)((\lambda z.zz)x)((\lambda z.(\lambda a.a)z)(\lambda y.(\lambda z.z)x))$$

It has five redexes depicted in the following figure by the red color.

The leftmost outermost redex is $(\lambda y.y)((\lambda z.zz)x)$. On the other hand, the leftmost innermost redex is $(\lambda z.zz)x$.

Let's see how the above lambda term would be evaluated by the evaluation strategy following the normal order. The reduced redexes are denoted by the line:

$$\begin{array}{rl}& \underset{―}{(\lambda y.y)((\lambda z.zz)x)}((\lambda z.(\lambda a.a)z)(\lambda y.(\lambda z.z)x))\\ & \underset{―}{((\lambda z.zz)x)}((\lambda z.(\lambda a.a)z)(\lambda y.(\lambda z.z)x))\\ & xx(\underset{―}{(\lambda z.(\lambda a.a)z)(\lambda y.(\lambda z.z)x)})\\ & xx(\underset{―}{(\lambda a.a)(\lambda y.(\lambda z.z)x)})\\ & xx(\lambda y.\underset{―}{(\lambda z.z)x})\\ & xx(\lambda y.x)\end{array}$$

Finally, we get to the Church-Rosser theorems. The first theorem states that no matter which reduction order we choose, we will always get the same normal form provided the reduction process terminates. The second theorem states that the normal order always terminates, provided a sequence of $\beta$-reduction steps leads to a normal form.

Church-Rosser Theorems:

1. Normal forms are unique (independently of evaluation strategy). Consequently, a lambda term cannot be reduced to two different normal forms.
2. Normal order always finds a normal form if it exists.

Programming in lambda calculus

Lambda calculus has no numbers, arithmetic, Booleans, etc. It has only anonymous functions and function calls. Thus it might look pretty useless. However, lambda calculus is Turing complete. So it should be no surprise that we can encode all the things like numbers and arithmetic. We will build a few basic encodings from scratch in the following sections. Besides numbers and arithmetic, the most interesting thing is how to represent recursive functions in lambda calculus. That is not straightforward, considering there are only anonymous (nameless) functions.

Although lambda cannot internally introduce names for terms or functions, we can do it in our exposition to simplify the notation. We will denote specific lambda terms (usually combinators) with uppercase letters. For instance, the identity function is denoted by $\mathsf{I}$:

$$\mathsf{I}\equiv \lambda x.x$$

We will also introduce one more convention for writing lambda terms. Each function defined by abstraction is unary. However, we can have functions of higher arity due to currying. For instance, a function of two arguments $x,y$ is defined by $\lambda x.(\lambda y.t)$. To simplify this notation, we often group all the arguments under a single $\lambda$. For example, we replace $\lambda x.(\lambda y.t)$ with $\lambda xy.t$.

Booleans

We start with Boolean values and operations. Since Boolean values often appear in conditional expressions, we model them so that they work directly as the if-then-else expression. The if-then-else expression consists of a condition (i.e., a term whose value is either to true or false) and two expressions to be evaluated depending on the result of the condition. Thus we model Boolean values as binary projection functions:

$$\begin{array}{rl}\mathsf{T}& \equiv \lambda xy.x\\ \mathsf{F}& \equiv \lambda xy.y\end{array}$$

Boolean values are functions of two arguments returning one of them. More precisely, $\mathsf{T}$ representing "true" returns the first argument, whereas $\mathsf{F}$ representing "false" returns the second. Consequently, if we have a lambda term $c{t}_1{t}_2$, where $c$ is a condition term that is evaluated either to $\mathsf{T}$ or $\mathsf{F}$, we get the behavior of the if-then-else expression:

$$\begin{array}{rl}\mathsf{T}ab& {\to }^{\beta }a\\ \mathsf{F}ab& {\to }^{\beta }b\end{array}$$

Considering the encoding of Boolean values, it is easy to encode basic Boolean operations conjunction $\wedge$, $\vee$, and negation $\mathrm{\neg }$. Recall that the conjunction of two arguments $x,y$ is false if the $x$ is false. If $x$ is true, the result of $x\wedge y$ is just $y$. This can be modeled as follows:

$$\wedge \equiv \lambda xy.xy\mathsf{F}$$

Let us check that the encoding works correctly. For an arbitrary lambda term $t$, we have

$$\begin{array}{rl}\wedge \mathsf{F}t& \equiv (\lambda xy.xy\mathsf{F})\mathsf{F}t{\to }^{\beta }\mathsf{F}t\mathsf{F}{\to }^{\beta }\mathsf{F}\\ \wedge \mathsf{T}t& \equiv (\lambda xy.xy\mathsf{F})\mathsf{T}t{\to }^{\beta }\mathsf{T}t\mathsf{F}{\to }^{\beta }t\end{array}$$

Similarly, we can define disjunction that is true if the first argument is true and the value of the second argument otherwise.

$$\vee \equiv \lambda xy.x\mathsf{T}y$$

For an arbitrary lambda term $t$, we have

$$\begin{array}{rl}\vee \mathsf{T}t& \equiv (\lambda xy.x\mathsf{T}y)\mathsf{T}t{\to }^{\beta }\mathsf{TT}t{\to }^{\beta }\mathsf{T}\\ \vee \mathsf{F}t& \equiv (\lambda xy.x\mathsf{T}y)\mathsf{F}t{\to }^{\beta }\mathsf{FT}t{\to }^{\beta }t\end{array}$$

Finally, it is straightforward to encode negation:

$$\mathrm{\neg }\equiv \lambda x.x\mathsf{FT}$$$$\mathrm{\neg }\mathsf{T}\equiv (\lambda x.x\mathsf{FT})\mathsf{T}{\to }^{\beta }\mathsf{TFT}{\to }^{\beta }\mathsf{F},\mathrm{\neg }\mathsf{F}\equiv (\lambda x.x\mathsf{FT})\mathsf{F}{\to }^{\beta }\mathsf{FFT}{\to }^{\beta }\mathsf{T}$$

Numbers and arithmetic

To encode numbers and arithmetic operations, we use the so-called Church numerals. They encode a natural number $n$ as a binary function applying the first argument $n$-times to the second.

$$\begin{array}{rl}\mathsf{0}& \equiv \lambda sz.z\\ \mathsf{1}& \equiv \lambda sz.sz\\ \mathsf{2}& \equiv \lambda sz.s(sz)\\ \mathsf{3}& \equiv \lambda sz.s(s(sz))\\ & ⋮\end{array}$$

Note that $\mathsf{0}\equiv \mathsf{F}$. It is usual in programming that a single value might have two different meanings depending on its context. For instance, the value $65$ can represent the number $65$ or the uppercase letter $A$.

Using the above encodings for numbers, one can easily define the successor function $n\mapsto n+1$.

$$\mathsf{S}\equiv \lambda nxy.x(nxy)$$

The input number $n$ in $(nxy)$ just applies $n$-times $x$ to $y$, i.e., this expression is equivalent to $n$. Next, we add one more application of $x$, i.e., $x(nxy)$. Let us compute the successor of $\mathsf{1}$:

$$\begin{array}{rl}\mathsf{S}\mathsf{1}& \equiv (\lambda nxy.s(nxy))(\lambda sz.sz)\\ & {\to }^{\beta }\lambda xy.x((\lambda sz.sz)xy)\\ & {\to }^{\beta }\lambda xy.x(xy)\equiv \mathsf{2}\end{array}$$

Once we have the successor function $\mathsf{S}$, we can define the addition because we can get the result of $n+m$ by applying $n$-times $\mathsf{S}$ to $m$. Thus we can model addition by the term:

$$\mathsf{A}\equiv \lambda nm.n\mathsf{S}m$$

For example, $2+3$ is computed as follows:

$$\begin{array}{rl}\mathsf{A}\mathsf{23}& {\to }^{\beta }\mathsf{2}\mathsf{S}\mathsf{3}\equiv (\lambda sz.s(sz))\mathsf{S}\mathsf{3}{\to }^{\beta }\mathsf{S}\mathsf{(}\mathsf{S}\mathsf{3}\mathsf{)}{\to }^{\beta }\mathsf{5}\end{array}$$

Multiplication of two numbers $n,m$ can be represented by the term

$$\mathsf{M}\equiv \lambda nmsz.n(ms)z$$

Since $n$ is a numeral, applying it to $(ms)$ and $z$ results in a term applying $n$-times $(ms)$ to $z$, i.e., $(ms)((ms)(\cdots ((ms)z)\cdots ))$. Analogously, each application of $(ms)$ results in $m$-many applications of $s$. Altogether, we get $n\cdot m$ many applications of $s$ to $z$. Let us see an example:

$$\begin{array}{rl}\mathsf{M}\mathsf{23}& \equiv (\lambda nmsz.n(ms)z)\mathsf{23}{\to }^{\beta }\lambda sz.\mathsf{2}(\mathsf{3}s)z{\to }^{\beta }\lambda sz.(\mathsf{3}s)((\mathsf{3}s)z)\\ & {\to }^{\beta }\lambda sz.(\mathsf{3}s)(s(s(sz))){\to }^{\beta }\lambda sz.(s(s(s(s(s(sz))))))\equiv \mathsf{6}\end{array}$$

The multiplication term $\mathsf{M}$ can be further simplified to

$${\mathsf{M}}^{\prime }\equiv \lambda nms.n(ms)$$

We can remove the variable $z$ because $\mathsf{M}$ and ${\mathsf{M}}^{\prime }$ behave identically. To see that, consider an abstraction $\lambda x.(tx)$ for a term $t$. It defines a function of an argument $x$ applying $t$ to $x$. If we apply it to any expression $e$, we obtain $(\lambda x.(tx))e{\to }^{\beta }te$. Thus we can directly replace $\lambda x.(tx)$ just with the term $t$. This simplification is known as $\eta$-reduction. Utilizing $\eta$-reduction to

$$\mathsf{M}\equiv \lambda nmsz.n(ms)z\equiv \lambda nms.(\lambda z.(n(ms))z),$$

we end up with ${\mathsf{M}}^{\prime }$.

In the following section on recursive functions, we will need the predecessor function on natural numbers $n\mapsto n-1$ ($0$ is mapped to $0$). Unfortunately, encoding it is more complex than encoding the successor function $\mathsf{S}$. We first need to introduce an encoding of pairs. Given two terms $t,s$, we define the pair consisting of $t$ and $s$ as follows:

$$⟨t,s⟩\equiv \lambda z.zts$$

Thus the pair $⟨t,s⟩$ is a function of argument $z$ that is applied to the pair's components. Note that this is exactly how we encoded a 2D point as a function closure in Lecture 3.

(define (point x y)
  (lambda (m) (m x y)))

Given a pair, we can access its components by applying it to the projection functions $\mathsf{T}$ and $\mathsf{F}$.

$$\begin{array}{rl}⟨t,s⟩\mathsf{T}& \equiv (\lambda z.zts)\mathsf{T}{\to }^{\beta }\mathsf{T}ts{\to }^{\beta }t\\ ⟨t,s⟩\mathsf{F}& \equiv (\lambda z.zts)\mathsf{F}{\to }^{\beta }\mathsf{F}ts{\to }^{\beta }s\end{array}$$

To create the predecessor function, we need to find a term such that if we apply it $n$-times to another term, it gets evaluated to $n-1$. We first define a function mapping a pair $⟨n,m⟩$ to $⟨n+1,n⟩$.

$$\mathsf{\Phi }\equiv \lambda pz.z(\mathsf{S}(p\mathsf{T}))(p\mathsf{T})$$

The function $\mathsf{\Phi }$ takes a pair $p$, extracts its first component $p\mathsf{T}$, computes its successor $\mathsf{S}(p\mathsf{T})$, and returns a pair consisting of the successor and the first component. The reason why $\mathsf{\Phi }$ is important is that when we apply it $n$-times to the pair $⟨\mathsf{0},\mathsf{0}⟩$, we get the pair $⟨n,n-1⟩$.

$$\mathsf{\Phi }⟨\mathsf{0},\mathsf{0}⟩{\to }^{\beta }⟨\mathsf{1},\mathsf{0}⟩,\mathsf{\Phi }⟨\mathsf{1},\mathsf{0}⟩{\to }^{\beta }⟨\mathsf{2},\mathsf{1}⟩,\mathsf{\Phi }⟨\mathsf{2},\mathsf{1}⟩{\to }^{\beta }⟨\mathsf{3},\mathsf{2}⟩,\dots$$

Consequently, we can define the predecessor function as the function applying $n$-times $\mathsf{\Phi }$ to $⟨\mathsf{0},\mathsf{0}⟩$ and extracting the second component:

$$\mathsf{P}\equiv \lambda n.n\mathsf{\Phi }⟨\mathsf{0},\mathsf{0}⟩\mathsf{F}$$

Zero test

To implement a recursive function, we need a condition when to stop the recursion. In the example we will discuss in the next section, we test whether a given number is zero. To check whether a given number is zero, recall that $\mathsf{0}\equiv \mathsf{F}$ is just the projection to the second argument. Consequently, $\mathsf{0}t\mathsf{T}{\to }^{\beta }\mathsf{T}$ for each lambda term $t$. Thus we need to find a term $t$ such that when applied to $\mathsf{T}$ at least once, it returns $\mathsf{F}$. Such a term $t$ is the constant function always returning $\mathsf{F}$, i.e., $\lambda x.\mathsf{F}$. Altogether, we define

$$\mathsf{Z}\equiv \lambda n.n(\lambda x.\mathsf{F})\mathsf{T}$$

So we have

$$\mathsf{Z}\mathsf{0}\equiv (\lambda n.n(\lambda x.\mathsf{F})\mathsf{T})\mathsf{0}{\to }^{\beta }\mathsf{0}(\lambda x.\mathsf{F})\mathsf{T}{\to }^{\beta }\mathsf{T}$$

and for $\mathsf{N}>\mathsf{0}$

$$\mathsf{ZN}\equiv (\lambda n.n(\lambda x.\mathsf{F})\mathsf{T})\mathsf{N}{\to }^{\beta }\mathsf{N}(\lambda x.\mathsf{F})\mathsf{T}{\to }^{\beta }(\lambda x.\mathsf{F})(\cdots ((\lambda x.\mathsf{F})\mathsf{T})\cdots ){\to }^{\beta }\mathsf{F}$$

Recursive functions

Finally, we are getting to the most exciting construction of how to encode recursive functions if we have only anonymous functions in lambda calculus. Recursive functions can be defined through the so-called $\mathsf{Y}$-combinator.

$$\mathsf{Y}\equiv \lambda y.(\lambda x.y(xx))(\lambda x.y(xx))$$

Note that it is an expanded version of the term $(\lambda x.xx)(\lambda x.xx)$ representing the infinite loop as $\mathsf{Y}$ applies a further argument $y$ to $xx$. Let us see what happens if we apply $\mathsf{Y}$ to any lambda term $\mathsf{R}$.

$$\mathsf{YR}{\to }^{\beta }(\lambda x.\mathsf{R}(xx))(\lambda x.\mathsf{R}(xx))\equiv {\mathsf{R}}^{\prime }$$

We obtained a term denoted ${\mathsf{R}}^{\prime }$, which is something like a recursive version of $\mathsf{R}$. To see that, note that ${\mathsf{R}}^{\prime }$ can produce as many applications of $\mathsf{R}$ as we wish:

$$\begin{array}{rl}{\mathsf{R}}^{\prime }& \equiv (\lambda x.\mathsf{R}(xx))(\lambda x.\mathsf{R}(xx))\\ & {\to }^{\beta }\mathsf{R}((\lambda x.\mathsf{R}(xx))(\lambda x.\mathsf{R}(xx)))\equiv \mathsf{R}{\mathsf{R}}^{\prime }\\ & {\to }^{\beta }\mathsf{R}(\mathsf{R}((\lambda x.\mathsf{R}(xx))(\lambda x.\mathsf{R}(xx))))\equiv \mathsf{R}(\mathsf{R}{\mathsf{R}}^{\prime })\\ & {\to }^{\beta }\dots \end{array}$$

Whether this process stops or not depends on $\mathsf{R}$. If it contains a leftmost outermost redex, whose reduction discards ${\mathsf{R}}^{\prime }$, the computation following the normal order terminates. However, the applicative order need not terminate as it can indefinitely reduce $\mathsf{R}$'.

To see an example, we implement a function computing for a given natural number $n$ the sum $\sum _{i=0}^{n}i$. We can define the function recursively using the fact $\sum _{i=0}^{n}i=n+\sum _{i=0}^{n-1}i$. In Racket, we would implement such a function as follows:

(define (sum-to n)
  (if (= n 0)
      0
      (+ n (sum-to (- n 1)))))

We must apply the $\mathsf{Y}$-combinator in lambda calculus instead. We define a function corresponding to the body of sum-to, but we replace the recursive call with a call of a function given as an argument.

$$\mathsf{R}\equiv \lambda rn.\mathsf{Z}n\mathsf{0}(n\mathsf{S}(r(\mathsf{P}n)))$$

The function tests if $n$ is zero by $\mathsf{Z}n$. If it is zero, $\mathsf{Z}n$ evaluates to $\mathsf{T}$, which consequently returns its first argument, i.e., $\mathsf{0}$. Otherwise, $\mathsf{Z}n$ evaluates to $\mathsf{F}$, which returns $n\mathsf{S}(r(\mathsf{P}n))$. This expression is nothing else than $n+r(n-1)$.

Now it remains to turn $\mathsf{R}$ into its recursive version by the $\mathsf{Y}$-combinator. We define

$${\mathsf{R}}^{\prime }\equiv \mathsf{YR}$$

Let us see if it correctly sums up all the natural numbers to $3$. Recall that ${\mathsf{R}}^{\prime }{\to }^{\beta }{\mathsf{RR}}^{\prime }$.

$$\begin{array}{rl}{\mathsf{R}}^{\prime }\mathsf{3}& \equiv \mathsf{YR}\mathsf{3}{\to }^{\beta }\mathsf{R}{\mathsf{R}}^{\prime }\mathsf{3}\equiv \mathsf{Z}\mathsf{30}(\mathsf{3}\mathsf{S}({\mathsf{R}}^{\prime }(\mathsf{P}\mathsf{3})))\\ & {\to }^{\beta }\mathsf{F}\mathsf{0}(\mathsf{3}\mathsf{S}({\mathsf{R}}^{\prime }(\mathsf{P}\mathsf{3}))){\to }^{\beta }\mathsf{3}\mathsf{S}({\mathsf{R}}^{\prime }(\mathsf{P}\mathsf{3}))\\ & {\to }^{\beta }\mathsf{3}\mathsf{S}({\mathsf{R}}^{\prime }\mathsf{2}){\to }^{\beta }\mathsf{3}\mathsf{S}({\mathsf{RR}}^{\prime }\mathsf{2})\\ & {\to }^{\beta }\mathsf{3}\mathsf{S}(\mathsf{Z}\mathsf{20}(\mathsf{2}\mathsf{S}({\mathsf{R}}^{\prime }(\mathsf{P}\mathsf{2})))){\to }^{\beta }\mathsf{3}\mathsf{S}(\mathsf{2}\mathsf{S}({\mathsf{R}}^{\prime }\mathsf{1}))\\ & {\to }^{\beta }\mathsf{3}\mathsf{S}(\mathsf{2}\mathsf{S}({\mathsf{RR}}^{\prime }\mathsf{1})){\to }^{\beta }\mathsf{3}\mathsf{S}\mathsf{(}\mathsf{2}\mathsf{S}\mathsf{(}\mathsf{1}\mathsf{S}\mathsf{(}{\mathsf{R}}^{\prime }\mathsf{0}\mathsf{)}\mathsf{)}\mathsf{)}\\ & {\to }^{\beta }\mathsf{3}\mathsf{S}\mathsf{(}\mathsf{2}\mathsf{S}\mathsf{(}\mathsf{1}\mathsf{S}\mathsf{(}{\mathsf{RR}}^{\prime }\mathsf{0}\mathsf{)}\mathsf{)}\mathsf{)}\equiv \mathsf{3}\mathsf{S}\mathsf{(}\mathsf{2}\mathsf{S}\mathsf{(}\mathsf{1}\mathsf{S}\mathsf{(}\mathsf{Z}\mathsf{00}\mathsf{(}\mathsf{0}\mathsf{S}\mathsf{(}{\mathsf{R}}^{\prime }\mathsf{(}\mathsf{P}\mathsf{0}\mathsf{)}\mathsf{)}\mathsf{)}\mathsf{)}\mathsf{)}\mathsf{)}\\ & {\to }^{\beta }\mathsf{3}\mathsf{S}\mathsf{(}\mathsf{2}\mathsf{S}\mathsf{(}\mathsf{1}\mathsf{S}\mathsf{(}\mathsf{T}\mathsf{0}\mathsf{(}\mathsf{0}\mathsf{S}\mathsf{(}{\mathsf{R}}^{\prime }\mathsf{(}\mathsf{P}\mathsf{0}\mathsf{)}\mathsf{)}\mathsf{)}\mathsf{)}\mathsf{)}\mathsf{)}{\to }^{\beta }\mathsf{3}\mathsf{S}\mathsf{(}\mathsf{2}\mathsf{S}\mathsf{(}\mathsf{1}\mathsf{S}\mathsf{0}\mathsf{)}\mathsf{)}{\to }^{\beta }\mathsf{6}\end{array}$$

Note that the expression in the last line contains several redexes. The one hidden in ${\mathsf{R}}^{\prime }$ can be reduced forever. On the other hand, if we reduce $\mathsf{T}\mathsf{0}\mathsf{(}\mathsf{0}\mathsf{S}\mathsf{(}{\mathsf{R}}^{\prime }\mathsf{(}\mathsf{P}\mathsf{0}\mathsf{)}\mathsf{)}\mathsf{)}{\to }^{\beta }\mathsf{0}$, the term ${\mathsf{R}}^{\prime }$ disappears. This is why the normal order terminates in this computation, whereas the applicative one does not.

---

1. If you are interested in the history, check the entry on Alonzo Church in the Stanford Encyclopedia of Philosophy. ↩︎

2. Benjamin C. Pierce: Types and programming languages. MIT Press 2002, ISBN 978-0-262-16209-8, pp. I-XXI, 1-623. ↩︎

3. Note that the terminology on bound and free occurrences is completely analogous to the terminology used in first-order logic, where variables can be bound by quantifiers instead of $\lambda$. ↩︎

4. We will abuse the notation and write $t{\to }^{\beta }s$ even if several $\beta$-reductions are needed to get from $t$ to $s$. ↩︎