I will give several reasons, which are all essentially the same, why “there is no difference between $\phi$ and $\forall x \,.\, \phi$” is a really bad opinion to have.

Reason 1: you wouldn’t equate a function with its definite integral

You would not claim that a real-valued function $f : \mathbb{R} \to \mathbb{R}$ is “the same thing” as its definite integral $\int_{\mathbb{R}} f(x) \, d x$, would you? One is a real function, the other is a real number. Likewise, $\phi$ is a truth function and $\forall x \,.\, \phi(x)$ is a truth value.

Reason 2: functions are not their own values

To be quite precise, the expression $\phi$ by itself is not a function, just like the expression $x + \sin x$ is not a function. To make it into a function we must first abstract the variable $x$, which is usually written as $x \mapsto x + \sin x$, or $\lambda x \,.\, x + \sin x$, or fun x -> x +. sin x. In logic we indicate the fact that $\phi$ is a function by putting it in a context, so we write something like $x : \mathbb{R} \vdash \phi$.

Why is all this nit-picking necessary? Try answering these questions with “yes” and “no” consistently:

1. Is $x + \sin x$ a function in variable $x$?
2. Is $x + \sin x$ a function in variables $x$ and $y$?
3. Is $y – y + x + \sin x$ a function in variables $x$ and $y$?
4. Is $x + \sin x = y – y + x + \sin x$?

A similar sort of mistake happens in algebra where people think that polynomials are functions. They are not. They are elements of a certain freely generated ring.

Reason 3: They are not logically equivalent

It is absurd to claim that $\phi$ and $\forall x \in \mathbb{R} \,.\, \phi$ are logically equivalent statements. Suppose $\forall x \in \mathbb{R} \,.\, x > 2$ were equivalent to $x > 2$. Then I could replace one by the other in any formula I wish. So I choose the formula $\exists x \in \mathbb{R} \,.\, x > 2$. It must be equivalent to $\exists x \in \mathbb{R} \,.\, \forall x \in \mathbb{R} \,.\, x > 2$, but since $\forall x \in \mathbb{R} \,.\, x > 2$ is false, we get $\exists x \in \mathbb{R} \,.\, \bot$, which is false. We proved that there is no number larger than 2.

Reason 4: They are not inter-derivable

If you can tell the difference between an implication and logical entailment, perhaps you might try to counter reason 3 by pointing out that $\phi$ and $\forall x \,.\, \phi$ are either both derivable, or both not derivable. That is to say, we can prove one if, and only if, we can prove the other. But again, this is not the case. We can prove $\forall x \in \emptyset \,.\, \bot$ but we cannot prove $\bot$.

Reason 5: Bound variables can be renamed but free variables cannot

The formula $x > 2$ is obviously not the same thing as the formula $y > 2$. But the formula $\forall x \in \mathbb{R} . x > 2$ is actually the same as $\forall y \in \mathbb{R} . y > 2$. If you find this confusing it is because you were never taught properly how to handle free and bound variables.

Reason 6: You cannot prove $\forall x \,.\, \phi$ without allowing $x$ to become free

Perhaps we can just forbid free variables altogether and stipulate that all variables must always be quantified. But how are you then going to prove $\forall x \in \mathbb{R} \,.\, \phi$? The usual way

“Consider any $x \in \mathbb{R}$. Then bla bla bla, therefore $\phi$.”

is now forbidden because the first sentence introduces $x$ as a free variable.

We can abolish variables altogether if we wish, by resorting to combinators, but it makes no sense to keep variables and make them all bound all the time.

Epilogue: so in what sense are they the same?

There is a theorem in model theory:

Let $\phi$ be a formula in context $x_1, \ldots, x_n$ and $M$ a structure in which we can interpret $\phi$. The following are equivalent:

1. the universal closure $\forall x_1, \ldots, x_n \,.\, \phi$ is valid in $M$,
2. for every valuation $\nu : \lbrace x_1, \ldots, x_n \rbrace \to M$, $\phi[\nu]$ is valid in $M$.

This is sometimes abbreviated (quite inaccurately) as “a formula and its universal closure are semantically equivalent”. This theorem is causing a lot of harm because mathematicians interpret it as “free variables are implicitly universally bound”. But the theorem itself clearly distinguishes a formula from its universal closure. It has a limited range of applications in model theory. It is not a general reasoning principle that would allow you to dispose of thinking about free variables.

You are in good company. Philosophers have thought about free variables for millennia, although they phrase the problem in the language of universals and particulars. They wonder whether “dog” is the same thing as the set of all dogs, or perhaps there is an ideal dog which is “pure dogness”, but then do we need two ideal dogs to make ideal pups, etc. The answer is simple: a free variable is a projection from a cartesian product.

Make a guess, how many lines of code does it take to implement a dependent type theory with universes, dependent products, a parser, lexer, pretty-printer, and a toplevel which uses line-editing when available?

If you ever looked at my Programming languages zoo you know it does not take that many lines of code to implement a toy language. On the other hand, dependent type theory is different from a typical compiler because we cannot meaningfully separate the type checking, compilation, and execution phases.

Dan Licata pointed me to A Tutorial Implementation of a Dependently Typed Lambda Calculus by Andreas Löh, Connor McBride, and Wouter Swierstra which is similar to this one. It was a great inspiration to me, and you should have a look at it too, because they do things slightly differently: they use de Bruijn indices, they simplify things by assuming (paradoxically!) that $\mathtt{Type} \in \mathtt{Type}$, and they implement the calculus in Haskell, while we are going to do it in OCaml.

A minimal type theory

I am going to assume you are already familiar with Martin-Löf dependent type theory. We are going to implement:

• a hierarchy of universes $\mathtt{Type}_0$, $\mathtt{Type}_1$, $\mathtt{Type}_2$, …
• dependent products $\prod_{x : A} B$
• functions $\lambda x : A . e$, and
• application $e_1 \; e_2$.

We are not going to write down the exact inference rules, although that would be a good idea in a serious experiment. Instead, we are going to read them off later by looking at the source code.

Syntax

We can directly translate the above to abstract syntax of expressions (in the code below think of the type variable as string, we will explain it later):

(** Abstract syntax of expressions. *)
type expr =
  | Var of variable
  | Universe of int
  | Pi of abstraction
  | Lambda of abstraction
  | App of expr * expr

(** An abstraction [(x,t,e)] indicates that [x] of type [t] is bound in [e]. *)
and abstraction = variable * expr * expr

We choose a concrete syntax that is similar to that of Coq:

• universes are written Type 0, Type 1, Type 2, …
• the dependent product is written forall x : A, B,
• a function is written fun x : A => B,
• application is juxtaposition e1 e2.

If x does not appear freely in B, then we write A -> B instead of forall x : A, B.

In this tutorial we are not going to learn how to write a lexer and a parser, but see a comment about it below.

Substitution

One way or another we have to deal with substitution. We could try to avoid it by compiling into OCaml functions, or we could use de Bruijn indices. The expert opinion is that de Bruijn indices are the way to go, but I want to keep things as simple as possible for now, so let us just implement substitution.

Substitution must avoid variable capture. This means that we have to be able to generate new variable names. We can do it by simply generating an infinite sequence of them $x_1, x_2, x_3, \ldots$ But what it the user already used $x_3$, then we should not reuse it? To solve the problem we define a datatype of variable names like this:

type variable =
 | String of string
 | Gensym of string * int
 | Dummy

When the user types x3 we represent this as String "x3", whereas we generate variables of the form Gensym("x",3). We make sure that the integer is unique, so the variable is fresh. The string is a hint to the pretty-printer, which should try to print the generated variable as the string, if possible. For example, suppose we have a $\lambda$-abstraction

Lambda (String "x", ..., ...)

and because of substitutions we refreshed the variable to something like

Lambda (Gensym("x", 4124), ..., ...)

It would be silly to print this as fun x4124 : ... => ... if we could first rename the bound variable back to x, or to x1 if x is taken already. This is exactly what the pretty printer will do.

The Dummy variable is one that is never used. It only appears for the purposes of pretty printing.

Here is the substitution code:

(** [refresh x] generates a fresh variable name whose preferred form is [x]. *)
let refresh =
  let k = ref 0 in
    function
      | String x | Gensym (x, _) -> (incr k ; Gensym (x, !k))
      | Dummy -> (incr k ; Gensym ("_", !k))

(** [subst [(x1,e1); ...; (xn;en)] e] performs the given substitution of
    expressions [e1], ..., [en] for variables [x1], ..., [xn] in expression [e]. *)
let rec subst s = function
  | Var x -> (try List.assoc x s with Not_found -> Var x)
  | Universe k -> Universe k
  | Pi a -> Pi (subst_abstraction s a)
  | Lambda a -> Lambda (subst_abstraction s a)
  | App (e1, e2) -> App (subst s e1, subst s e2)

and subst_abstraction s (x, t, e) =
  let x' = refresh x in
    (x', subst s t, subst ((x, Var x') :: s) e)

Type inference

Our calculus is such that an expression has at most one type, and when it does the type can be inferred from the expression. Therefore, we are going to implement type inference. During inference we need to carry around a context which maps variables to their types. And since we will allow global definitions on the toplevel, the context should also store (optional) definitions. So we define contexts to be association lists.

type context = (Syntax.variable * (Syntax.expr * Syntax.expr option)) list

We need functions that lookup up types and values of variables, and one for extending a context with a new variable:

(** [lookup_ty x ctx] returns the type of [x] in context [ctx]. *)
let lookup_ty x ctx = fst (List.assoc x ctx)

(** [lookup_ty x ctx] returns the value of [x] in context [ctx], or [None]
    if [x] has no assigned value. *)
let lookup_value x ctx = snd (List.assoc x ctx)

(** [extend x t ctx] returns [ctx] extended with variable [x] of type [t],
    whereas [extend x t ~value:e ctx] returns [ctx] extended with variable [x]
    of type [t] and assigned value [e]. *)
let extend x t ?value ctx = (x, (t, value)) :: ctx

Notice that extending with a variable which already appears in the context shadows the old variable, as it should.

We said we would read off the typing rules from the source code:

(** [infer_type ctx e] infers the type of expression [e] in context [ctx].  *)
let rec infer_type ctx = function
  | Var x ->
    (try lookup_ty x ctx
     with Not_found -> Error.typing "unkown identifier %t" (Print.variable x))
  | Universe k -> Universe (k + 1)
  | Pi (x, t1, t2) ->
    let k1 = infer_universe ctx t1 in
    let k2 = infer_universe (extend x t1 ctx) t2 in
      Universe (max k1 k2)
  | Lambda (x, t, e) ->
    let _ = infer_universe ctx t in
    let te = infer_type (extend x t ctx) e in
      Pi (x, t, te)
  | App (e1, e2) ->
    let (x, s, t) = infer_pi ctx e1 in
    let te = infer_type ctx e2 in
      check_equal ctx s te ;
      subst [(x, e2)] t

Ok, here we go:

1. The type of a variable is looked up in the context.
2. The type of $\mathtt{Type}_k$ is $\mathtt{Type}_{k+1}$.
3. The type of $\prod_{x : T_1} T_2$ is $\mathtt{Type}_{\max(k, m)}$ where $T_1$ has type $\mathtt{Type}_k$ and $T_2$ has type $\mathtt{Type}_m$ in the context extended with $x : T_1$.
4. The type of $\lambda x : T \; . \; e$ is $\prod_{x : T} T’$ where $T’$ is the type of $e$ in the context extended with $x : T$.
5. The type of $e_1 \; e_2$ is $T[x/e_2]$ where $e_1$ has type $\prod_{x : S} T$ and $e_2$ has type $S$.

The typing rules refer to auxiliary functions infer_universe, infer_pi, and check_equal, which we have not defined yet. The function infer_universe infers the type of an expression, makes sure that the type is of the form $\mathtt{Type}_k$, and returns $k$. A common mistake is to think that you can implement it like this:

(** Why is this infer_universe wrong? *)
and bad_infer_universe ctx t =
    match infer_type ctx t with
      | Universe k -> u
      | App _ | Var _ | Pi _ | Lambda _ -> Error.typing "type expected"

This will not do. For example, what if infer_type ctx t returns the type $(\lambda x : \mathtt{Type}_{4} \; . \; x) \mathtt{Type}_3$? Then infer_universe will complain, because it does not see that the type it got is equal to $\mathtt{Type}_3$, even though it is not syntactically the same expression. We need to insert a normalization procedure which converts the type to a form from which we can read off its shape:

(** [infer_universe ctx t] infers the universe level of type [t] in context [ctx]. *)
and infer_universe ctx t =
  let u = infer_type ctx t in
    match normalize ctx u with
      | Universe k -> k
      | App _ | Var _ | Pi _ | Lambda _ -> Error.typing "type expected"

We shall implement normalization in a moment, but first we write down the other two auxiliary functions:

(** [infer_pi ctx e] infers the type of [e] in context [ctx], verifies that it is
    of the form [Pi (x, t1, t2)] and returns the triple [(x, t1, t2)]. *)
and infer_pi ctx e =
  let t = infer_type ctx e in
    match normalize ctx t with
      | Pi a -> a
      | Var _ | App _ | Universe _ | Lambda _ -> Error.typing "function expected"

(** [check_equal ctx e1 e2] checks that expressions [e1] and [e2] are equal. *)
and check_equal ctx e1 e2 =
  if not (equal ctx e1 e2)
  then Error.typing "expressions %t and %t are not equal" (Print.expr e1) (Print.expr e2)

Normalization and equality

We need a function normalize which takes an expression and “computes” it, so that we can tell when something is a universe, and when something is a function. There are several strategies on how we might do this, and any will do as long as we have the following property: if $e_1$ and $e_2$ are equal (type theorists say that they are judgmentally equal, or sometimes that they are definitionally equal) then after normalization they should become syntactically equal, up to renaming of bound variables.

Our judgmental equality essentially has just two simple rules, $\beta$-reduction and unfolding of definitions. So this is what the normalization procedure does:

(** [normalize ctx e] normalizes the given expression [e] in context [ctx]. It removes
    all redexes and it unfolds all definitions. It performs normalization under binders.  *)
let rec normalize ctx = function
  | Var x ->
    (match
        (try lookup_value x ctx
         with Not_found -> Error.runtime "unkown identifier %t" (Print.variable x))
     with
       | None -> Var x
       | Some e -> normalize ctx e)
  | App (e1, e2) ->
    let e2 = normalize ctx e2 in
      (match normalize ctx e1 with
        | Lambda (x, _, e1') -> normalize ctx (subst [(x,e2)] e1')
        | e1 -> App (e1, e2))
  | Universe k -> Universe k
  | Pi a -> Pi (normalize_abstraction ctx a)
  | Lambda a -> Lambda (normalize_abstraction ctx a)

and normalize_abstraction ctx (x, t, e) =
  let t = normalize ctx t in
    (x, t, normalize (extend x t ctx) e)

How about testing for equality of expressions, which was needed in the rule for application? We normalize, then compare for syntactic equality. We make sure that in comparison of abstractions both bound variables are the same:

(** [equal ctx e1 e2] determines whether normalized [e1] and [e2] are equal up to renaming
    of bound variables. *)
let equal ctx e1 e2 =
  let rec equal e1 e2 =
    match e1, e2 with
      | Var x1, Var x2 -> x1 = x2
      | App (e11, e12), App (e21, e22) -> equal e11 e21 && equal e12 e22
      | Universe k1, Universe k2 -> k1 = k2
      | Pi a1, Pi a2 -> equal_abstraction a1 a2
      | Lambda a1, Lambda a2 -> equal_abstraction a1 a2
      | (Var _ | App _ | Universe _ | Pi _ | Lambda _), _ -> false
  and equal_abstraction (x, t1, e1) (y, t2, e2) =
    equal t1 t2 && (equal e1 (subst [(y, Var x)] e2))
  in
    equal (normalize ctx e1) (normalize ctx e2)

And that is it! We have the core of the system written down. The rest is just chrome: lexer, parser, toplevel, error reporting, pretty printer. Those are the things nobody ever explains because they are boring as soon as you have managed to implement them once. Nevertheless, let us have a brief look. The code is accessible at the Github project andrejbauer/tt (the branch blog-part-I).

The infrastructure

Parser: parser.mly and lexer.mll

You never ever want to write a parser with your bare hands. Insetad, you should use a parser generator. There are many, I used menhir. Parser generators can be a bit scary, but a good way to get started is to take someone else’s parser and fiddle with it.

Pretty printer: print.ml and beautify.ml

Pretty printing is the opposite of parsing. In a usual programming language we do not have to print expressions with bound variables (because they get converted to non-printable closures), but here we do. It is worthwhile renaming the bound variables before printing them out, which is what Beautify.beautify does.

Error reporting: error.ml

Not surprisingly, errors are reported with exceptions. The only thing to note here is that it should be possible to do pretty printing in error messages, otherwise you will be tempted to produce uninformative error messages. Our implementation of course does that.

Toplevel: tt.ml

The toplevel does nothing suprising. After parsing the command-line arguments and loading files, it enters an interactive toplevel loop. One neat trick that the toplevel does is that it looks for rlwrap or ledit and wraps itself with it. This gives us line-editing capabilities for free.

The toplevel commands are:

• Help. print a description of toplevel commands.
• Context. print current context.
• Parameter x : t. assume that variable x has type t.
• Definition x := e. define x to be e.
• Check e. infer the type of e.
• Eval e. normalize e.

Here is a sample session:

tt blog-part-I
[Type Ctrl-D to exit or "Help." for help.]
# Parameter N : Type 0.
N is assumed
# Parameter z : N. Parameter s : N -> N.
z is assumed
s is assumed
# Definition three := fun f : N -> N => fun x : N => f (f (f x)).
three is defined
# Context.
three = fun f : N -> N => fun x : N => f (f (f x))
    : (N -> N) -> N -> N
s : N -> N
z : N
N : Type 0
# Check (three (three s)).
three (three s)
    : N -> N
# Eval (three (three s)) z.
    = s (s (s (s (s (s (s (s (s z))))))))
    : N

Where to go from here?

The whole program is 618 lines of code, and only 312 if we discount empty lines and comments. The core is just 92 lines, the rest is infrastructure. Not too bad. There are many ways in which we can improve tt, such as:

1. Improve efficiency by implementing de Bruijn indice or some other mechanism that avoids substitutions.
2. Improve the normalization procedure so that it unfolds definitins on demand, rather than eagerly.
3. Improve the parser so that it accepts more flexible syntax.
4. Improve type inference so that not all bound variables have to be explicitly typed.
5. Add basic datatypes unit, bool and nat.
6. Add simple products, coproducts and dependent sums.
7. Implement a cummulative hierachy so that $\mathtt{Type}_k$ is a subtype of $\mathtt{Type}_{k+1}$.
8. Add inductive datatypes.
9. Add a stronger judgmental equality, for example $\eta$-reduction.
10. Implement tactics and an interactive proof mode.
11. Rule the world.

I may do some of these in subsequent blog posts, if there is interest. Or if you do it, make a pull request on git and write a guest blog post!

What could be more appealing to a mathematician than the idea that there is not one, but many, infinitely many worlds of mathematics? Would he not want to visit them all, understand how they are related, and see what happens to his favorite subject as he moves between them?

Let us consider an example. The real numbers are a mathematical object of fundamental importance, and have many aspects:

1. The reals as a set are uncountable and in bijection with the powerset of natural numbers.
2. The reals as an algebraic structure form a linearly ordered field.
3. The reals as a space are locally compact, Hausdorff, and connected.
4. The reals are a measurable space on which measure theory rests.
5. The reals of non-standard analysis contain infinitesimals.
6. The reals as understood by Leibniz contain nilpotent infinitesimals.
7. The reals as Brouwerian continuum cannot be decomposed into two disjoint inhabited subsets.
8. The reals are overt.

We can have some of these properties but not all at once. History has chosen for us a combination that is taught today as a dogma. Any attempt to deviate from it is met with opposition. Thus you probably consider 1, 2, 3, and 4 as true, 5 as something exotic you heard of, 6 as Leibniz’s biggest mistake, 7 as intuitionistic hallucination (because obviously the reals can be decomposed into the non-negative and negative numbers), and 8 as something you never heard of (but you should have because it is the concept dual to compactness and you have been using it all your life).

Once we break free from Cantor’s paradise that Hilbert threw us in we discover unsuspected possibilities:

1. It may happen that the reals are in 1-1 correspondence with a subset of the natural numbers, while at the same time they form an uncountable set.
2. It may happen that the reals form a proper class.
3. It may happen that every real number has a Turing machine computing its digits.
4. It may happen that the reals are not linearly ordered.
5. It may happen that the reals are locally non-compact, in the sense that every interval contains a sequence without an accumulation point.
6. It may happen that every subset of the reals is measurable.
7. It may happen that the reals can be covered by a sequence of intervals whose cumulative length does not exceed $1$.
8. It may happen that the reals contain nilpotent infinitesimals, which validate the 17th century calculations that physicists still use because, luckily, they did not subscribe entirely to the $\epsilon\delta$-dogma of analysis.
9. It may happen that every real function is continuous, and consequently the reals are not decomposable into two disjoint inhabited subsets.
10. It may happen that the reals are not overt, whatever that means.

It should be admitted that some of the possibilities are rather bizarre. For example, I do not know what good it is to have the reals as a subset of the naturals, but I am sure somebody could think of something. But why should a measure theorist ignore a world of mathematics in which every subset is measurable, or a computer scientist one in which all reals are computable, or a topologist one in which all functions are continuous, or an analyst one in which all functions are smooth?

I am not proposing that mathematics should be compartmentalized so that each branch sits in its world of mathematics, incompatible with others. That would be a grave mistake indeed. In fact, the unification of mathematics under the umbrella of classical set theory has been immensely successful precisely because it allowed mathematicians to discover deep and unsuspected connections between different branches of mathematics. We have learnt to look for connections between branches of mathematics, and now we must also learn to look for connections that span worlds of mathematics.

We cannot ignore the many worlds of mathematics. Therefore, mathematics must become applicable in a wide variety of worlds. Mathematicians have to be educated so that they develop multiple mathematical intuitions that help them feel how the worlds of mathematics behave.

I have so far not given you any technical definition of a mathematical world. Such a definition may be useful for showing meta-theorems, but I think it can never be exhaustive. A world of mathematics may be a forcing extension of set theory, or a topos, or a pretopos, or a model of type theory, or any other structure within which it is possible to interpret the basic language of mathematics.

At the moment I am visiting the Institute for Advanced Study as a member of the Univalent Foundations group. We are building a new foundation of mathematics whose language is type theory rather than set theory, and whose primary objects are homotopy types and not just bare sets. Do I think this is an exciting new development? Certainly! Will the Univalent foundations disrupt the monopoly of Set-theoretic foundations? I certainly hope so! Will it become the new monopoly? It must not!