# fun a b c -> (a b) (fun d -> d c) ;;
- : ('a -> (('b -> 'c) -> 'c) -> 'd) -> 'a -> 'b -> 'd = <fun>

# fun a b c d e e' f g h i j k l m n o o' o'' o''' p q r r' s t u u' v w x y z ->
    q u i c k b r o w n f o' x j u' m p s o'' v e r' t h e' l a z y d o''' g;;
  - : 'a -> 'b -> 'c -> 'd -> 'e -> 'f -> 'g -> 'h -> 'i -> 'j ->
    'k -> 'l -> 'm -> 'n -> 'o -> 'p -> 'q -> 'r -> 's -> 't ->
    ('u -> 'j -> 'c -> 'l -> 'b -> 'v -> 'p -> 'w -> 'o -> 'g ->
     'q -> 'x -> 'k -> 'y -> 'n -> 't -> 'z -> 'r -> 'a1 -> 'e ->
     'b1 -> 'c1 -> 'i -> 'f -> 'm -> 'a -> 'd1 -> 'e1 -> 'd -> 's
     -> 'h -> 'f1) -> 'v -> 'b1 -> 'z -> 'c1 -> 'u -> 'y -> 'a1
     -> 'w -> 'x -> 'e1 -> 'd1 -> 'f1 = <fun>

What is the easiest way to see that this really is the case?

A related question is this (I am sure people have thought about it): how big can a type of a typeable $\lambda$-term be? For example, the Ackermann function can be typed as follows, although the type prevents it from doing the right thing in a typed setting:

# let one = fun f x -> f x ;;
val one : ('a -> 'b) -> 'a -> 'b =
# let suc = fun n f x -> n f (f x) ;;
val suc : (('a -> 'b) -> 'b -> 'c) -> ('a -> 'b) -> 'a -> 'c =
# let ack = fun m -> m (fun f n -> n f (f one)) suc ;;
val ack :
  ((((('a -> 'b) -> 'a -> 'b) -> 'c) ->
   (((('a -> 'b) -> 'a -> 'b) -> 'c) -> 'c -> 'd) -> 'd) ->
   ((('e -> 'f) -> 'f -> 'g) -> ('e -> 'f) -> 'e -> 'g) -> 'h) -> 'h = <fun>

That’s one mean type there! Can it be “explained”? Hmm, why does ack compute the Ackermann function in the untyped $\lambda$-calculus?

The original reference for the first part is Troelstra and van Dalen, Constructivism in mathematics, volume 2, Sections 9.6.10 and 9.6.11, page 500. I transformed the proof to make the main idea a bit more transparent.

Define the finite type hierarchy $N_0, N_1, N_2, \ldots$ over the natural numbers $\mathtt{nat}$ by $$N_0 = \mathtt{nat}$$ and $$N_{k+1} = N_k \to \mathtt{nat}.$$ The elements of $N_k$ are known as the type $k$ functionals. Let us keep the exact meaning of $\mathtt{nat}$ and $\to$ a bit unspecified. We might interpret $\mathtt{nat}$ as the set of natural numbers, or as a domain, or as a datatype in a programming language. We require that $\mathbb{N} \subseteq N_0$, i.e., each number has a representation, but there might be extra elements in $\mathtt{nat}$, such as the undefined value $\perp$. The arrow $\to$ may represent the function type in a programming language, or an exponential in a cartesian-closed category. We will make things precise when we need to.

For each $k$ we define the (hereditarily) total functionals $T_k$ by $T_0 = \mathbb{N}$ and
$$T_{k+1} = \lbrace f \in N_{k+1} \mid \forall x \in T_k . f x \in T_0 \rbrace.$$
We call $T_0, T_1, T_2, \ldots$ the total hierachy derived from $N_0, N_1, N_2, \ldots$. When $N_k = T_k$ for all $k$, we say that the hierarchy $N_0, N_1, N_2, \ldots$ is total.

We also need the notion of extensionality. For each $k$ define the relation $\approx_k$ on $T_k$ by
$$n \approx_0 m \iff n = m$$
and
$$f \approx_{k+1} g \iff \forall x, y \in T_k . x \approx_k y \Rightarrow f(x) = g(y).$$
It can be shown easily that $\approx_k$ is symmetric and transitive, but it need not be reflexive. We say that $f \in N_k$ is extensional when $f \approx_k f$.

After this barrage of definitions some examples will hopefully make things a bit clearer. It is easy to come up with non-total functions, just write down something that cycles. A total function which is not extensional would be the following type 3 functional u, implemented in ocaml:

let u f =
  let k = ref 0 in
  let a n = (k := 1 ; 0) in
    ignore (f a) ;
    !k

The functional u returns 1 if its argument f evaluates its argument, and 0 otherwise. It is not extensional:

# u (fun a -> 0) ;;
- : int = 0
# u (fun a -> 0 * a 0) ;;
- : int = 1

As you can see, u uses local references. Exercise: write a total non-extensional functional using some other computational effect, such as exceptions or continuations.

To avoid writing down lots of ugly types, we preassign types to the following variables:

• $i, j, k, m, n$ stand for the elements of $N_0 = \mathtt{nat}$,
• $\alpha, \beta, \gamma$ stand for functions, which are the elements of $N_1 = \mathtt{nat} \to \mathtt{nat}$
• $F, G, H$ stand for type 2 functionals, which are the elements of $N_2 = (\mathtt{nat} \to \mathtt{nat}) \to \mathtt{nat}$,
• $\Phi, \Psi, \Xi$ stand for type 3 functionals, which are the elements of $N_3 = ((\mathtt{nat} \to \mathtt{nat}) \to \mathtt{nat}) \to \mathtt{nat}$.

We need one further piece of notation. Given a total function $\alpha$, let $\overline{\alpha}(k) = [\alpha(0), \alpha(1), \ldots, \alpha(k-1)]$ be the finite list of the first $k$ values of $\alpha$.

A total type 2 functional $F$ is continuous at a total $\alpha$ if the value $F(\alpha)$ depends only on a finite prefix $\overline{\alpha}(k)$: $$\forall \beta \in T_1 . \overline{\beta}(k) = \overline{\alpha}(k) \Rightarrow F(\alpha) = F(\beta).$$
You should convince yourself that this definition of continuity coincides with the usual one for metric spaces if we equip $\mathbf{N}$ with the usual metric $d(i,j) = |i – j|$ and $T_1$  with the comparison metric $$d(\alpha, \beta) = 2^{-\min \lbrace k \mid \alpha(k) \neq \beta(k) \rbrace}.$$ We would expect every computable total functional to be continuous because, intuitively speaking, in order for an algorithm to compute $F(\alpha)$ in finitely many steps it can only inspect finitely many values of $\alpha$. Of course, this assumes that an algorithm cannot do anything other than test $\alpha$ at various values. There are theorems in computability theory which say that, provided $F$ is extensional (see below), an algorithm cannot really do much else, even if it has the source code of $\alpha$ at its disposal. (Exercise: which theorems?)

To keep things simple we shall concentrate on continuity at the zero sequence $o = \lambda n . 0$. Thus a functional $F$ is continuous at $o$ if there exists $k$ such that $\forall \beta \in T_1. \overline{\beta}(k) = \overline{o}(k) \Rightarrow F(\beta) = F(o)$. We say that $k$ is a modulus of continuity for $F$ at $o$.

It is natural to ask whether we can compute a modulus of continuity $k$ from the total functional $F$, i.e., whether there is a modulus of continuity functional $\Phi$ of type 3 such that $$\forall F \in T_2. \forall \beta \in T_1. \overline{\beta}(\Phi(F)) = \overline{o}(\Phi(F)) \Rightarrow F(\beta) = F(o).$$ The existence of $\Phi$ implies that all type 2 functionals are continuous. So clearly we cannot expect to have one in a setting where discontinuous functional can be defined, such as the category of sets.

But how about defining $\Phi$ in a programming language? Well, you can do it in ocaml by using local references:

let phi f =
  let k = ref (-1) in
  let o i = (k := max !k i ; 0) in
    ignore (f o) ;
    !k + 1

We expect phi to compute a modulus of continuity if f is extensional, but I would really like to see a carefully written proof of this fact.

The modulus of continuity phi defined above is not extensional, since it returns different values for extensionally equal functionals:

# phi (fun alpha -> 0) ;;
- : int = 0
# phi (fun alpha -> 0 * alpha 41) ;;
- : int = 42

Can we have an extensional modulus of continuity? Suppose $\Psi$ were one. Let $n = \Psi (\lambda \alpha . 0)$ and consider the type 2 functional $H$ defined by
$$H(\beta) = \Psi (\lambda \alpha . \beta(\alpha(n))).$$
Because $\lambda \alpha . 0$ is extensionally equal to $\lambda \alpha . o(\alpha(n))$ we get by extensionality of $\Psi$ that
$$H(o) = \Psi(\lambda \alpha . o (\alpha(n))) = \Psi (\lambda \alpha . 0) = n.$$
On the other hand, if $\beta \neq o$ then the value $\beta(\alpha(n))$ cannot be determined by looking at $\alpha(0), \ldots, \alpha(n-1)$, therefore $H(\beta) = \Psi (\beta (\alpha (n))) > n$. This means that $H$ is not continuous at $o$, which contradicts existence of $\Psi$. We have shown:

Theorem: There is no extensional modulus of continuity functional.

The consequence of this is that we cannot define the modulus of continuity functional in a programming language in which all total functionals are extensional. Next time we will see that PCF is such a language.

Suppose we want to know how many rings of size $n$ there are. If $n$ happens to be the square of a prime we are lucky because a theorem says there are exactly 11 of them. Otherwise we can look up the numbers on the on The On-Line Encyclopedia of Integer Sequences (OEIS). But what if we want to know the number of semirings of a given size? Unfortunately OEIS does not seem to have an answer. And what if we want the list of 52 non-isomorphic commutative rings of size 8?

Alg comes in handy when you need the list or the count of non-isomorphic copies of an ad-hoc structure. For example, the number of non-isomorphic rings satisfying the additional equation $x \cdot y \cdot x = 0$ is:

$ ./alg.native theories/ring.th --axiom 'x * y * x = 0' --size 1-8 --count
    ...
    size | count
    -----|------
       1 | 1
       2 | 1
       3 | 1
       4 | 4
       5 | 1
       6 | 1
       7 | 1
       8 | 19

We could have guessed that there would be at least one such ring of each size, namely the one with zero multiplication. But what are the non-trivial examples of size 4?

$ ./alg.native --size 4 theories/ring.th \
  --axiom 'x * y * x = 0' --axiom 'exists x y, x * y <> 0'
# Theory ring_with_extras

    Constant 0.
    Unary ~.
    Binary + *.

    Axiom plus_commutative: x + y = y + x.
    Axiom plus_associative: (x + y) + z = x + (y + z).
    Axiom zero_neutral_left: 0 + x = x.
    Axiom zero_neutral_right: x + 0 = x.
    Axiom negative_inverse: x + ~ x = 0.
    Axiom negative_inverse: ~ x + x = 0.
    Axiom zero_inverse: ~ 0 = 0.
    Axiom inverse_involution: ~ (~ x) = x.

    Axiom mult_associative: (x * y) * z = x * (y * z).
    Axiom distrutivity_right: (x + y) * z = x * z + y * z.
    Axiom distributivity_left: x * (y + z) = x * y + x * z.

    # Extra command-line axioms
    Axiom: exists x y, x * y <> 0.
    Axiom: x * y * x = 0.

# Size 4

### ring_with_extras_4_1

    ~ |  0  a  b  c
    --+------------
      |  0  a  b  c

    + |  0  a  b  c
    --+------------
    0 |  0  a  b  c
    a |  a  0  c  b
    b |  b  c  0  a
    c |  c  b  a  0

    * |  0  a  b  c
    --+------------
    0 |  0  0  0  0
    a |  0  0  0  0
    b |  0  0  a  a
    c |  0  0  a  a

- - - - - - - - - - - - - - - - - - - - - - - - - - - -

### ring_with_extras_4_2

    ~ |  0  a  b  c
    --+------------
      |  0  a  c  b

    + |  0  a  b  c
    --+------------
    0 |  0  a  b  c
    a |  a  0  c  b
    b |  b  c  a  0
    c |  c  b  0  a

    * |  0  a  b  c
    --+------------
    0 |  0  0  0  0
    a |  0  0  0  0
    b |  0  0  a  a
    c |  0  0  a  a

- - - - - - - - - - - - - - - - - - - - - - - - - - - -

# Statistics

    size | count
    -----|------
       4 | 2

Interesting. Or maybe not. Anyhow, it is easy to play with alg this way.

The supported output formats are text (actually Markdown), HTML, LaTeX and JSON. If you generate three or more sizes, alg outputs a URL for checking the numbers in OEIS.

On my dekstop with Intel Core 2 Duo E8500 @ 3.16 Ghz groups of size up to 9 are enumerated in 3.4 seconds, up to size 10 in 50 seconds, and up to size 11 in 9.5 minutes. Clearly, we are hitting a combinatorial explosion here. As there are $n^{n^2}$ multiplication tables of size $n \times n$, alg has to do something better than trying out $10^{100}$ possible multiplication tables to enumerate groups of size 10. Indeed, alg fights the combinatorial explosion by filling in the tables in a smart way. It uses two algorithms:

1. The default algorithm fills in the tables in all possible ways, where with each new entry it also checks the axioms to see what else could be filled in. The algorithm works well for equational theories, such as groups, rings, and lattices.
2. The alternative algorithm, which is enabled with the --sat command-line option, transforms the axioms into a set of constraints that it then satisfies in all possible ways. The algorithm works well for relational theories, such as posets and graphs.

Each new solution is compared for isomorphism with previously found ones. Since isomorphism testing is hard, alg uses simple invariants to quickly test for non-isomorphism. The memory footprint is very low. As I type this alg is using measly 880 kB of memory for enumeration of groups of size 11.

Alg is possibly the right tool for you if you want to:

• list or count non-isomorphic structures satisfying certain axioms, and you cannot find the data on the web
• quickly check a hypothesis such as “In which rings does $x^5 + x^3 + 1 = 0$ have a solution?”
• quickly check if two axiomatizations describe the same structures (count models of both and compare)

Can you trust alg? Well, we do not have  a formal proof of correctness. If the results seem fishy there are several things you can do:

1. Check the counts at OEIS.
2. Compare the answers given by the default and the alternative algorithm.
3. Use the --paranoid option for double-checking of results.

Here are the promised download instructions. You may:

1. Check out the source code from Mercurial repository with

   hg clone http://hg.andrej.com/alg/
   

   and compile the program yourself (you need Ocaml 3.11 and menhir, as described in the manual).

2. Download the archived source code from Mercurial repository: alg.zip, alg.tar.gz or alg.tar.bz2.
3. Download precompiled executables with the manual and example theories included. We have prepared executables for Linux, MacOS X and Windows 7.

The alg manual explains how to compile and use alg and how to write your own theories.

Alg is open source software released under the simplified BSD license. Feedback and bug reports are appreciated!