A challenge for evolutionary biology, therefore, is to provide a detailed mathematical account of how natural selection can enable the emergence of human language from animal communication. Animal communication is based on three basic designs: a finite repertoire of calls (territorial calls or warning of predators); a continuous analogue signal (for example, the dance of bees); and a series of random variations on a theme (such as the song of birds). All natural animal communication appears to be non-syntactic; some caution, however, seems appropriate as the final verdict on complex vocalization patterns of certain primate species or dolphins has not been reached. In contrast, human language is clearly syntactic: messages consist of components that have their own meaning. We compare non-syntactic and syntactic communication and evaluate their relative performance in an evolutionary setting.

First, we formulate a mathematical model for the population dynamics of language evolution. Suppose a language contains n words. Each individual is born not knowing any of the words, but can acquire words by learning from other individuals. Individuals are characterized by the subset of words that they know. The general equations for the resulting evolutionary dynamics are complicated (see Methods), but an analytical approach is possible if we describe the process in terms of independent, elementary steps on the basis of two assumptions: first, in any one interaction between two individuals only a single new word can be learned; second, words are memorized independently of each other. With these assumptions, we obtain for the population dynamics of x_i, which is the relative abundance of individuals who know word W_i

Assuming that word frequency distributions follow Zipf's law22, 23, _i = C/i, where C is a constant, we find that the maximum number of words is roughly given by the equation n_maxln(n _max) = bq (Fig. 1).

Figure 1 How many word-learning events per individual are required for a population to maintain a certain number of words in its combined lexicon assuming that word frequencies follow Zipf's law? High resolution image and legend (7k)

We now use this mathematical framework to analyse how natural selection can guide the transition from non-syntactic to syntactic communication. Imagine a group of individuals who communicate about events in the world around them. Events are combinations of objects, places, times and actions. (We use 'object' and 'action' in a general way to represent everything that can be referred to by nouns and verbs of current human languages.) For notational simplicity, suppose that each event consists of one object and one action. Thus, event E_ij consists of object i and action j. Non-syntactic communication uses words for events, whereas syntactic communication uses words for objects and actions (Fig. 2). Events occur at different rates, which are specified by the entries of an 'event rate matrix', .

Figure 2 To understand the essence of the evolution of syntax, we imagine a world where each event consists of one object and one action. High resolution image and legend (8k)

For natural selection to operate on language design, language must confer fitness. A plausible assumption is that correct communication about events provides a fitness advantage to the interacting individuals. In terms of our model, the fitness contribution of a language can be formulated as the probability that two individuals know the correct word for a given event summed over all events and weighted with the rate of occurrence of these events.

The population dynamics of non-syntactic communication are again given by equation (1) with word W_ij referring to event E_ij. As before, the maximum number of words that can be maintained in the population is limited by bq. We calculate the fitness of individuals using non-syntactic communication (see Methods).

We now turn to syntactic communication. Noun N_i refers to object i and verb V_j refers to action j, hence the event E_ij is described by the sentence N_iV_j. For the basic reproductive ratios we obtain R(N_i) = (b/2)q_s(N _i) and R(V _j) = (b/2)q_s( V_j). Here (N _i) and (V_j) denote the frequency of occurrence of noun N_i and verb V _j, respectively. The factor 1/2 appears because either the noun or the verb is learned in any one of the b learning events. The probability of memorizing a noun or a verb is given by q_s. We expect q_s to be smaller than q, which simply means that it is a more difficult task to learn a syntactic signal than a non-syntactic signal. For both signals, the (arbitrary) meaning has to be memorized; for a syntactic signal one must also memorize its relation to other signals (whether it is a noun or a verb, for example).

For noun N_i to be maintained in the lexicon of the population, its basic reproductive ratio must exceed one, implying that (N_i) > 2/(bq _s). Similarly, for verb V_j we find (V_j) > 2/(bq _s). This means that the total number of nouns plus verbs is limited by bq_s, which is always less than b. The maximum number of grammatical sentences, however, which consist of one noun and one verb, is given by (bq_s)²/4. Hence syntax makes it possible to maintain more sentences than the total number of sentences, b, encountered by a learning individual. All words have to be learned, therefore, but syntactic signals enable the formulation of 'new' sentences that have not been learned beforehand.

For calculating the fitness of syntactic communication, note that two randomly chosen individuals can communicate about event E_ij if they both know noun N_i and verb V _j. If we denote the relative abundance of individuals who know N_i and V_j by x(N _iV_j), we obtain

The abundance of individuals who know noun N_i and verb V_j increases if someone who knows N _i meets someone who knows V_j but not N_i. Similarly, the abundance increases if someone who knows verb V_j meets someone who knows N _i but not V_j. We calculate the equilibrium abundances and thence the fitness of individuals using syntactic communication (see Methods). Figure 3 shows the fitness of syntactic and non-syntactic communication as a function of b for different examples of the event rate matrix, .

Figure 3 The fitness of non-syntactic and syntactic communication, F n and Fs, as function of the total number of word learning events per individual, b, for three different choices of the event rate matrix, . High resolution image and legend (24k)

When does syntactic communication lead to a higher fitness than non-syntactic communication? Suppose there are n objects and m actions. Suppose a fraction, p, of these mn events occur (all at the same frequency), and the other events do not occur. In this case, R( W_ij) = bq/( pmn) for those events that occur. Making the (somewhat rough) assumption that all nouns and all verbs, respectively, occur on average at the same frequency, we obtain R(N _i) = bq_s/(2n) and R(V_j) = bq_s/(2m). If all involved basic reproductive ratios are well above one, the fitness of syntactic communication exceeds the fitness of non-syntactic communication provided (m ²n + mn²)/ (m² + mn + n ²) > (2q)/(pq_s) (see Methods). If this inequality holds then syntactic communication will be favoured by natural selection; otherwise non-syntactic communication will win. Observe that m 3 and n 3 are necessary conditions for the evolution of syntax. For m = n, the relevant condition is

Hence, the size of the system has to exceed a critical value for syntactic communication to evolve. For example, if it is twice as hard to memorize a syntactic signal than a non-syntactic signal, q/q _s = 1/2, and if a fraction p = 1/3 of all noun verb combinations describe meaningful events, then at least an 18 18 system is required for syntactic communication to have any chance of evolving. Figure 4 shows the excellent agreement between our approximative analytical results and exact numerical computations.

Figure 4 Numerical validation of the approximate threshold condition given by equation (3). High resolution image and legend (10k)

The parameter p quantifies to what extent the perceived world has a compositional structure. A small p means that events often consist of unique pairings of objects and actions. The smaller the value of p, the harder it is for syntactic communication to evolve (the critical n in equation (3) is large).

Our results suggest that the crucial step that guided the transition from non-syntactic to syntactic communication was an increase in the number of relevant events that could be referred to. 'Relevant event' means there is a fitness contribution for communication about this event. As the number of such 'relevant communication topics' increased, natural selection could begin to favour syntactic communication and thereby lead to a language design where messages could be formulated that were not learned beforehand. Syntactic messages can encode new ideas or refer to extremely rare but important events. Our theory, however, does not suggest that syntactic communication is always at an advantage. Many animal species probably have a syntactic understanding of the world, but natural selection did not produce a syntactic communication system for these species because the number of relevant signals was below the threshold illustrated by equation (3) . Presumably the increase in the number of relevant communication topics was caused by changes in the social structure²⁴ and interaction of those human ancestors who evolved syntactic communication.

where I = 0,..., . We have ₀ = 1 and _I = 0 otherwise; thus all individuals are born not knowing any of the words. Individuals die at a constant rate, which we set to 1, thereby defining a time scale. The quantities Q_IJK denote the probabilities that individual I learning from J will become K. Equation (4) is a general framework for the population dynamics of the lexical aspects of language. Assuming that in any one interaction between two individuals only a single new word can be acquired and that words are memorized independently of each other, we obtain the specific system described by equation (1).

Let us now assume that the world is made up of events E _ij consisting of objects i and actions j. The 'event rate matrix', has the entries _ij which specify the relative rate of occurrence of event E_ij. Denote by _ij the frequency of occurrence of event E_ij. We have _ij = _ij/_{ij ij}.

Non-syntactic communication uses words, W_ij, for events E_ij. The basic reproductive ratio of W _ij is given by R(W _ij) = bq_ij. If R(W_ij) > 1, the word W_ij will persist in the population, and at equilibrium the relative abundance of individuals who know this word is given by x*(W_ij) = 1 - 1/R(W_ij) .

The fitness contribution of a language can be formulated as the probability that two individuals know the correct word for a given event summed over all events and weighted with the rate of occurrence of these events. Hence, at equilibrium, the fitness of individuals using non-syntactic communication is given by

For syntactic communication, we assume the event E_ij is described by the sentence N_iV_j . The population dynamics of individuals knowing both N _i and V_j are described by equation (2). The basic reproductive ratios are given by R(N_i) = (b/2) q_s(N_i) and R(V_j) = (b/2)q_s(V _i). The frequencies of occurrence are ( N_i) = _{j ij} and (V_j) = _iij. If the basic reproductive ratios, R(N_i) and R(V_j), are greater than one, the equilibrium frequency of individuals who know both N_i and V_j is given by