QRLG in the MIT News

The complexities of cognitive comparisons

In experiments, linguists examine how we make everyday judgments about groups of objects.

Which mountain range is bigger, the Rockies or the Alps?

The answer depends on how you compare them. The Alps reach a greater height, and the 10 highest peaks in the Alps are taller in sum than the 10 highest peaks in the Rockies. Then again, the Rockies have three times as many peaks over 4,000 meters, or 13,000 feet, and the main spine of the Rockies runs about five times as far as that of the Alps — meaning the Rockies surely contain a greater volume of mountains.

In short, comparing groups of objects can be tricky, especially when we are evaluating their characteristics, and not just the number of members in each group. So how does the mind process these kinds of comparisons and turn them into linguistic statements?

In the past, linguists exploring this issue have asserted that “plural comparison follows from comparing every individual in one set with every individual in another set,” says Peter Graff, a doctoral student in MIT’s Department of Linguistics and Philosophy. Suppose we ranked every high peak in the Alps and Rockies, put the lists side by side, and found that at each point in the rankings, the Alpine peak was taller. Then we would have a straightforward method for stating that the Alps are bigger, or at least taller, based on a series of individual comparisons.

But in research recently published in the journal Cognition, Graff and two co-authors dig into the nature of comparison, and find something different. When we compare groups of this sort, our minds don’t construct a series of individual comparisons. Instead, “plural entities are represented as entities with their own properties,” as their paper states. Our minds choose some line of comparison — average size, or sum of the sizes, for instance — and then compare groups, as groups, on that basis.

“You can ascribe a property to a plurality, namely an average statistic, which may not necessarily be true of any of its members,” says Gregory Scontras, a doctoral student in linguistics at Harvard University and another co-author of the paper.

Probably right

To arrive at this conclusion, the researchers — Graff, Scontras, and Noah Goodman, an assistant professor of cognitive psychology at Stanford University and a former research scientist in MIT’s Department of Brain and Cognitive Sciences — conducted a series of experiments in which 43 respondents compared groups of five blue dots and five red dots, and were asked if the red dots were bigger than the blue dots. The subjects saw 32 iterations of the comparison, in which the sizes of all the dots varied; the largest dot size was seven times bigger than the smallest. But in very few iterations was there a rule of thumb that could be used, such as all blue dots being bigger than the biggest red dot.

The result? Respondents scored highest, over the 32 rounds of comparisons, when intuitively averaging the mean size of the dots, rather than looking at any attribute of any particular dot.

In a follow-up experiment, the researchers had 40 respondents look at 32 iterations of the dots again to see if people were judging size according to the mean size or the sum of the sizes. When the mean size of the dots conflicted with the sum size, respondents would render their verdict largely based on the mean size.

For the researchers, two conclusions follow: First, Scontras says, the point of the research “wasn’t that in all cases you’re going to find an average statistic from each plurality and compare that.” Rather, he points out, groups “can have aggregate statistics attributed to them that might not be true of any individual member,” something that “has not yet been shown in the linguistics literature.”

Second, as Graff notes, “The underlying process that underlies those judgments is probabilistic in nature.” That is, respondents do not have a deterministic method for deriving the results, and they do not always judge the same evidence the same way. That creates a little statistical noise around the results; for any given comparison, answers will take the form of a probability distribution. However, the respondents erred most frequently when the mean sizes of the red and blue dots were, in fact, very close to each other. Thus, Graff adds, “This work shows that the noise in these judgments is not random. So it becomes harder to maintain the hypothesis that the underlying process is categorical,” or rule-based.

That, in turn, may have implications for the way linguists analyze how we create and express truth statements. “The goal of formal semantics is to come up with the algorithm we use to determine if something is true or false,” Scontras says. “We want something deterministic. And what we’re finding here is that … the processes that determine the meanings we are using in language are themselves probabilistic.”

Other scholars have found the research promising. “This work is important because it shows that very abstract conceptual principles guide how we organize and store basic perceptual information,” says David Barner, a professor of psychology at the University of California at San Diego, who has read the paper. He adds: “Logical models of reasoning and language since Aristotle have treated individuals as fundamental, and so you might think that all sentence meanings could be described building up from statements about individuals. … This work suggests that language may be more clever than this, and may allow us to create complex things, or ‘plural objects.’”

Barner, who specializes in language acquisition among children, notes that the paper also relates to the issue of whether “infants could acquire abstract logical representations like those we find in language, and whether some of these capacities are present before infants begin to use and comprehend language. … Getting an idea of how these [capacities] work in adults, using a simple method like this, lays out a very clear path forward for understanding how language emerges in humans.”

The researchers are now conducting additional experiments about the ways people compare pluralities. By creating a variety of scenarios and changing the particular measure being compared — such as weight, height and size — they are looking at how the linguistic context can influence the way people compare these complex groups.


Scontras, Graff and Goodman to appear in Cognition.

My paper with Gregory Scontras and Noah Goodman on how people compare pluralities of objects along a given dimension was just accepted for publication in Cognition. The pre-print version can be downloaded here. Here's one of our stimuli. Do you think the red dots are bigger than the blue dots? Feedback on the paper is of course very much appreciated. Abstract: What does it mean to compare sets of objects along a scale, for example by saying “the men are taller than the women”? We explore comparison of pluralities in two experiments, eliciting comparison judgments while varying the properties of the members of each set. We find that a plurality is judged as “bigger”  when the  mean size of its members is larger than  the mean size  of the competing plurality. These results  are incompatible with previous accounts, in which plural comparison is inferred from many instances of singular comparison between the members of the sets (Matushansky and Ruys, 2006).  Our results  suggest the need for a type of predication that ascribes properties to plural entities, not just individuals, based on aggregate statistics of their members. More generally, these results support the idea that sets and their properties are actively represented as single units.


Dictionaries digitized as of today.

We're currently in the process of compiling a huge corpus of dictionaries, ranging in size from 894 to 824 to 95,674 words from many different languages. As of today, we have the following 61 languages in our data-base: Acholi (Nilo-Saharan), Alekano (Trans-New Guinea), Amharic (Afro-Asiatic), Armenian (Indo-European), Arrernte (Australian), Ata (Austronesian), Bargam (Trans-New Guinea), Benabena (Trans-New Guinea), Bunama (Austronesian), Chickasaw (Muskogean), Chinese Mandarin (Sino-Tibetan), Dadibi (Teberan-Pawaian), Daga (Dagan), Delaware (Algic), Dobu (Austronesian), Dutch (Indo-European), French (Indo-European), Georgian (Kartvelian), German (Indo-European), General American English (Indo-European), Greek (Indo-European), Guarani (Tupian), Haitian Creole (HaitianCreole), Hausa (Afro-Asiatic), Hebrew (Afro-Asiatic), Hindi (Indo-European), Iamalele (Austronesian), Iduna (Austronesian), Javanese (Austronesian), Kewa (Trans-New Guinea), Khmer (Austro-Asiatic), Mountain Koiali (Trans-New Guinea), Lake Miwok (Penutian), Lithuanian (Indo-European), Maisin (Austronesian), Mauwake (Trans-New Guinea), Mende (Sepik), Mengen (Austronesian), Mianmin (Trans-New Guinea), Moroccan Arabic (Afro-Asiatic), Muna (Austronesian), Muyuw (Austronesian), Polish (Indo-European), Ayacucho Quechua (Quechuan), Romanian (Indo-European), Rotokas (West Bougainville), Siroi (Trans-New Guinea), Southern British English (Indo-European), Sudest (Austronesian), Suena (Trans-New Guinea), Tatar (Altaic), ThompsonSalish (Salishan), Turkish (Altaic), Waffa (Trans-New Guinea), Wantoat (Trans-New Guinea), Waris (Border), Waskia (Trans-New Guinea), Woleaian (Austronesian), Yana (Hokan), Yup'ik (Eskimo-Aleut), Zulu (Niger-Congo).