Published online 24 January 2011 | Nature | doi:10.1038/news.2011.40
Corrected online: 25 January 2011
News
Languages are adapted to deliver information efficiently and smoothly.
However, physicist Damián Zanette of the Centro Atómico Bariloche in San Carlos
de Bariloche, Argentina, who has studied Zipf-type relationships in linguistics,
is not persuaded that the MIT group's method accurately captures the real
information content of a word in context. This, he says, is typically determined
by several hundred surrounding words, not just a few3.
Piantadosi and colleagues suggest that the relationship of word length to
information content might not only make it more efficient to convey information
linguistically but also make language cognition a smoother ride for the reader
or listener. If shorter and briefer words carry less information, then the
density of information throughout a phrase or sentence will be smoothed out, so
that it is delivered at a roughly steady rate rather than in lumps. In this way,
the results suggest how the structure of language might aid communication.
Surprising though it may seem, some linguists, such as Noam Chomsky, have
suggested that communication might not be the primary purpose of language - that
it might, for example, be primarily about establishing social relations. Yet
according to cognitive scientist Florian Jaeger at the University of Rochester
in New York, these new results "suggest that communication is a sufficiently
important aspect of language to shape it over time".
An editing error inadvertantly identified Piantadosi's research group as being
based at Harvard, rather than MIT. The text has been corrected to reflect this.
1.
Zipf, G. The Psychobiology of Language (Routledge, 1936).
2.
Piantadosi, S. T., Tily, H. & Gibson, E. Proc.
Natl Acad. Sci. USA doi:10.1073/pnas.1012551108 (2011).
3.
Montemurro, M. A. & Zanette, D. H. Adv. Complex
Syst. 13, 135-153 (2010). | Article | ISI
If you find something abusive or inappropriate or which does not otherwise
comply with our
Terms or
Community Guidelines, please select the
relevant 'Report this comment' link.
Comments on this thread are vetted after posting.
·
#17624
One would be intrested to see contrasting statistics for Dutch, Welsh,
Estonian, and Inuinnaqtun.
It is surprising that Chomsky thinks "establishing social relations" does
not require--is in opposition to--"communication." In fact, communication of
social messages necessary for establishing and maintaining social relations
exists both without language and alongside--augmented by--language.
Would be interesting to read Ref.2,
but DOI doesn't exist, the paper is not reached by search in pnas.org, there
aren't papers in the web with that authors (with exception of Steven
Piantadosi's home page), and there is only one preprint in arXiv.org with
au:Piantadosi but in other topic... very strange...
Does Philip Ball check its references?
Could anyone help me?
Pedro, PNAS has not published the article yet. It should be available later
today I believe, and you can get to it via this link when it is:
http://www.pnas.org/cgi/doi/10.1073/pnas.1012551108
"Longer words tend to carry more information, according to research by a
team of cognitive scientists."
So, "supercalifragilisticexpialidocious" should be the word with the
greatest content of information?
Test
The mind that acts out as if it is separate from itself in order to make of
its self an object will develop its own languages that assert assumed
meanings about itself but from a false pretext. "I think therefore I am". A
whole world can be predicated on such an 'I' even though there is absolutely
no direct experience of such an entity. Seriously, where is it?
The ‘language’ of true intimacy is life itself; un-interpreted or
modified and directly felt and shared. When we are identified with a seeming
capacity to be independent powers unto our selves we necessarily shun such
Intimacy or at least modify it to such manageable moments as 'normal
service' can be quickly resumed from. The mind can be used to limit
communication.
To regard information or data as communication is to forget that data is a
carrier of a contextual instance – because it is given and received in mind
– and the context of mind is either a willingness for intimacy or truly
shared purpose – or the obfuscation of such in order to play out a sense of
self exceptionalism.
My own sense of language and meaning has been an experiential education. For
when I sought for meaning outside the true context of Intimate Life – I
acquired 'concepts, ideas and meanings' and associated and related them
together in ways that constituted 'my' current understanding or handle on
the world. But when I abandoned such a futility, the meaning of life became
less obfuscated to a unified mind – and such currents of insight or intuited
knowings found clothes from precisely the structures of ability that I had
grown as an ability to articulate. This is much more a listening and
trusting relationship than a formulation of ideas – and of course its
underlying nature or intent – is to share of the same simple Intimacy of
being that is free of the man-made attempt to make or master Reality. Even a
'reality' of our own definitions.
The attempt to reverse engineer the most crude and reducible common elements
of our world is like wandering in the mind with the light off, using a
pencil torch – and then trying to reconstitute models of 'how it works' or
'what it means' - from a mechanism of delivery that is itself a
communication device and not in itself otherwise meaningful.
Hmmm... quite a few longish words here. Probably no meaning at all to a mind
that is still trying to get a handle on life so as to turn it to serve a
privately personal sense. But I am communicating as I feel moved to do –
from a sense of a communioned life. That I feel addresses the fundamental
presumptions from which the human mind tends to operate like a programmed
robot rather than an expression of intimacy.
A very interesting study, but more than anything it makes me call into
question information theory, or at least this bit that, a word's
"information content is then proportional
to the negative logarithm of [its] probability."
The probability of a given word must be influenced by the number of its
synonyms. The information content added by a given word as opposed to a
another option would also seem to be affected by the number of its synonyms
(it will connote some additional information that another word would not),
but I don't see how the information content that it actually adds to a
sentence is at all affected by the number of synonyms.
So, you have a situation where probability of a word is affected by N
synonyms, but information delivered to reader or listener by a word is not.
Maybe someone could explain how my thinking is incorrect?
So does it mean that words in German or Russian carry more information than
ones in the English language? Also it's not clear what the authors would say
about proto-languages such as Proto-Germanic or PIE in which words were,
ahem, somewhat shorter than in Modern English?
The length of words is related to how much information they convey.iStockphoto.com/Pgiam
Longer words tend to carry more information, according to research by a team of
cognitive scientists.
It's a suggestion that might sound intuitively obvious, until you start to think
about it. Why, then, the difference in length between 'now' and 'immediately'?
For many years, linguists have tended to believe that the length of a word was
associated with how often it was used, and that short words are used more
frequently than long ones. This association was first proposed in the 1930s by
the Harvard linguist George Kingsley Zipf1.
Zipf believed that the relationship between word length and frequency of use
stemmed from an impulse to minimize the time and effort needed for speaking and
writing, as it means we use more short words than long ones. But Steven
Piantadosi and colleagues at the Massachusetts Institute of Technology in
Cambridge say that, to convey a given amount of information, it is more
efficient to shorten the least informative — and therefore the most predictable
— words, rather than the most frequent ones.
Zipf's original association is roughly correct, as implied by how much more
often 'a', 'the' and 'is' are used in English than, say, 'extraordinarily'. And
this relationship of length to use seems to hold up in many languages. Because
written and spoken length are generally similar, it applies to both speech and
text.
But after analysing word use in 11 different European languages, Piantadosi and
colleagues found that word length was more closely correlated with their
information content than with how often they are used. They describe their
results in the Proceedings of the National Academy of Sciences2.
"This is a landmark study", says linguist Roger Levy of the University of
California at San Diego. "Our understanding of the relationship between word
frequency and length has remained relatively static since Zipf's discoveries,"
he says, and he feels that this new study may now supply "the largest leap
forward in 75 years" in understanding how the evolution of words is governed by
the efficiency with which they can be used to communicate.
Measuring the information content of a word isn't easy, especially because it
can vary depending on the context. But Piantadosi and colleagues make the
assumption that the more predictable a word is, the less informative it is. So
the word 'nine' in 'A stitch in time saves nine' contains less information than
it does in the phrase 'The word that you will hear is nine', because in the
first case it is highly predictable - when it comes, it doesn't significantly
add to the information already in the phrase.
The MIT group devised a method for estimating the information content of words
in digitized texts by looking at how it is correlated with — and thus
predictable from — the preceding words. For just a single preceding word,
Piantadosi explains, "we count up how often all pairs of words occur together in
sequence, such as 'the man', 'the boy', 'a man', 'a tree' and so on. Then we use
this count to estimate the probability of a word conditioned on the previous
word — or more generally, the probability of any word conditioned on any
preceding sequence of a given number of words." According to information theory,
the information content is then proportional to the negative logarithm of this
probability.
However, physicist Damián Zanette of the Centro Atómico Bariloche in San Carlos
de Bariloche, Argentina, who has studied Zipf-type relationships in linguistics,
is not persuaded that the MIT group's method accurately captures the real
information content of a word in context. This, he says, is typically determined
by several hundred surrounding words, not just a few3.
Piantadosi and colleagues suggest that the relationship of word length to
information content might not only make it more efficient to convey information
linguistically but also make language cognition a smoother ride for the reader
or listener. If shorter and briefer words carry less information, then the
density of information throughout a phrase or sentence will be smoothed out, so
that it is delivered at a roughly steady rate rather than in lumps. In this way,
the results suggest how the structure of language might aid communication.
Surprising though it may seem, some linguists, such as Noam Chomsky, have
suggested that communication might not be the primary purpose of language - that
it might, for example, be primarily about establishing social relations. Yet
according to cognitive scientist Florian Jaeger at the University of Rochester
in New York, these new results "suggest that communication is a sufficiently
important aspect of language to shape it over time".
An editing error inadvertently identified Piantadosi's research group as being
based at Harvard, rather than MIT. The text has been corrected to reflect this.
1.
Zipf, G. The Psychobiology of Language (Routledge, 1936).
2.
Piantadosi, S. T., Tily, H. & Gibson, E. Proc.
Natl Acad. Sci. USA doi:10.1073/pnas.1012551108 (2011).
3.
Montemurro, M. A. & Zanette, D. H. Adv. Complex
Syst. 13, 135-153 (2010). | Article | ISI
If you find something abusive or inappropriate or which does not otherwise
comply with our
Terms or
Community Guidelines, please select the
relevant 'Report this comment' link.
Comments on this thread are vetted after posting.
·
#17624
One would be intrested to see contrasting statistics for Dutch, Welsh,
Estonian, and Inuinnaqtun.
It is surprising that Chomsky thinks "establishing social relations" does
not require--is in opposition to--"communication." In fact, communication of
social messages necessary for establishing and maintaining social relations
exists both without language and alongside--augmented by--language.
Would be interesting to read Ref.2,
but DOI doesn't exist, the paper is not reached by search in pnas.org, there
aren't papers in the web with that authors (with exception of Steven
Piantadosi's home page), and there is only one preprint in arXiv.org with
au:Piantadosi but in other topic... very strange...
Does Philip Ball check its references?
Could anyone help me?
Pedro, PNAS has not published the article yet. It should be available later
today I believe, and you can get to it via this link when it is:
http://www.pnas.org/cgi/doi/10.1073/pnas.1012551108
"Longer words tend to carry more information, according to research by a
team of cognitive scientists."
So, "supercalifragilisticexpialidocious" should be the word with the
greatest content of information?
Test
The mind that acts out as if it is separate from itself in order to make of
its self an object will develop its own languages that assert assumed
meanings about itself but from a false pretext. "I think therefore I am". A
whole world can be predicated on such an 'I' even though there is absolutely
no direct experience of such an entity. Seriously, where is it?
The ‘language’ of true intimacy is life itself; un-interpreted or
modified and directly felt and shared. When we are identified with a seeming
capacity to be independent powers unto our selves we necessarily shun such
Intimacy or at least modify it to such manageable moments as 'normal
service' can be quickly resumed from. The mind can be used to limit
communication.
To regard information or data as communication is to forget that data is a
carrier of a contextual instance – because it is given and received in mind
– and the context of mind is either a willingness for intimacy or truly
shared purpose – or the obfuscation of such in order to play out a sense of
self exceptionalism.
My own sense of language and meaning has been an experiential education. For
when I sought for meaning outside the true context of Intimate Life – I
acquired 'concepts, ideas and meanings' and associated and related them
together in ways that constituted 'my' current understanding or handle on
the world. But when I abandoned such a futility, the meaning of life became
less obfuscated to a unified mind – and such currents of insight or intuited
knowings found clothes from precisely the structures of ability that I had
grown as an ability to articulate. This is much more a listening and
trusting relationship than a formulation of ideas – and of course its
underlying nature or intent – is to share of the same simple Intimacy of
being that is free of the man-made attempt to make or master Reality. Even a
'reality' of our own definitions.
The attempt to reverse engineer the most crude and reducible common elements
of our world is like wandering in the mind with the light off, using a
pencil torch – and then trying to reconstitute models of 'how it works' or
'what it means' - from a mechanism of delivery that is itself a
communication device and not in itself otherwise meaningful.
Hmmm... quite a few longish words here. Probably no meaning at all to a mind
that is still trying to get a handle on life so as to turn it to serve a
privately personal sense. But I am communicating as I feel moved to do –
from a sense of a communioned life. That I feel addresses the fundamental
presumptions from which the human mind tends to operate like a programmed
robot rather than an expression of intimacy.
A very interesting study, but more than anything it makes me call into
question information theory, or at least this bit that, a word's
"information content is then proportional
to the negative logarithm of [its] probability."
The probability of a given word must be influenced by the number of its
synonyms. The information content added by a given word as opposed to a
another option would also seem to be affected by the number of its synonyms
(it will connote some additional information that another word would not),
but I don't see how the information content that it actually adds to a
sentence is at all affected by the number of synonyms.
So, you have a situation where probability of a word is affected by N
synonyms, but information delivered to reader or listener by a word is not.
Maybe someone could explain how my thinking is incorrect?
So does it mean that words in German or Russian carry more information than
ones in the English language? Also it's not clear what the authors would say
about proto-languages such as Proto-Germanic or PIE in which words were,
ahem, somewhat shorter than in Modern English?