Discuss the Article : Quizzacious: The Mystery Of Word Frequency
<a href='/article/555/quizzacious-the-mystery-of-word-frequency' target='_blank'>Quizzacious: The Mystery Of Word Frequency</a>
Quizzacious. You have almost certainly heard of this word in the last 10 days. Michael Stevens, creator of the popular Vsauce YouTube channel posted a video on the topic exploring Zipf’s Law, the Pareto Principle, and the frequency of specific word occurrence in language and literature. Why this pattern occurs consistently remains one of the most fascinating unsolved mysteries of human language.
@Stephen - the sample size is indeed a concern, but there shouldn't be this much deviation with the amount of text we had (we analyzed all of the english-learning articles for the last year). The "the" issue is curious though, because it is relatively rare for a language to have a non-gender version of a word that functions as "the" does. Many languages don't have the word at all.
Thank you. Very interesting. I`ve just finished my deck for Anki based on words graded according to zipf law. I have 14000 the most frequent words in English.
Very interesting article, Ivan. Thank you!
Maybe you just needed a bigger sample of English to square your own data with that expected by Zipf’s Law.
