R Tutorial : Counting words

Want to learn more? Take the full course at https://learn.datacamp.com/courses/topic-modeling-in-r at your own pace. More than a video, you'll learn hands-on coding & quickly apply skills to your daily work. --- In order to fit a topic model, we must prepare a document-term matrix that will contain counts of word occurrences in documents. In this lesson, we will cover how to do it using packages tidytext and dplyr. In-text processing, the process of splitting a text is referred to as tokenization. In our case, we will be splitting text into words, but in general, tokens can be a sequence of characters or a sequence of words. Package tidytext has a function unnest_tokens() that performs tokenization. The function takes a column from a table, splits it into words and, by default, it will drop the column with text. It will also convert the output to lower case. We have a data frame named "book". It has two columns: chapter and text. We call unnest_tokens(), instructing that the column with tokens should be named "word" and that column "text" should be dropped. We get back a table in which each word is in its own row. We will use function count() from package dplyr to obtain frequencies of words within chapters. This function, essentially, groups the rows by chapter and word, and returns the number of rows in each group. This corresponds to the number of times a specific word occurs in a specific chapter. The result is a table with one row per each combination of chapters and words. For example, the word "is" occurs twice in chapter 1. Once we have the counts, we often will want to examine the top words, for example, the top 10. This can be done by grouping the rows by chapter, sorting the rows within each group in order of descending counts, and then realizing that the rank of a word is equal to its row number. Most frequent word will be in row 1, second most frequent - in row 2, and so on. dplyr has a function row_number() that returns the row number. A