Python Tutorial: Charting word length with NLTK

Want to learn more? Take the full course at https://learn.datacamp.com/courses/introduction-to-natural-language-processing-in-python at your own pace. More than a video, you'll learn hands-on coding & quickly apply skills to your daily work. --- Hi everyone! In this video, we are going to learn about using charts with our NLP tools. Matplotlib is a charting library used by many different open-source Python projects to create data visualizations, charts and graphs. It has fairly straightforward functionality with lots of options for graphs like histograms, bar charts, line charts and scatter plots. It even has advanced functionality like generating 3D graphs and animations. Matplotlib is usually imported by simply aliasing the pyplot module as plt. If we want to plot a basic histogram, which is a type of plot used to show distribution of data, we can pass in a small array to the `hist` function. The array has 5 appearing twice and 7 appearing three times, so it's a good candidate to show distribution. Finally, we call the plt.show function and matplotlib will show us the generated chart in our system's standard graphics viewing tool. This is the chart that we generated using the previous code. We notice that indeed it has determined proper bins for each entry and we can see that the 7 and 5 bins reflect the distribution we expected to see. It's not the prettiest chart by default, but making it look nicer is fairly easy with more arguments and several available helper libraries. We can then use skills we have learned throughout this first chapter to tokenize text and chart word length for a simple sentence. First, we perform the necessary imports to use NLTK for word tokenization and matplotlib charting. Then, we tokenize the words and punctuation in a short sentence. Finally, we can use Python list comprehension with our tokenized words array to transform it to a list of lengths. As a brief refresher on list comprehensions, it is a succint way to write a