Python Tutorial: Statistical Models

Want to learn more? Take the full course at https://learn.datacamp.com/courses/advanced-nlp-with-spacy at your own pace. More than a video, you'll learn hands-on coding & quickly apply skills to your daily work. --- Let's add some more power to the NLP object! In this video, you'll learn about spaCy's statistical models. Some of the most interesting things you can analyze are context-specific: for example, whether a word is a verb or whether a span of text is a person name. Statistical models enable spaCy to make predictions in context. This usually includes part-of-speech tags, syntactic dependencies and named entities. Models are trained on large datasets of labeled example texts. They can be updated with more examples to fine-tune their predictions – for example, to perform better on your specific data. spaCy provides a number of pre-trained model packages you can download. For example, the "en_core_web_sm" package is a small English model that supports all core capabilities and is trained on web text. The spacy dot load method loads a model package by name and returns an NLP object. The package provides the binary weights that enable spaCy to make predictions. It also includes the vocabulary and meta information to tell spaCy which language class to use and how to configure the processing pipeline. Let's take a look at the model's predictions. In this example, we're using spaCy to predict part-of-speech tags, the word types in context. First, we load the small English model and receive an NLP object. Next, we're processing the text "She ate the pizza". For each token in the Doc, we can print the text and the "pos underscore" attribute, the predicted part-of-speech tag. In spaCy, attributes that return strings usually end with an underscore – attributes without the underscore return an ID. Here, the model correctly predicted "ate" as a verb and "pizza" as a noun. In addition to the part-of-speech tags, we can also predict how the words are relate