Features for Textual Data

Abstract

In the previous chapters we discussed the general learning problem, and saw some machine learning models and algorithms for training them. All of these models take as input vectors x and produce predictions. Up until now we assumed the vectors x are given. In language processing, the vectors x are derived from textual data, in order to reflect various linguistic properties of the text. The mapping from textual data to real valued vectors is called feature extraction or feature representation, and is done by a feature function. Deciding on the right features is an integral part of a successful machine learning project. While deep neural networks alleviate a lot of the need in feature engineering, a good set of core features still needs to be defined. This is especially true for language data, which comes in the form of a sequence of discrete symbols. This sequence needs to be converted somehow to a numerical vector, in a non-obvious way.