Python Feature Engineering Cookbook

Extracting Features from Text Variables

Text can be part of the variables in our datasets. For example, in insurance, some variables that capture information about an incident may come from a free text field in a form. In data from a website that collects customer reviews or feedback, we may also encounter variables that contain short descriptions provided by text that has been entered manually by the users. Text is unstructured, that is, it does not follow a pattern, like the tabular pattern of the datasets we have worked with throughout this book. Text may also vary in length and content, and the writing style may be different. How can we extract information from text variables to inform our predictive models? This is the question we are ...