Java: Data Science Made Easy

Simple text cleaning

We will use the string shown before from Moby Dick to demonstrate some of the basic String class methods. Notice the use of the toLowerCase and trim methods. Datasets often have non-standard casing and extra leading or trailing spaces. These methods ensure uniformity of our dataset. We also use the replaceAll method twice. In the first instance, we use a regular expression to replace all numbers and anything that is not a word or whitespace character with a single space. The second instance replaces all back-to-back whitespace characters with a single space:

out.println(dirtyText); dirtyText =    dirtyText.toLowerCase().replaceAll("[\\d[^\\w\\s]]+", " "); dirtyText = dirtyText.trim(); while(dirtyText.contains("  ")){  dirtyText ...