Getting the data
The data we are going to use is a set of blog posts that are labeled for age, gender, industry (that is, work) and, funnily enough, star sign. This data was collected from http://blogger.com in August 2004 and has over 140 million words in more than 600,000 posts. Each blog is probably written by just one person, with some work put into verifying this (although, we can never be really sure). Posts are also matched with the date of posting, making this a very rich dataset.
To get the data, go to http://u.cs.biu.ac.il/~koppel/BlogCorpus.htm and click on Download Corpus. From there, unzip the file to a directory on your computer.
The dataset is organized with a single blog to a file, with the filename giving the classes. For ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access