Python Social Media Analytics

Data type and encoding

Comments and conversation are textual data that we retrieve as strings. In brief, a string is a sequence of characters represented by code points. Every string in Python is seen as a Unicode covering the numbers from 0 through 0x10FFFF (1,114,111 decimal). Then, the sequence has to be represented as a set of bytes (values from 0 to 255) in memory. The rules for translating a Unicode string into a sequence of bytes are called encoding.

Encoding plays a very important role in natural language processing, because people use more and more characters such as emojis or emoticons, which replace whole words and express emotions . Moreover, in many languages there are accents that go beyond the regular English alphabet. In order ...