Hands-On Machine Learning for Algorithmic Trading

Parsing HTML using regular expressions

To collect structured data from the unstructured transcripts, we can use regular expressions in addition to BeautifulSoup.

They allows us to collect detailed information not only about the earnings call company and timing but also capture who was present and attribute the statements to analysts and company representatives:

def parse_html(html):    date_pattern = re.compile(r'(\d{2})-(\d{2})-(\d{2})')    quarter_pattern = re.compile(r'(\bQ\d\b)')    soup = BeautifulSoup(html, 'lxml')    meta, participants, content = {}, [], []    h1 = soup.find('h1', itemprop='headline').text    meta['company'] = h1[:h1.find('(')].strip()    meta['symbol'] = h1[h1.find('(') + 1:h1.find(')')]    title = soup.find('div', class_='title').text