Regular Expressions Cookbook, 2nd Edition

Here Documents

Problem

You need a regex that matches here documents in source files for a scripting language in which a here document can be started with << followed by a word. The word may have single or double quotes around it. The here document ends when that word appears at the very start of a line, without any quotes, using the same case.

Solution

<<(["']?)([A-Za-z]+)\b\1.*?^\2\b

Regex options: Dot matches line breaks, ^ and $ match at line breaks

Regex flavors: .NET, Java, XRegExp, PCRE, Perl, Python, Ruby

<<(["']?)([A-Za-z]+)\b\1[\s\S]*?^\2\b

Regex options: ^ and $ match at line breaks

Regex flavors: .NET, Java, JavaScript, PCRE, Perl, Python, Ruby

Discussion

This regex may look a bit cryptic, but it is very straightforward. ‹<<› simply matches <<. ‹(["']?)›, then matches an optional single or double quote. The parentheses form a capturing group to store the quote, or the lack thereof. It is important that the quantifier ‹?› is inside the group rather than outside of it, so that the group always participates in the match. If we made the group itself optional, the group would not participate in the match when no quote can be matched, and a backreference to that group would fail to match.

The capturing group with character class ‹([A-Za-z]+)› matches a word and stores it into the second backreference. The word boundary ‹\b› makes sure we match the entire word after ‹<<›. If we were to omit the word boundary, the regex engine would backtrack. It would try to match the word partially if ...