Skip to Content
bash Cookbook
book

bash Cookbook

by Carl Albing, JP Vossen, Cameron Newham
May 2007
Beginner
628 pages
15h 46m
English
O'Reilly Media, Inc.
Content preview from bash Cookbook

Removing Smart Quotes

Problem

You want simple ASCII text out of a document in MS Word, but when you save it as text some odd characters still remain.

Solution

Translate the odd characters back to simple ASCII like this:

$ tr '\221\222\223\224\226\227' '\047\047""--' <odd.txt >plain.txt

Discussion

Such “smart quotes” come from the Windows-1252 character set, and may also show up in email messages that you save as text. To quote from Wikipedia on this subject:

A few mail clients send curved quotes using the Windows-1252 codes but mark the text as ISO-8859-1 causing problems for decoders that do not make the dubious assumption that C1 control codes in ISO-8859-1 text were meant to be Windows-1252 printable characters.

To clean up such text, we can use the tr command. The 221 and 222 (octal) curved single-quotes will be translated to simple single quotes. We specify them in octal (047) to make it easier on us, since the shell uses single quotes as a delimiter. The 223 and 224 (octal) are opening and closing curved quotes, and will be translated to simple double quotes. The double quotes can be typed within the second argument since the single quotes protect them from shell interpretation. The 226 and 227 (octal) are dash characters and will be translated to hyphens (and no, that second hyphen in the second argument is not technically needed, since tr will repeat the last character to match the length of the first argument, but it’s better to be specific).

Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

AirBnbBlueOriginElectronic ArtsHomeDepotNasdaqRakutenTata Consultancy Services

QuotationMarkO’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.
Julian F.
Head of Cybersecurity
QuotationMarkI wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.
Addison B.
Field Engineer
QuotationMarkI’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.
Amir M.
Data Platform Tech Lead
QuotationMarkI'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.
Mark W.
Embedded Software Engineer

You might also like

bash Cookbook, 2nd Edition

bash Cookbook, 2nd Edition

Carl Albing, JP Vossen
Linux Shell Scripting Cookbook - Third Edition

Linux Shell Scripting Cookbook - Third Edition

Clif Flynt, Sarath Lakshman, Shantanu Tushar

Publisher Resources

ISBN: 0596526784Errata Page