Skip to Content
Microsoft® SQL Server® 2008 Integration Services: Problem-Design-Solution
book

Microsoft® SQL Server® 2008 Integration Services: Problem-Design-Solution

by Erik Veerman, Jessica M. Moss, Brian Knight, Jay Hackney
November 2009
Beginner to intermediate
476 pages
13h 20m
English
Wrox
Content preview from Microsoft® SQL Server® 2008 Integration Services: Problem-Design-Solution

6

Data-Cleansing Design

At some point in your career, you’ve probably been told that a given file should always be clean, and there’s no reason to spend time working on procedures to protect yourself from a problem. Inevitably, the impossible happens, and that previously perfect file has a problem, causing you to wake up at two o’clock in the morning to correct the production problem. If this has ever happened to you, or if you’re trying to not fall victim to this scenario, then this chapter is for you.

In the classic Problem-Design-Solution pattern, this chapter teaches you how to protect your package from breakages from future data problems, and provides new patterns for exploring every realm of data cleansing prior to loading or updating the data. Here’s what you can expect in this chapter:

  • “Problem” — The “Problem” section answers the question, “How clean is my data?” The discussion shows how to determine the answer to this question by employing the built-in functionality of the Data Profiling Task.
  • “Design” — After the file or data set is in the pipeline, you may need to protect yourself from unexpected conditions prior to loading the data. In the “Design” section of this chapter, you learn about the methods for using the Script transform to either cleanse or direct bad data elsewhere. You also see how to use the Fuzzy Grouping and Lookup transforms to match the data on non-exact means. Finally, you find out how to cleanse a single file that has two different types of data ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.

Read now

Unlock full access

More than 5,000 organizations count on O’Reilly

AirBnbBlueOriginElectronic ArtsHomeDepotNasdaqRakutenTata Consultancy Services

QuotationMarkO’Reilly covers everything we've got, with content to help us build a world-class technology community, upgrade the capabilities and competencies of our teams, and improve overall team performance as well as their engagement.
Julian F.
Head of Cybersecurity
QuotationMarkI wanted to learn C and C++, but it didn't click for me until I picked up an O'Reilly book. When I went on the O’Reilly platform, I was astonished to find all the books there, plus live events and sandboxes so you could play around with the technology.
Addison B.
Field Engineer
QuotationMarkI’ve been on the O’Reilly platform for more than eight years. I use a couple of learning platforms, but I'm on O'Reilly more than anybody else. When you're there, you start learning. I'm never disappointed.
Amir M.
Data Platform Tech Lead
QuotationMarkI'm always learning. So when I got on to O'Reilly, I was like a kid in a candy store. There are playlists. There are answers. There's on-demand training. It's worth its weight in gold, in terms of what it allows me to do.
Mark W.
Embedded Software Engineer

You might also like

Microsoft® SQL Server® 2012 Integration Services

Microsoft® SQL Server® 2012 Integration Services

Rakesh Parida Wee-Hyong Tok Matt Masson, Xiaoning Ding, and Kaarthik Sivashanmugam
Professional SQL Server™ 2005 Integration Services

Professional SQL Server™ 2005 Integration Services

Brian Knight, Allan Mitchell, Darren Green, Douglas Hinson, Kathi Kellenberger, Andy Leonard, Erik Veerman, Jason Gerard, Haidong Ji, Mike Murphy

Publisher Resources

ISBN: 9780470525760Publisher Website