O'Reilly logo

Practical Predictive Analytics by Ralph Winters

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Eliminating duplicate records

In many cases, you will only want to retain one record that results from a join. Let's say we only want to keep one record for each memberid, and we want it to be the highest TotalPurchase for each member. To eliminate duplicates, we can first sort the data frame by memberid and TotalPurchase amount. Then we will use the rev and duplicated functions to keep only the last purchase record for each memberid. This will leave us with one record for each of the original 1,000 members:

join3 <- join3[order(join3$memberid, join3$TotalAmount),] View(join3) dedup <- join3[!rev(duplicated(rev(join3$memberid))),] nrow(join3) nrow(dedup) > nrow(join3) [1] 1105 > nrow(dedup) [1] 1000 

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required