Eliminating duplicate records

In many cases, you will only want to retain one record that results from a join. Let's say we only want to keep one record for each memberid, and we want it to be the highest TotalPurchase for each member. To eliminate duplicates, we can first sort the data frame by memberid and TotalPurchase amount. Then we will use the rev and duplicated functions to keep only the last purchase record for each memberid. This will leave us with one record for each of the original 1,000 members:

join3 <- join3[order(join3$memberid, join3$TotalAmount),] View(join3) dedup <- join3[!rev(duplicated(rev(join3$memberid))),] nrow(join3) nrow(dedup) > nrow(join3) [1] 1105 > nrow(dedup) [1] 1000 

Get Practical Predictive Analytics now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.