Data and spreadsheets
Plan a safe review of duplicate rows
Two rows with the same name can represent different people, orders or visits. Decide what makes a record unique before asking for duplicates to be removed. Start with a copy of the file and a few anonymous examples that show the awkward cases.
A text to start with
This table records [one row represents what]. Its columns are [column names and meanings]. Here are anonymous examples, including records that look similar: [rows]. Propose a duplicate-review rule. Explain which fields must match, which differences are acceptable and which pairs need a person to decide. Produce a review list only. Do not delete or merge records.
Example request. Change the details to fit your situation.Duplicate-review rule and decision log
One row represents: [ ] Original file retained at: [ ] Fields required to match: [ ] Differences that may be ignored, with a reason: [ ] Cases requiring manual review: [ ] First row reference: [ ] Second row reference: [ ] Matching evidence: [ ] Important difference: [ ] Decision and reviewer: [ ] Information to preserve: [ ]
Test cases that should remain separate
Include a repeated customer with two different orders, a blank identifier and a corrected spelling if those cases occur in your data. A matching rule should explain how it treats each one. Check whether spaces, capitalization or formatting differences are meaningful in the source system before normalizing them. A shared email address may identify a household rather than one person. Keep the original row references in the review list so you can inspect both records later. If the rule cannot distinguish two legitimate records, it is too broad for automatic deletion.
Ask another agent to check the result
Try to find a legitimate pair of records that this rule would incorrectly merge. Check repeated transactions, blank identifiers and shared contact details where applicable. Explain each failure using the sample rows. Then identify duplicates the rule would miss. Do not propose deleting rows until the ambiguous cases have an explicit decision.
Review a small batch first
Apply the proposed rule to a small copy and compare its matches with your own decisions. Revise the rule when it groups records that should remain separate. Decide who can approve a merge and what information must survive it. After any authorized change, compare record counts and totals with the original, and keep a change log that explains the difference.