How to Remove Duplicate Rows in a CSV or Excel File

Duplicate rows creep in when files are merged, exported twice or copied by hand. Removing them takes seconds — but in financial data, two identical rows are sometimes two real transactions. This guide shows the quick methods, and how to tell which repeats to keep.

Before you delete anything

Decide what “duplicate” means for your data:

• Exact duplicates — every column the same. In a contact list or a product list, these are almost always mistakes.
• Repeated transactions — in bank data, the same date, payee and amount can be two genuine purchases. If the file has a balance column, the balance differs between them, so they aren’t exact duplicates at all.
• Overlap from merged files — two downloads that cover the same days. Here the repeats are mistakes, but only the ones that come from different files.

Always keep a copy of the original file before removing anything.

Method 1: Excel’s Remove Duplicates

1. Click anywhere in the data and choose Data → Remove Duplicates.
2. Tick My data has headers.
3. Choose the columns that must all match for a row to count as a duplicate. Leave every column ticked to remove only exact duplicates.
4. Click OK. Excel tells you how many rows it removed and how many unique values remain.

Remove Duplicates keeps the first row of each group and deletes the rest, with no list of what it removed. If you need to see the deleted rows, flag them first (method 2).

Method 2: flag duplicates with a formula

To see duplicates before deleting them, add a helper column. With data in columns A to D starting in row 2, enter this in E2 and fill it down:

=IF(COUNTIFS($A$2:A2,A2,$B$2:B2,B2,$C$2:C2,C2,$D$2:D2,D2)>1,"duplicate","")

Each row is compared only with the rows above it, so the first occurrence stays blank and later copies say duplicate. Filter column E to review them, then delete the flagged rows. In Excel 365 you can also produce a de-duplicated copy in one step with =UNIQUE(A2:D500).

Method 3: Google Sheets

Select the data and choose Data → Data cleanup → Remove duplicates, tick Data has header row, choose the columns to compare and confirm. The COUNTIFS formula above works in Google Sheets too.

Method 4: a free tool, without opening Excel

Combine CSV and Excel files removes duplicates from one file or several, in your browser — the file isn’t uploaded. Add one file and it removes repeated rows, keeping the first of each. Add several and it can remove only the overlap between files, keeping repeats that appear within a single file. It compares rows after tidying them, so capitals, extra spaces, 2,000.00 and 2000, and a date written as text or stored as a date all count as the same.

The Excel download lists every removed row and which row it duplicated, so you can check before you rely on the result.

Duplicates that aren’t exactly the same

Rows that differ only in spacing, capitals or number formatting won’t be caught by Excel’s Remove Duplicates. Tidy them first: =TRIM(LOWER(A2)) for text, and convert numbers stored as text with Data → Text to Columns → Finish. For near-duplicates — the same transaction with a slightly different description from two sources — compare on date and amount only, and review the matches by hand.

Duplicates already in QuickBooks or Xero

Removing duplicates from a file before you import it is easy. Once they’re in your books, they have to be excluded or undone transaction by transaction — see how to fix duplicate bank transactions.